The Radio NZ Insight program this week by Teresa Cowie annoyed me no end. It was about the Predictive Risk Model (PRM) developed by Professor Rhema Vaithianathan and her team. The model uses government data to calculate the probability of a child being abused by its carers.
I concluded Teresa and the people selected for interview did not understand statistic modelling, misframed the debate and cherry-picked data. Teresa also asked leading questions.
Don’t mistake me. Everyone was clearly concerned about child abuse in New Zealand but we ended up with half an hour of radio which bashed “the model”, like shooting the messenger. Overall it seemed a sad indictment of the paucity of knowledge of statistics in New Zealand and how to use them for policy purposes. It also struck me as an exercise in group think by progressives as portrayed by this John Palethorpe blog. [edit:Nov2016 – John has made his website private and you’ll need his permission to view it.]
The debate should be about how to bring down the horrific levels of child abuse in New Zealand. Instead the debate was framed as “profiling” (via PRM demonised as invasion of privacy and an attack on human rights) versus the rights of parents not to be profiled. The rights of the child not to be bashed got small mention. The editorialising Tersa Cowie said beneficiaries at WINZ “hand over information about themselves in exchange for social services”.
So what went wrong?
Firstly I don’t think anyone had focussed on this introduction to the PRM. The intro shows that of all children abused by age 5, 83% are seen on a benefit before age 2. That’s astounding. That’s a horrible fact. That wasn’t mentioned in the program.
Teresa interviewed social worker academic Emily Keddell from Otago University. You can read her paper here in which she argues about the accuracy of the model and I think misunderstands the statistical tools. I’m not sure she understands ROC curve analysis and her rhetoric is ill-chosen. Take this example from her paper: “I think if you got on a plane and were told that there was only a 48% chance of reaching your destination, or told that the TV guide you were looking at was only 37% accurate on the programmes that were screening that night, calling this ‘accurate’ would be a stretch.” Now let me reframe. If you were told there was a 48% chance of you dying of cancer within a year, would you accept treatment? 37% chance of your house burning down and the premium was $1 week …? Or how about 83% chance of a child being bashed unless you agreed to pay a few more cents a year in your taxes on a Nurse Family Partnership program?
Emily Keddell also says in her paper that the model uses data that doesn’t include the whole population, that it’s mostly data from poor people who come into more contact with government departments. That may be so and it may also be misused by neo-liberals, but to throw it out is clearly to ignore a huge part of the population that is at risk.
Teresa also interviewed Starship Hospital pediatrician and clinical director of child abuse team Dr Patrick Kelly who cherry picked data and marred his interview with his phone repeatedly ringing. He claimed the accuracy of the model wasn’t good enough. He said the model assumes people will abuse their kids. Wrong Patrick. The model reports a probability. What we do with that information is another thing. I thought a better message would be to emphasise the support our society needs to give to all new parents and the extra support we need to give when professionals in a Nurse Family Partnership program see alarm bells ringing. The model might be only one part of an alarm bell system. Instead we got hyperbole about the dangers of stigmatising beneficiaries.
As Plunket said In their submission to Health Select Committee, Inquiry into preventing child abuse and improving children’s health:
“Home visiting is evidence based, and an effective mode of care delivery to improve access to service for at risk, vulnerable families (Olds, 2007). Programmes that employ home visiting and outreach to deliver both universal, and especially, targeted interventions have considerable potential; they are acceptable to families and improve access. We advocate for ongoing and increased investment to improve access to and the quality of existing home visiting programmes. International evidence from studies of early childhood programmes suggests that early engagement and intensive home visiting programmes can improve outcomes for the children and families identified as most vulnerable (The Nurse Family Partnership (Olds, 2007) is one good example of such a model). As a provider with a large and well qualified nursing workforce Plunket is acutely aware of the range of skills held by nurses. We suggest well qualified and supported nurses are integral to the community child wellbeing workforce.”
Kay Brereton of the Beneficiary Advocacy Federation was interviewed. I can’t find them on the web. She said in relation to beneficiaries that the PRM could result in “a whole lot of more stress in their lives”. I know that WINZ and some social workers have a lot to answer for, but dissing the model on the basis that targeted help might be unwanted seems silly to me. Despite being pushed to diss the model, one of the beneficiaries interviewed actually said he wouldn’t mind his caseworker having access to data.
Teresa would counter that the program was about PRM, not about what to do about child abuse. My simple answer is that the Predictive Risk Model (PRM) is merely a way of gathering data. To diss the model is like dissing your weighing scales because you don’t like them telling you your weight.
The answer to child abuse stats in NZ is simple: 1. give people a Universal Basic Income 2. implement a decent Nurse Family Partnership program. 3. Treat drug and alcohol dependency as a medical not criminal issue. It’s not rocket science, but may be aided by a good understanding of statistical science.