[Image description: photo of baby in diapers playing with a laptop. He looks geeky and intelligent with his large black glasses.]
The Radio NZ Insight program this week by Teresa Cowie annoyed me no end. It was about the Predictive Risk Model (PRM) developed by Professor Rhema Vaithianathan and her team. The model uses government data to calculate the probability of a child being abused by its carers.
I concluded Teresa and the people selected for interview did not understand statistic modelling, misframed the debate and cherry-picked data. Teresa also asked leading questions.
Don’t mistake me. Everyone was clearly concerned about child abuse in New Zealand but we ended up with half an hour of radio which bashed “the model”, like shooting the messenger. Overall it seemed a sad indictment of the paucity of knowledge of statistics in New Zealand and how to use them for policy purposes. It also struck me as an exercise in group think by progressives as portrayed by this John Palethorpe blog. [edit:Nov2016 – John has made his website private and you’ll need his permission to view it.]
And also don’t mistake me. I’m fully aware that the horrific status of Maori kids in Aotearoa is a direct result of colonialism and the deliberate attempt by colonisers to destroy Maori culture and society. The debate needs to take this into account.
The debate should be about how to bring down the horrific levels of child abuse in New Zealand. Instead the debate was framed as “profiling” (via PRM demonised as invasion of privacy and an attack on human rights) versus the rights of parents not to be profiled. The rights of the child not to be bashed got small mention. The editorialising Tersa Cowie said beneficiaries at WINZ “hand over information about themselves in exchange for social services”.
So what went wrong?
Firstly I don’t think anyone had focused on this introduction to the PRM. The intro shows that of all children abused by age 5, 83% are seen on a benefit before age 2. That’s astounding. That’s a horrible fact. That wasn’t mentioned in the program.
Teresa interviewed postmodernist* social worker academic Emily Keddell from Otago University. You can read her paper here in which she argues about the accuracy of the model and I think misunderstands the statistical tools. I’m not sure she understands ROC curve analysis and her rhetoric is ill-chosen. Take this example from her paper: “I think if you got on a plane and were told that there was only a 48% chance of reaching your destination, or told that the TV guide you were looking at was only 37% accurate on the programmes that were screening that night, calling this ‘accurate’ would be a stretch.” Now let me reframe. If you were told there was a 48% chance of you dying of cancer within a year, would you accept treatment? 37% chance of your house burning down and the premium was $1 week …? Or how about 83% chance of a child being bashed unless you agreed to pay a few more cents a year in your taxes on a Nurse Family Partnership program?
Emily Keddell also says in her paper that the model uses data that doesn’t include the whole population, that it’s mostly data from poor people who come into more contact with government departments. That may be so and it may also be misused by neo-liberals, but to throw it out is clearly to ignore a huge part of the population that is at risk.
Teresa also interviewed Starship Hospital pediatrician and clinical director of child abuse team Dr Patrick Kelly who cherry picked data and marred his interview with his phone repeatedly ringing. He claimed the accuracy of the model wasn’t good enough. He said the model assumes people will abuse their kids. Wrong Patrick. The model reports a probability. What we do with that information is another thing. I thought a better message would be to emphasise the support our society needs to give to all new parents and the extra support we need to give when professionals in a Nurse Family Partnership program see alarm bells ringing. The model might be only one part of an alarm bell system. Instead we got hyperbole about the dangers of stigmatising beneficiaries.
As Plunket said In their submission to Health Select Committee, Inquiry into preventing child abuse and improving children’s health:
“Home visiting is evidence based, and an effective mode of care delivery to improve access to service for at risk, vulnerable families (Olds, 2007). Programmes that employ home visiting and outreach to deliver both universal, and especially, targeted interventions have considerable potential; they are acceptable to families and improve access. We advocate for ongoing and increased investment to improve access to and the quality of existing home visiting programmes. International evidence from studies of early childhood programmes suggests that early engagement and intensive home visiting programmes can improve outcomes for the children and families identified as most vulnerable (The Nurse Family Partnership (Olds, 2007) is one good example of such a model). As a provider with a large and well qualified nursing workforce Plunket is acutely aware of the range of skills held by nurses. We suggest well qualified and supported nurses are integral to the community child wellbeing workforce.”
Kay Brereton of the Beneficiary Advocacy Federation was interviewed. I can’t find them on the web. She said in relation to beneficiaries that the PRM could result in “a whole lot of more stress in their lives”. I know that WINZ and some social workers have a lot to answer for, but dissing the model on the basis that targeted help might be unwanted seems silly to me. Despite being pushed to diss the model, one of the beneficiaries interviewed actually said he wouldn’t mind his caseworker having access to data.
Teresa would counter that the program was about PRM, not about what to do about child abuse. My simple answer is that the Predictive Risk Model (PRM) is merely a way of gathering data. To diss the model is like dissing your weighing scales because you don’t like them telling you your weight.
The answer to child abuse stats in NZ is simple: 1. give people a Universal Basic Income 2. implement a decent Nurse Family Partnership program. 3. Treat drug and alcohol dependency as a medical not criminal issue. It’s not rocket science, but may be aided by a good understanding of statistical science.
*Postmodernism is the silly belief that there can be no objective truth and that all opinions are valid. Pseudo-academics who don’t understand science spout postmodernism. It was nicely blown out of the water by Alan Sokal in this little sting.
Kevin McCready 
Hi Kevin. You’re quite right that the application of stats into policy is a key issue in this debate. As I’ve argued elsewere, the PRM could be useful at a population level, because the nature of statistical likelihood is just that: an indicator of likelihood across a population, not individually deterministic…and that’s my beef with PRM. It attempts to use a statistical likelihood across a population to infer something about specific individuals. Oh and I do understand the ROC curve…but as you point out, many people don’t. To quote myself: “using risk factor science (alone) to construct risk reifies risk itself in practice contexts, even though developers and policy makers alike may understand that risk factor analysis yields a broad-brush picture, rather than absolute determinants. Risk factor analysis models, while not claiming to be deterministic, nevertheless are often used as if they can give surety about future behaviour”. Could this be ameliorated with proper implementation? Perhaps. Perhaps not. Interestingly, the Olds Nurse-Family partnership programme uses a very simple three- factor criteria for access to services based on age, low income, and first time mothers. We certainly need more preventive targetted services. But do we need a PRM to target them?
LikeLike
As human beings are not statistics there is no place for using (incomplete) predictive stats in human social policy making that assumes no change and does not account for variables. Imagine if you were found just by a collection of other peoples past data collection( your income, race,past and age) to be a criminal- its just “minority report” stuff!
The fact is that human behaviour is not accurately predicted for an individual. This is because the stats don’t work on people as INDIVIDUALS as you can’t use other peoples past data to predict another individuals future behaviour . All good psychologist know this fact and that is why they don’t treat people as non changing past statistics of others .
Note that even when new parents are living in poverty the govt does not assist them or their children by removing any of the stressors( survival) for new parents. The way they are treated by the ministry when they ask for needed financial assistance is with neglectful disrespect. The ministry should target its own behaviour first as cutting these new parents off and threatening their family’s very survival is a form of abuse. So the Crown agency that delivers or denies assistance should be examined as it is part of the problem. The welfare culture of denial and demonizing poor young parents is a factor as it can set up and maintain a false negative self identity for these parents.
LikeLike
To solve a problem you must first understand it.
Why do people abuse children?
I would say it is ignorance of who they are. Instead of freeing themselves from a conditioned belief structure, a damaged and lacking psyche with the same thoughts of being the victim, living feeling isolated with a fake negative identity embedded with of the past trauma( their own abuse). Being abused as a child does not mean you will abuse your child.
As you are dealing with the mind there is nothing that can be identified as a risk factor( trigger) for an individual by using other peoples triggers life . For some one negative word, a foul look a drink is enough for some to elicit loss of Self, turning the person into an automation, a repeat, a take over of past negative emotions.
Living alone in poverty with a feeling of stress for just your survival can make people fearful and desperate and so shut down higher thinking capabilities.
Poverty has been cultivated in NZ by the Crown’s pyramid structure of control, having a growing number of poor angry fearful people which are controlled by providing them with a small weekly wage/benefit most of which goes to the Crown’s utility corporations or retail food monopoly .
There are plenty of middle class child abusers( I’ve known rich ones) isn’t it interesting how these people where exempt here and this just enforces the govt created identity for beneficiaries as child abusers. How much does the govt cause the very problems it pretends to want to solve ( but really just wants to breach privacy law have more control an “ownership of children”).
Love heals, mindfulness works but there is little access to healing therapies .
What these people get from the Crown (and its entities) in cases of child abuse is anything but what is needed.
LikeLike
Where does the Observer effect fit into this model?
How can we ever think we can target individual’s behavior that has not even happened?
Are we robots or human beings?
To solve the problem of violence towards children we need to understand the problem completely. (That means giving it attention- as you are doing).
Violence in a society that has been told:
“if you are Maori you will be more likely to beat up your wife and kids”.{The Crown’s created Maori ‘victim’ identity always kept warm}
“We have illegally invaded both Afghanistan and now Syria, escalating the war, violence and chaos”.
“Drink up”.(” responsibly”= responsible to drink)
Rugby.
Wouldn’t it be better to have a patriarchal society that does not value violence as a “manly Jack” quality.
Culture/belief system is eliminated when we talk about violence, as is the level of escalating world violence that the politicians call “valid”, the media has ‘normalized’ violence over generations.
People believe that there is a real thing called ” humanitarian bombing”!!!!
NZ Police have been widely known to brutalize people and they never get told it was wrong for their violent pro rape culture they get given tasers!!
The banksters head PM John Key is called the silent assassin. He is seen by these “statistics” ( or ‘people’ )to be stealing public assets and giving them to his Rothschild master’s corporations.
LikeLike
Govt data on child abuse is incomplete, unscientific and completely bias. I believe these media stories serve the purpose to disenfranchise the poor. its more verbal violence against the poor.
Look at the Whitehall pedophile ring, all wealthy employed high income earners.This is said to be global, but of course the evidence is being destroyed by the police and “investigators”.
Most of the child abuse of wealthy and upper class children is covered up .
LikeLike
Hi Kevin,
This is the second bullet point in the Summary of Findings from the report: “A maltreatment finding is defined as a substantiated finding of emotional, physical or sexual abuse or neglect by age 5.” I think this criterion may be behind much of the concern.
‘Substantiated findings’ of abuse or neglect are operationalised, in effect, by the criterion of coming under notice of the authorities. It’s understandable as a criterion from a methodological perspective (‘some’ measure of the phenomenon is necessary if a predictive model is to be possible at all). But that doesn’t make it a ‘good’ – or even the ‘best possible’ – measure.
It is not unreasonable to suggest that children from families on benefits – or which have been on benefits – may well be more likely to come to such notice by the authorities than those not on benefits, or in middle class families, etc.. This is because by virtue of being on a benefit they are already being ‘noticed’ by the authorities and, additionally, they may be less likely to have the social, cultural and economic capital to escape notice by such authorities. The 83% figure may therefore be misleading – largely ‘low hanging fruit’ (which is not to say it shouldn’t be ‘picked’).
Do we have any reason to think that abuse and neglect of children is being fully reported at present across society? Or under-reported? Is some neglect and abuse better able to escape ‘notice’ by the authorities? (Especially when such abuse includes emotional and sexual abuse and general neglect.)
Given that some prevalent discourses portray social ills as primarily being confined to specific groups (e.g., beneficiaries) there’s a legitimate concern over operationalising this particular ‘social ill’ in this way. More importantly, it may mean that resources are not necessarily being directed at all cases of greatest need.
It could be a bit like looking for your car keys under the street light.
LikeLike
Agreed Kevin Moore. I addressed that by saying “a better message would be to emphasise the support our society needs to give to all new parents and the extra support we need to give when professionals in a Nurse Family Partnership program see alarm bells ringing. ” We need to tackle the abuse that happens in rich homes too. Part of that is a cultural change to speak up whenever we suspect abuse.
LikeLike