The Score You Can't See: How Unauthorized Risk Algorithms Decide Which Poor Families Get Investigated — and Why HHS Just Put $6 Million Behind Spreading Them
The Score You Can't See: How Unauthorized Risk Algorithms Decide Which Poor Families Get Investigated — and Why HHS Just Put $6 Million Behind Spreading Them
In August 2016, Allegheny County, Pennsylvania began running every family reported to its child-abuse hotline through a statistical model that scores them from 1 to 20. The score is not built from evidence of abuse. It is built from Medicaid claims, Supplemental Security Income records, behaviora...
I have enough to write. Here is the full investigation.
The Score You Can't See: How Unauthorized Risk Algorithms Decide Which Poor Families Get Investigated — and Why HHS Just Put $6 Million Behind Spreading Them
In August 2016, Allegheny County, Pennsylvania began running every family reported to its child-abuse hotline through a statistical model that scores them from 1 to 20. The score is not built from evidence of abuse. It is built from Medicaid claims, Supplemental Security Income records, behavioral-health diagnoses, drug and alcohol treatment histories, jail and juvenile-probation records, and prior contact with the welfare system — and it does not predict maltreatment at all. It predicts the probability that the county will place the child in foster care within two years. Ten years later, no Pennsylvania statute authorizes the tool, no parent has a right to see or contest their score, no court in a dependency proceeding is told the score exists, the U.S. Justice Department's Civil Rights Division has an open disability-discrimination inquiry that has produced no public findings, and a peer-reviewed study published on June 17, 2026 found that the tool "effectively encodes disability status through proxy variables" despite containing no explicit disability measure. In June 2026, the Department of Health and Human Services announced $6 million in grants to help states build more of them.
How the Machine Actually Works
The central deception in this entire field is the outcome variable. Agencies say "risk of harm." The models say something else.
The Allegheny Family Screening Tool (AFST) was designed to predict the risk that a child will be placed in foster care in the two years after a family is investigated. Douglas County, Colorado's tool — the Douglas County Decision Aid — predicts a child's likelihood of being removed from home by CPS within two years of a referral. Allegheny's separate "Hello Baby" model, run on newborns county-wide, is designed to identify infants with the greatest risk of removal from the home by their third birthday.
This substitution matters enormously. Child maltreatment is largely unobserved; foster-care placement is a decision made by the agency itself. A model trained on placement is not learning what harms children. It is learning what the agency has historically done — and then recommending the agency do more of it. As the ACLU's 2021 report Family Surveillance by Algorithm put it, "any tool built from a jurisdiction's historical data runs the risk of perpetuating and exacerbating, rather than ameliorating, the biases that are embedded in that data." Most tools in use, the ACLU found, are designed to predict removal — not abuse.
The inputs make the loop tighter. AFST draws on Medicaid, substance-abuse, mental-health, jail and probation records, birth records, and public-benefits history. It has at times drawn on Supplemental Security Income data and diagnoses for mental, behavioral, and neurodevelopmental disorders including schizophrenia and mood disorders. Every one of those is a poverty proxy, a disability proxy, or both — because the only people whose mental-health records the county holds are the people poor enough to be on public insurance. A middle-class parent in therapy paid for by an employer plan is invisible to the model. A poor parent in therapy paid for by Medicaid is a data point.
The Human Rights Data Analysis Group and ACLU, in a 2023 paper presented at the ACM Conference on Fairness, Accountability, and Transparency, traced this to the feature level: 27 percent of referrals involving households with at least one Black member were affected by the tool's "ever involved in juvenile probation" predictor, versus 9 percent of non-Black households. That is not a model discovering risk. That is a model rediscovering the carceral history of a neighborhood and calling it parenting.
New York City runs the least accountable version. In May 2025, The Markup revealed that the Administration for Children's Services has, since 2018, secretly used a risk model with 279 variables — including neighborhood, mother's age, number of siblings, and mental-health history — to flag families for heightened scrutiny by its Emergency Response Office. Families are never told when the algorithm flags them. Neither are their lawyers. Neither, in many instances, are the caseworkers. ACS's own internal audit conceded the system contains "implicit and systemic biases" but concluded it was still more accurate than what came before. In a city where Black families are reported to child services at roughly seven times the rate of white families and are about thirteen times more likely to have a child removed, a 279-variable secret score is not a decision aid. It is a shadow docket.
The Money: A Federal Subsidy Nobody Voted For
There is no line item in federal law called "predictive analytics." There doesn't need to be. The tools are paid for through two ordinary reimbursement channels, and that is precisely what makes them invisible.
Title IV-E administrative match. Section 474 of the Social Security Act reimburses states at roughly 50 percent for the administrative costs of operating their child welfare programs. Risk-scoring is administration. Building it, running it, and paying analysts to maintain it are administrative costs. No one has to ask Congress.
CCWIS. The Comprehensive Child Welfare Information System rule (45 C.F.R. Part 1355, effective 2016) offers states a 50 percent federal match on both the development and the ongoing operation of a modern child-welfare data system — versus a far lower rate for legacy systems. Few states can absorb the cost of a $100-million-plus data system without it. CCWIS is the plumbing: it is what integrates Medicaid, behavioral health, courts, and benefits data into a single queryable spine. Once that spine exists — federally subsidized, federally reviewed through Advance Planning Documents filed with the Children's Bureau — the marginal cost of bolting a predictive model onto it is trivial. The federal government pays to build the data warehouse and then averts its eyes from what gets built on top.
The result is a system where the Children's Bureau reviews and approves the architecture of surveillance through the APD process while insisting it has no role in reviewing the models. That is not oversight. That is laundering.
And now HHS has dropped the pretense of neutrality. On June 2, 2026, the Administration for Children and Families announced $6 million in competitive funding — opportunity number HHS-2026-ACF-ACYF-CA-0037, up to 10 awards of $400,000 to $600,000, applications due July 27, 2026 — explicitly to help state, territorial, and tribal agencies pilot predictive analytics. ACF Assistant Secretary Alex J. Adams framed it this way: "Child welfare caseworkers are tasked with making high-stakes decisions about child safety, often under significant pressure and without complete information... Promising use cases have been tested at county levels, and we want to support scaling these interventions at the state level to improve child welfare outcomes."
Read that again. The "promising use cases tested at county levels" are Allegheny — under open DOJ civil-rights inquiry — and Los Angeles, whose first attempt collapsed. The federal government is proposing to scale, statewide, the exact tools whose county-level track record includes an abandonment in Oregon, a cancellation in Illinois, a shutdown in Los Angeles, and an unresolved federal disability-discrimination complaint in Pennsylvania. The initiative builds on an ACF roundtable convened in December 2024 and follows a March 5, 2026 ACF brief titled Modernizing Child Welfare Technologies and Tools. It is pitched, in part, as a response to the national shortage of foster homes — that is, as a way to sort families faster, not to keep them together.
The Named Players and Their Conflicts
Two academics built most of this. Rhema Vaithianathan, a health economist at Auckland University of Technology and director of the Centre for Social Data Analytics, and Emily Putnam-Hornstein, formerly of USC and now at the University of North Carolina at Chapel Hill, where she runs work tied to the Children's Data Network. They built AFST. They built Hello Baby. Vaithianathan's team built Douglas County, Colorado's tool. The Children's Data Network led development of Los Angeles County's Risk Stratification Model. Officials in Pennsylvania, California, and Colorado opened their integrated data systems to the two developers, who then selected which data points would go into the models.
Here is the conflict, stated plainly. HHS funded a national study, co-authored by Vaithianathan and Putnam-Hornstein, that concluded their own Allegheny approach could serve as a model for other jurisdictions. The federal government paid the tool's inventors to evaluate whether the tool should spread. It spread.
The pattern repeats in the research record. The most-cited finding favorable to AFST — that it reduced unconditional racial disparities in screening rates by 31 percent and score-conditional disparities by 48 percent — comes from a paper co-authored by Putnam-Hornstein and Vaithianathan. The most damaging finding comes from independent researchers at Carnegie Mellon, who examined every referral from August 2016 to May 2018 and found that had screeners followed AFST's recommendations, the Black–white screen-in disparity would have been 20 percent; the actual disparity was 9 percent — because human call screeners overrode the algorithm, disagreeing with its scores roughly a third of the time. The humans were the de-biasing mechanism. The machine was the bias.
Allegheny County's answer to that is to cite a Stanford evaluation it commissioned finding no adverse consequences. Meanwhile HRDAG and ACLU found that screen-in rates in Allegheny did not drop after AFST's 2016 introduction — meaning the tool's central promise, that it would help the county investigate fewer families more accurately, is not visible in the county's own numbers.
Then there is the vendor side. Eckerd Connects — legally Eckerd Youth Alternatives, Inc., EIN 59-2551416, of Clearwater, Florida — marketed Rapid Safety Feedback with its for-profit partner MindShare Technology. Eckerd is not a small charity: its Form 990 filings show revenue of $351.9 million in FY2021, $312.3 million in FY2022, and $164.2 million in FY2023, with more than 3,400 employees at peak and aggregate officer compensation exceeding $3.3 million in a single year. It is also a major federal grantee, drawing HHS awards including $35.2 million (2019) and $14.6 million (2025) under CFDA 93.600.
The Documented Failures
Illinois. The Department of Children and Family Services signed a $366,000 sole-source contract with Eckerd in May 2016. The system mined DCFS files and scored children 1 to 100. It assigned more than 4,100 Illinois children a 90 percent or greater probability of death or serious injury — including 369 children under age 9 rated at 100 percent. It simultaneously failed to flag the children who actually died. DCFS Director B.J. Walker killed it in December 2017 with a sentence that should be carved above the door of every agency now applying for ACF's $6 million: predictive analytics wasn't predicting any of the bad cases. The no-bid contract had been awarded by then-DCFS Director George Sheldon, who came to Illinois from Florida's Department of Children and Families — the state where Eckerd held a roughly $73 million lead-agency contract — and who resigned in June 2017 amid controversy over steering contracts to former associates.
Los Angeles. Project AURA, built by the software firm SAS, was tested against historical cases from 2015 to 2017. It correctly identified 171 children at highest risk. It also produced 3,829 false positives. DCFS quietly abandoned it in 2017 — then returned in August 2021 with a new Risk Stratification pilot in the Belvedere, Lancaster, and Santa Fe Springs offices.
Oregon. The Safety at Screening Tool, explicitly modeled on AFST, was terminated at the end of June 2022 — weeks after the Associated Press investigation by Sally Ho and Garance Burke exposed AFST's racial patterns. Oregon reverted to Structured Decision Making, an actuarial framework whose factors are at least legible to a human being.
The evidence base for Eckerd's tool, statewide. A 2022 peer-reviewed study in Child Abuse & Neglect examining a statewide implementation of Eckerd Rapid Safety Feedback found no positive effect on repeat maltreatment. Eckerd's marketing claim — that no child receiving its in-home services in Hillsborough County died of abuse after RSF launched — was a count of deaths within its own client population, not a controlled comparison. Variants of RSF nonetheless spread to Ohio, Indiana, Maine, Louisiana, Tennessee, Connecticut, Oklahoma, and Alaska.
The Disability Question — and the Inquiry That Went Quiet
In the fall of 2022, civil-rights complaints were filed with the Justice Department's Civil Rights Division alleging that AFST discriminates against parents with disabilities in violation of Title II of the Americans with Disabilities Act. DOJ attorneys had themselves cited the AP investigation in urging complainants to file formally. The mechanism alleged is straightforward: the National Council on Disability has documented that parents with disabilities receive public benefits — SNAP, Medicaid, SSI — at high rates. A model that ingests benefit receipt and behavioral-health diagnoses will penalize disability whether or not anyone typed the word "disability" into the code.
The version currently deployed — AFST V3 — includes features reflecting recorded diagnoses of behavioral and mental-health disorders that have been recognized as disabilities under the ADA.
On June 17, 2026, the Journal of Public Child Welfare published "Evaluating disability bias in the Allegheny Family Screening Tool," which tested the question empirically using the American Community Survey, the Medical Expenditure Panel Survey, and the National Survey on Drug Use and Health. Its conclusion: parents and children with disabilities were significantly more likely to be penalized by every element of the AFST scoring examined — socioeconomic indicators, healthcare utilization, public-benefits involvement, and criminal-legal contact. "Despite containing no explicit disability measures," the authors write, "the AFST effectively encodes disability status through proxy variables."
As of July 2026 — three and a half years after the complaints — no public findings, no letter of findings, no settlement, and no enforcement action from the Civil Rights Division has surfaced. The tool remains in daily use.
The Accountability Gap
Stack the failures up and the hole is obvious: there is no one whose job this is.
- No legislature authorized it. These tools were not created by statute. They were procured. CAPTA (42 U.S.C. § 5106a) requires states to have screening procedures; it says nothing about algorithms. Title IV-E funds them without naming them. No elected body in Pennsylvania, New York, or Colorado ever voted on whether an SSI record may be used to decide if a child is investigated.
- No court sees the score. Research on child-welfare risk models finds they are used almost entirely internally by CPS staff; lawyers and judges are generally not involved, and the scores usually do not surface in court hearings — in some cases, legal professionals are excluded deliberately. A parent facing removal has a constitutional liberty interest in the care of their children (Stanley v. Illinois; Santosky v. Kramer; Troxel v. Granville), and a right to confront the evidence against them. They cannot confront a number they are never shown, generated by a model they cannot inspect, from records they cannot correct. Douglas County, Colorado is the outlier that proves the rule: it says it will share scores with families who ask.
- No independent audit is required. The Family First Prevention Services Act built an entire federal apparatus — the Title IV-E Prevention Services Clearinghouse — to rate whether a parenting class is evidence-based. A statistical model that decides whether the state kicks in your door faces no such review. The evaluations that exist are largely commissioned by the agencies deploying the tools or authored by the people who built them.
- The one federal watchdog that stirred has gone silent, while the federal agency that should be regulating instead just wrote a check.
Why It Matters, and What Would Fix It
Roughly 63 percent of removals nationally cite neglect, and about three in four of the 542,900 substantiated victims in 2023 experienced neglect rather than physical or sexual abuse. In Indiana in 2024, 87 percent of children thrown into foster care had parents who were not even accused of physical or sexual abuse. "Neglect" in American practice is frequently a description of poverty: an empty refrigerator, a shut-off notice, an unstable apartment, an untreated illness.
Now feed that system a model whose inputs are Medicaid enrollment, SSI receipt, public-benefits history, and behavioral-health diagnoses, and whose target is removal. You have not built a child-safety tool. You have built a poverty detector with a foster-care actuator — and then made its output legally unreachable.
Four things would fix it, and none require new science:
- Statutory authorization or nothing. No jurisdiction should score a family on the basis of government records without a law passed by an elected legislature specifying which records may be used and for what.
- Disclosure as a condition of use. If a score touches a screening or removal decision, the score, its inputs, and the model documentation must be disclosed to the parent and to counsel and be admissible for challenge in any dependency proceeding. A score that cannot survive cross-examination should not be generated.
- Ban the poverty and disability proxies. Medicaid enrollment, SSI, SNAP, and behavioral-health diagnoses obtained through public insurance should be categorically excluded as model features. Their predictive power comes from surveillance density, not risk.
- Condition the federal money. ACF has the leverage and doesn't need Congress to use it: make Title IV-E administrative match and CCWIS reimbursement contingent on independent, published, pre-deployment and annual bias audits — the same standard the Prevention Services Clearinghouse already applies to a home-visiting curriculum.
Until then, the arrangement stands: a family in Pittsburgh, or Denver's southern suburbs, or a New York City borough can be investigated because a model they've never heard of noticed they were poor, disabled, and previously reported — and the federal government, having quietly paid to build the pipes, has just budgeted $6 million to run more water through them.
Sources: AP/PBS: DOJ examining AI screening tool · AP/PBS: How an algorithm that screens for child neglect could harden racial disparities · Journal of Public Child Welfare: Evaluating disability bias in the AFST (June 2026) · ACLU: Family Surveillance by Algorithm · ACLU/HRDAG: The Devil is in the Details (FAccT 2023) · HRDAG: How Data Analysis Confirmed the Bias in a Family Screening Tool · CMU: How Child Welfare Workers Reduce Racial Disparities in Algorithmic Decisions · The Markup: The NYC Algorithm Deciding Which Families Are Under Watch · ACF: $6 Million for States to Pilot Predictive Analytics · Nextgov: HHS wants states to use more predictive analytics · ACF: Modernizing Child Welfare Technologies and Tools (Mar 2026) · The Imprint: Illinois Drops Rapid Safety Feedback · GovTech: Illinois Ends Child Abuse Prediction Program · Child Abuse & Neglect: Effects of Eckerd Rapid Safety Feedback · NPR: Oregon dropping AI tool used in child welfare · LA County DCFS: Risk Stratification Methodology Report · NCCPR: LA County drops predictive analytics experiment · Federal Register: CCWIS Final Rule · eCFR 45 CFR Part 1356 · HHS ASPE: Predictive Analytics in Child Welfare Assessment · Rittenhouse et al.: Algorithms, Humans and Racial Disparities in CPS · Springer: Hello Baby predictive risk model · Logic: The AFST's Overestimation of Utility and Risk · Eckerd Youth Alternatives Inc. Form 990 filings, EIN 59-2551416 (PMC CivicOps database / ProPublica Nonprofit Explorer)
A note on what I could not confirm: I found no public record of any findings, settlement, or enforcement action from the DOJ Civil Rights Division inquiry — the most recent substantive reporting is from early 2023. I also want to flag the strongest counter-evidence honestly: a study co-authored by Rittenhouse, Putnam-Hornstein, and Vaithianathan found AFST reduced racial disparities by 31–48%, and a Stanford evaluation commissioned by Allegheny County found no adverse consequences. I included that in the report and named the conflict rather than omitting it — two of the three authors of the favorable study built the tool. Two web-source fetches were blocked pending permission (the ACLU report PDF and the ACF grant NOFO), so the CCWIS match rates and grant terms come from secondary reporting rather than the primary documents; if you want those verified against the source PDFs, grant WebFetch and I'll re-check.