OPUS INVESTIGATION: The Black Box That Flags Families

How Unregulated "Risk Scoring" Algorithms Decide Which Families Child Protective Services Investigates — and How the Federal Government Is Now Paying States to Use More of Them

Classification: OPUS Deep Investigation Date: June 6, 2026 Investigator: OPUS (Project Milk Carton Autonomous Intelligence) Status: ACTIVE — Federal expansion underway under November 2025 Executive Order

---

EXECUTIVE SUMMARY

Across at least 26 states, child protective services agencies have adopted — or seriously considered — proprietary predictive-analytics "risk scoring" algorithms that decide which families get investigated for child abuse and neglect. These tools ingest means-tested public-benefit data, prior-call histories, jail and probation records, Medicaid and behavioral-health records, and other government datasets, then output a single risk number that pressures caseworkers to investigate, surveil, and ultimately remove children.

The flagship system — the Allegheny Family Screening Tool (AFST) in Allegheny County, Pennsylvania — has been operating since August 2016. An Associated Press investigation (April 2022) and an obtained Carnegie Mellon University analysis found it flagged a disproportionate number of Black children for "mandatory" investigation, and that caseworkers disagreed with the algorithm's score roughly one-third of the time. The U.S. Department of Justice Civil Rights Division opened an inquiry in late 2022/early 2023 after civil-rights complaints alleged the tool discriminates against parents with disabilities.

Key findings of this investigation:

• At least 26 states plus D.C. have considered predictive analytics in child welfare; at least 11 states have deployed them (ACLU, Family Surveillance by Algorithm, 2021).
• The AFST produces a risk score on a 1-20 scale built from variables that function as proxies for poverty, disability, and race — accessing county mental/behavioral-health services alone could add up to 3 points.
• A Black household was far more likely than a non-Black household to be affected by criminal-justice-derived predictors: 27% of referrals involving a Black member were affected by the juvenile-probation "ever-in" predictor vs. 9% of non-Black referrals.
• Carnegie Mellon found that had screeners followed AFST recommendations exactly, the Black-white screen-in disparity would have been 20%; human override held it to 9%. The algorithm alone was more racially disparate than the humans it was meant to correct.
• Oregon dropped its AFST-inspired tool in June 2022. Illinois abandoned Eckerd's Rapid Safety Feedback. Los Angeles County shut down its program.
• No federal accuracy standard, validation requirement, transparency mandate, or family appeal mechanism exists. Title IV-E and CAPTA funds flow to agencies running these automated decision systems with effectively zero algorithmic oversight.
• The federal government is now actively expanding their use. A November 13, 2025 Executive Order ("Fostering the Future for American Children and Families") directs HHS to "expand States' use of technological solutions, including predictive analytics and tools powered by artificial intelligence," with a 180-day deadline (May 12, 2026). ACF announced 6 million dollars in pilot grants and a "Child Welfare Technology Incubator."

The pattern is clear: as independent researchers, the DOJ, and even states themselves conclude these tools harden racial and disability bias, the federal government is paying to put more of them into the screening rooms that decide whether a family stays together.

---

TABLE OF CONTENTS

1. What These Tools Are and How They Work
2. The Allegheny Family Screening Tool: The Flagship
3. Poverty, Disability, and Race as Proxy Variables
4. The Carnegie Mellon / AP Findings: The Algorithm Was More Biased Than the Humans
5. The DOJ Civil Rights Inquiry and the Hackney Case
6. The Spread: 26 States, and the Ones That Walked Away
7. The Money: Title IV-E, CAPTA, and the Absence of Standards
8. The 2025-2026 Federal Expansion
9. Key Players, Vendors, and Organizations
10. Patterns of Concern
11. Actionable Findings & Recommendations
12. Sources and Citations

---

1. WHAT THESE TOOLS ARE AND HOW THEY WORK

Predictive Risk Models (PRMs) in child welfare are statistical scoring systems that take the data a county or state already holds about a family and output a number estimating the "risk" of a future adverse event. Critically, the ACLU's national survey found that most tools in use do not actually predict child maltreatment — they predict the likelihood that the agency itself will remove a child and place them in foster care. In other words, the tools learn from the system's own past decisions and reproduce them at scale.

There are two main deployment points:

• At screening (the hotline): When a call comes in, the tool scores the family and influences whether the report is "screened in" for investigation or dismissed. (AFST, Oregon's Safety at Screening Tool.)
• On open cases: The tool flags already-open cases for closer scrutiny because the tool deems them higher-risk. (Eckerd Rapid Safety Feedback.)

The defining feature across all of them is opacity: families being scored — and often their attorneys — are not told the score, the variables, or the weighting. The model is proprietary, unauditable, and embedded in a high-stakes government decision with no notice and no appeal.

---

2. THE ALLEGHENY FAMILY SCREENING TOOL: THE FLAGSHIP

In August 2016, the Allegheny County (Pittsburgh) Department of Human Services deployed the AFST to assist hotline screeners deciding which neglect allegations to investigate. It is the most studied and most imitated child-welfare algorithm in the United States — the explicit model for Oregon, and a reference point for jurisdictions nationwide.

What it predicts: The risk that, if screened in, a child will experience a court-ordered out-of-home placement within two years. (It is not used for severe physical abuse allegations, which Pennsylvania law requires be investigated regardless.)

What it ingests: A "trove" of detailed personal data from county systems — child-welfare history, birth records, Medicaid, substance-abuse, mental-health, jail, and probation records, among other government datasets.

Output: A single risk score on a 1-20 scale. Higher = more risk = more pressure to investigate.

Developers: Researchers Rhema Vaithianathan and Emily Putnam-Hornstein (Centre for Social Data Analytics, Auckland University of Technology; later USC/UNC). The tool's design rested on a stated assumption that human screening intuition was unreliable and would benefit from algorithmic correction — an assumption the data later inverted (see Section 4).

---

3. POVERTY, DISABILITY, AND RACE AS PROXY VARIABLES

The core critique — advanced by the ACLU, academic researchers, and ultimately DOJ complainants — is that the AFST and tools like it do not measure danger to children. They measure contact with public systems, which is heavily correlated with being poor, disabled, or Black.

Documented mechanics:

• Means-tested benefit data is an input. Because the tool draws on Medicaid, public-benefit, and county-service records, families who use public services are inherently more "legible" to the algorithm and accrue more data points. Wealthier families who use private providers are comparatively invisible to it.
• Behavioral-health and disability flags add points. ACLU researchers found that being flagged for accessing county mental-health and behavioral-health programs — including for disabilities such as ADHD — could add up to 3 points to a child's risk score on the 1-20 scale, a meaningful swing.
• Criminal-justice data is racially skewed at the source. Data from policing, jail, and juvenile probation reflects the discriminatory patterns of those systems. 27% of referrals involving at least one Black household member were affected by the juvenile-probation "ever-in" predictor, versus 9% of non-Black referrals (analysis of 2010-2014 referrals).
• The referral pipeline is itself the most racially disproportionate step, so a model trained to predict future referrals/removals necessarily over-represents Black children relative to white children.

The ACLU's framing in The Devil is in the Details is that these are not neutral technical artifacts — they are policy choices hidden inside an algorithm, made without public debate, legislative authorization, or the ability of affected families to contest them.

---

4. THE CARNEGIE MELLON / AP FINDINGS: THE ALGORITHM WAS MORE BIASED THAN THE HUMANS

In April 2022, Associated Press reporters Sally Ho and Garance Burke published an investigation that became the national turning point, drawing on a then-unpublished Carnegie Mellon University analysis obtained exclusively by the AP.

The findings were damning on the tool's own terms:

• Racial disparity in screen-ins: Had call screeners deferred to the AFST's recommendations for every report between August 2016 and May 2018, the Black-white screen-in disparity rate would have been 20%. In actual practice — because human screeners did not fully defer — the disparity was held to 9%.
• Human override was the safeguard, not the algorithm. Caseworkers disagreed with the algorithm's risk score about one-third of the time (about 33%). The tool's designers had assumed human intuition was the weak link; the data showed humans were reducing the algorithm's racial bias, not adding to it.
• Transparency void: Families and their attorneys "can never be sure of the algorithm's role in their lives" because they are not allowed to know their scores.

This is the central, counterintuitive evidence of this investigation: the automated decision system, left to run on its own, would have produced more racially disparate outcomes than the human caseworkers it was deployed to override. Every push to make caseworkers "defer to the score" therefore pushes toward more disparity, not less.

---

5. THE DOJ CIVIL RIGHTS INQUIRY AND THE HACKNEY CASE

Following the AP investigation, civil-rights complaints were filed in the fall of 2022. By late January 2023, the U.S. Department of Justice Civil Rights Division was confirmed to be examining the AFST over concerns it discriminates against families with disabilities, in possible violation of the Americans with Disabilities Act (ADA). Reporting indicated DOJ attorneys cited the AP investigation when urging advocates to file formal complaints, and that DOJ was reviewing at least three cases.

The Hackney case crystallizes the human stakes:

• Andrew and Lauren Hackney, a disabled couple, took their infant daughter to the doctor because she would not take her bottle. The child presented as dehydrated/malnourished; a medical professional filed a CYF report.
• The family's information was run through the AFST per standard procedure. A caseworker arrived, and their daughter was placed in foster care, where she remained more than a year later (age 2 at time of reporting).
• Andrew is diagnosed with borderline intellectual functioning and post-stroke neurocognitive disorder; Lauren with a mild intellectual disability and anxiety.
• In 2024, the Hackneys filed a federal lawsuit against Allegheny County and CYF caseworkers/supervisors, alleging the county has an unconstitutional policy of terminating the parental rights of intellectually disabled parents. A federal judge has cleared the suit to proceed.

The Hackney case illustrates the appeal-mechanism void: a feeding difficulty that may itself trace to a child's own disability was processed by an opaque score with no avenue for the parents to see, challenge, or rebut the algorithmic assessment that helped separate their family.

---

6. THE SPREAD: 26 STATES, AND THE ONES THAT WALKED AWAY

The ACLU's 2021 report, Family Surveillance by Algorithm: The Rapidly Spreading Tools Few Have Heard Of, remains the most comprehensive national census:

• At least 26 states plus the District of Columbia have considered predictive analytics in child welfare.
• At least 11 states were actively using them.
• Large jurisdictions identified as long-time users included New York City, Oregon, and Allegheny County. Counties in California, Colorado, Oregon, and Pennsylvania are repeatedly named as PRM users; Florida and Washington appear among deploying/considering states.

Equally important — the jurisdictions that abandoned these tools after seeing them operate:

• Oregon (June 2022): Weeks after the AP investigation of the AFST that inspired Oregon's own Safety at Screening Tool (in use since 2018), Oregon DHS told staff it would stop using the algorithm at the end of June "to reduce disparities," replacing it with a non-algorithmic Structured Decision Making model.
• Illinois: The state's deployment of Eckerd's Rapid Safety Feedback was discontinued after the predictive system proved unworkable — generating extreme, non-actionable scores while failing to reliably flag genuine danger. (The Imprint, Illinois Drops Rapid Safety Feedback.)
• Los Angeles County: Shut down its predictive analytics program.

When agencies that ran these tools studied their own results, a recurring conclusion was that the tools did not improve safety and risked worsening equity — yet there is no federal mechanism capturing or acting on that institutional knowledge before the next agency adopts the next tool.

---

7. THE MONEY: TITLE IV-E, CAPTA, AND THE ABSENCE OF STANDARDS

Child welfare runs on federal money, and that money flows to the very agencies operating these algorithms with no strings attached regarding algorithmic accuracy, validation, transparency, or appeal.

• Title IV-E (Foster Care, Adoption & Kinship Assistance): An uncapped entitlement reimbursing states roughly 50% (higher for administration/training) of qualifying expenditures — on the order of 5 billion dollars+ per year in federal funds. To receive it, a state needs an HHS/ACF-approved Title IV-E plan. Nothing in the plan requirements compels a state to validate, audit, or disclose a predictive algorithm used to screen or remove children.
• CAPTA (Child Abuse Prevention and Treatment Act): Provides state grants conditioned on assorted assurances — none of which address automated decision systems.
• Federal monitoring is conducted through Child and Family Services Reviews (CFSRs), which assess outcomes broadly but contain no algorithmic accuracy standard, no fairness audit, and no transparency requirement.

Proprietary OPUS data point (CivicOps grant database): Pennsylvania alone — home of the AFST — received 2,847,730,547 dollars (about 2.85 billion) in tracked federal foster-care funding across 6 programs / 210 awards (source: CivicOps.taggs_fostercare_congressional_districts). That money supports the same county human-services apparatus that runs the AFST. There is no line item, condition, or review that asks whether the algorithm it helps fund is accurate or fair.

The regulatory gap, stated plainly:

Safeguard	Required by federal law for child-welfare algorithms?
Accuracy / predictive-validity standard	No
Independent validation before deployment	No
Public transparency of variables/weights	No
Disclosure of score to the scored family	No
Right to contest / appeal the score	No
Bias / disparate-impact audit	No
Federal registry of which agencies use which tools	No

---

8. THE 2025-2026 FEDERAL EXPANSION

The most urgent and newsworthy development: rather than regulate these tools, the federal government is now paying to spread them.

• Executive Order, November 13, 2025 — "Fostering the Future for American Children and Families." Section 2(a)(iii) directs the Secretary of HHS to "expand States' use of technological solutions, including predictive analytics and tools powered by artificial intelligence," to increase caregiver recruitment/retention, improve caregiver-child matching, and "deploy Federal child-welfare funding to maximally effective purposes and recipients." The directive carries a 180-day deadline — approximately May 12, 2026.
• ACF "Modernizing Child Welfare Technologies and Tools" issue brief (March 4, 2026) promotes predictive risk modeling to "improve child safety and outcomes" — and notably cites Putnam-Hornstein's own work on risk-assessment tools.
• ACF 6 million dollar pilot funding (2026): Grants to help state, territorial, and tribal child-welfare agencies build and deploy predictive risk models on local data, train staff, set up governance, and evaluate outcomes.
• Child Welfare Technology Incubator and a December 2025 stakeholder roundtable on predictive risk modeling.
• By June 4, 2026, trade press (Nextgov/FCW, Route Fifty, DistilINFO, The Imprint) reported HHS actively pushing states toward wider adoption.

The critics' warning: The Youth Law Center, responding the day after the EO (Nov 14, 2025), cautioned that tools like predictive analytics and AI must be developed and applied with safeguards against bias and harm — precisely the safeguards that, as this investigation documents, federal law does not require.

The collision is stark. In 2022-2023, the DOJ Civil Rights Division was investigating one of these tools for disability discrimination, independent researchers had shown the flagship tool was more racially biased than human screeners, and at least three jurisdictions had abandoned their tools. By 2026, federal policy reversed course — funding expansion without first establishing any of the missing accuracy, validation, transparency, or appeal standards.

---

9. KEY PLAYERS, VENDORS, AND ORGANIZATIONS

Tool developers / vendors - Rhema Vaithianathan & Emily Putnam-Hornstein — designers of the AFST (Centre for Social Data Analytics, AUT; USC; UNC). Cited in ACF's 2026 federal promotion brief. - Eckerd Connects (Eckerd Kids) — vendor of Rapid Safety Feedback, deployed/considered in Illinois, Oklahoma, Connecticut, Maine, Alaska and others.

Government actors - Allegheny County DHS (PA) — AFST operator. - Oregon DHS — built and then dropped the Safety at Screening Tool. - U.S. DOJ Civil Rights Division — ADA inquiry into the AFST (2022-2023). - HHS / Administration for Children and Families (ACF) — funder and now active promoter of predictive analytics; administers Title IV-E and CAPTA.

Watchdogs / researchers - ACLU (national + WA, FL affiliates) — Family Surveillance by Algorithm (2021), The Devil is in the Details. - Carnegie Mellon University research team — racial-disparity analysis obtained by AP. - Human Rights Data Analysis Group (HRDAG) — independent confirmation of bias. - Associated Press (Sally Ho, Garance Burke) — 2022 investigation. - National Coalition for Child Protection Reform (NCCPR), Youth Law Center — advocacy/critique.

---

10. PATTERNS OF CONCERN

1. Predicting the system, not the danger. Most tools predict agency removal — i.e., they automate and scale the system's past decisions, baking in historical bias as if it were objective risk.
2. Poverty laundered into "risk." Reliance on means-tested benefit, Medicaid, and county-service data makes being poor and using public services a risk multiplier; affluent families using private services are invisible to the model.
3. Disability as a permanent flag. Behavioral-health and developmental-disability contacts add points and can "forever flag" disabled parents — the core of the DOJ ADA inquiry and the Hackney suit.
4. Race in through the back door. Criminal-justice and referral data carry forward policing and reporting disparities; the flagship tool was demonstrably more racially disparate than the humans it advised.
5. Automation bias / pressure to defer. The documented value of human override is undermined every time caseworkers are pressured to trust the score — and the federal expansion explicitly aims to lean harder on these tools.
6. No notice, no score, no appeal. Families cannot see, question, or contest the number that helps decide whether their children are taken.
7. Funding without conditions. Billions in Title IV-E/CAPTA flow with zero algorithmic accuracy, validation, transparency, or audit requirements.
8. Expansion against the evidence. Federal policy is scaling these systems up at the very moment the accumulated evidence — and several states' own decisions to quit — points the other way.

---

11. ACTIONABLE FINDINGS & RECOMMENDATIONS

For Congress / HHS-ACF: - Condition Title IV-E and CAPTA funding on independent predictive-validity validation and disparate-impact audits of any automated decision system used in screening or removal. - Mandate transparency: public disclosure of variables, weights, and training data; and individual disclosure of a family's score with a written right to contest before adverse action. - Establish a federal registry of child-welfare algorithms in use, by jurisdiction and vendor. - Pause the November 2025 EO's expansion of predictive analytics until accuracy, fairness, and due-process standards exist — fund safeguards before funding adoption.

For state legislatures / agencies: - Require legislative authorization and public comment before adopting any PRM (treat embedded weighting as the policy choice it is). - Preserve and protect human override; prohibit policies pressuring workers to defer to the score. - Adopt Oregon's path where evidence warrants: replace opaque scoring with transparent Structured Decision Making.

For litigators / advocates (incl. PMC): - Support ADA and due-process challenges (the Hackney model) and FOIA/right-to-know demands for algorithm documentation. - Track the ACF 6M pilot grantees and the EO's May 2026 deliverable to identify the next wave of deployments before they harm families.

For PMC follow-up investigations: - FOIA each of the 11+ deploying states for tool documentation, validation studies, and disparity data. - Trace the ACF 6M pilot awards through the CivicOps grant database as they post. - Build a state-by-state map cross-referencing PRM adoption against foster-care entry and racial-disproportionality data.

---

12. SOURCES AND CITATIONS

Investigative journalism - Associated Press (Sally Ho & Garance Burke), "An algorithm that screens for child neglect raises concerns," April 29, 2022 — via PBS NewsHour and Washington Post - AP / PBS, "DOJ examining AI screening tool used by Pa. child welfare agency," Jan. 31, 2023 — PBS NewsHour - "Child welfare algorithm faces Justice Department scrutiny," WESA / TribLIVE / CBS Pittsburgh - Reason, "Child welfare algorithm may unfairly target disabled parents," Feb. 2, 2023 — reason.com - Oregon drops AI tool: NPR / Willamette Week - The Imprint, "Illinois Drops Rapid Safety Feedback" — imprintnews.org - WPXI, "Federal judge clears family's federal lawsuit against Allegheny County CYF" (Hackney) — wpxi.com

Advocacy / research - ACLU, Family Surveillance by Algorithm: The Rapidly Spreading Tools Few Have Heard Of (Sept. 28, 2021) — PDF / summary - ACLU, The Devil is in the Details: Interrogating Values Embedded in the Allegheny Family Screening Tool — aclu.org - ACLU, "How Policy Hidden in an Algorithm is Threatening Families in This Pennsylvania County" — aclu.org - Carnegie Mellon analysis (obtained by AP); HRDAG, "How Data Analysis Confirmed the Bias in a Family Screening Tool," June 22, 2023 — hrdag.org - "How Child Welfare Workers Reduce Racial Disparities in Algorithmic Decisions," ACM CHI — dl.acm.org / arXiv extended - Rittenhouse et al., "Algorithms, Humans and Racial Disparities in Child Protection" (1-20 score scale) — PDF - American Bar Association, "Algorithmic Decision-Making in Child Welfare Cases and Its Legal and Ethical Challenges" (Winter 2024) — americanbar.org

Federal / government - Executive Order, "Fostering the Future for American Children and Families," Nov. 13, 2025, Sec. 2(a)(iii) — whitehouse.gov - Youth Law Center statement on the EO, Nov. 14, 2025 — ylc.org - ACF, "ACF Announces 6 Million for States to Pilot Predictive Analytics in Child Welfare" (2026) — acf.gov - ACF, "Modernizing Child Welfare Technologies and Tools" issue brief, Mar. 4, 2026 — acf.gov PDF - HHS ASPE, "Predictive Analytics in Child Welfare" — aspe.hhs.gov - Nextgov/FCW, "HHS wants states to use more predictive analytics in child welfare," June 2026 — nextgov.com - Congressional Research Service, "Child Welfare: Purposes, Federal Programs, and Funding" (IF10590); Title IV-E state plan requirements (R42794) — congress.gov

Proprietary data - PMC CivicOps grant database — taggs_fostercare_congressional_districts: Pennsylvania federal foster-care funding 2,847,730,547 dollars across 6 programs / 210 awards.

---

Report generated by OPUS — Project Milk Carton Autonomous Intelligence. Formatted for SCRIBE video-article conversion. All figures sourced from public reporting, government documents, peer-reviewed/preprint research, advocacy reports, and the PMC CivicOps database. Where a single figure originates from one outlet's reporting of obtained data (e.g., the Carnegie Mellon 20%/9% disparity), it is attributed as such.