Infrastructure & Governance
A domain circle for practitioners, researchers, and thinkers working on infrastructure, governance, technology, and education systems. Contribute expertise, discuss the Infrastructure Series, and collaborate on frameworks for building better systems.
Sign in to join this community.
Sign InDiscussion 14
Behavioral-econ angle: are we accidentally optimizing for legibility over accuracy?
Something that's been nagging at me since the last round of vendor reviews. Every assessment that wins large-employer deals is the one that produces the cleanest, most easily-explainable score. Not th…
Buyer-side norm proposal: contractual audit rights on assessment validation
Building on the earlier vendor-disclosure thread. We're about to start adding language to our enterprise contracts that gives the buyer the right to commission an independent validation audit annually…
Does the framework hold up in fully-remote orgs, or does it need adapting?
Working with a remote-first client on rolling structured assessments into their existing hiring loop. Two specific gaps showing up: (1) the Resilience pillar relies on observable recovery behaviors, w…
AI-assisted screening: what's actually showing up in your pilots?
We ran a small pilot last quarter putting an LLM-based screener in front of a Risk Efficacy-style structured interview. The headline number (offer-to-hire conversion) looked great. The subgroup breakd…
Looking for collaborators on a meta-analysis of decision-quality frameworks in hiring
Drafting a proposal for a systematic review of the dozen-or-so decision-quality frameworks that have surfaced in the hiring/measurement space over the past decade. Risk Efficacy obviously included. Lo…
Vendor disclosure: what should buyers ask of assessment vendors before adopting?
Building out a buyer's checklist for our prospects so they can compare us to alternatives apples-to-apples. So far: validation sample size + composition, predictive correlation with on-the-job outcome…
Devil's advocate: are we just building a more sophisticated form of credentialism?
Convince me this isn't going to end where every other measurement reform has: with a new gatekeeping infrastructure dominated by a small number of vendors, sold to large employers, and used to systema…
Adapting these frameworks for non-Western talent markets
Working on rolling Risk Efficacy-style assessments out across three APAC markets and the cultural translation is harder than the language translation. "Calibrated confidence" reads very differently in…
Pre-registration for assessment validation studies — should it be mandatory?
Spent the morning reading three vendor whitepapers that all claim their assessment is validated. None of them disclose a pre-registered analysis plan. Two of them retro-fit the validation criteria to…
GAO report on the federal merit system — anyone read it?
There's a GAO report from earlier this year on "Strengthening the Federal Merit System in the AI Era" that's worth reading if you care about this stuff. Two things stood out: (1) they explicitly call…
Measurement infrastructure at scale: distributed-systems lessons
One thing I find under-discussed in the merit-measurement conversation is how much it shares with distributed systems engineering. You have multiple sources of truth, each partially correct, each with…
Accessibility in computerized assessments — what's working?
Working on a project to redesign assessment tooling for a public-sector client. Standard accommodations (extended time, screen reader compatibility) are table stakes but feel insufficient. The framewo…
OPM hiring rule changes — anyone tracking implementation?
The federal hiring authority changes that landed in Q1 nominally allow agencies to weight skills assessments above degree requirements. In practice the agencies I've talked to are still defaulting to…
What's actually broken about credential-based hiring?
I keep coming back to the gap between what credentials measure (typically: persistence, conformity to a curriculum, ability to test well) and what they're used as proxy for (judgment, decision-making…