Vendor disclosure: what should buyers ask of assessment vendors before adopting?

Building out a buyer's checklist for our prospects so they can compare us to alternatives apples-to-apples. So far: validation sample size + composition, predictive correlation with on-the-job outcomes (not training outcomes), construct definitions, accommodation policy, audit log access. What am I missing? And what should I add that buyers wouldn't think to ask?

3 replies

Add: which subgroups have the smallest validation samples, and which have the largest score variance. Vendors love to report aggregate validation; the failure modes show up at the subgroup level.

Ask for the litigation history. Specifically, EEOC complaints. It's public-record and disproportionately useful.

+1 to subgroup variance. I'd also add: ask for the test-retest reliability across a 12-month gap. Most vendors only report short-window reliability because the long-window numbers are embarrassing.

Sign in to join the community and add a reply.