Evidence and limits
Finding: Humans completed 72.4% of tasks while the best evaluated agents completed less than 12.2%, exposing major grounding and operational-knowledge gaps in realistic computer use.
Impact: Strengthen executable-environment validation and bounded authority. Reject fluent planning as evidence of reliable computer action.
Boundary: Models and systems have advanced since the study, and benchmark success is not deployment safety. Next: what current, owning-system test is required before a computer-use capability receives write authority?
Flags: environment and model versions age quickly
Record State: Catalog review recorded; not a performance endorsement
What was reviewed: Structure and relevance screened; not a full evaluation