AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents

Catalog evidence and recorded chart appearances. Inclusion is not an endorsement.

Source snapshot imported 2026-10-20. This is not a live feed. Source and contributions.

Evidence and limits

Finding: Synthetic tools and task-specific grading distinguish refusal from completion of harmful multi-step requests.

Impact: Watch the recorded review topic (see source), the recorded review topic (see source), the recorded review topic (see source).

Boundary: English prompts, no multi-turn attacker follow-up, custom-tool dependence and grading-path misses bound transfer; refusal is not evidence of safe owner-authorized behavior. Next: Keep harmfulness, capability and permission compliance separate and use only harmless synthetic fixtures for local evaluation. Added candidate with Selected full-text review; not an exhaustive evaluation appraisal; not full-text validated, replicated or adopted

Flags: Selected full-text review; not an exhaustive evaluation: English prompts, no multi-turn attacker follow-up, custom-tool dependence and grading-path misses bound transfer; refusal is not evidence of safe owner-authorized behavior.

Record State: Candidate only: not accepted into the reviewed catalog

What was reviewed: Selected full-text review; not an exhaustive evaluation

Inspect original source

Recorded chart appearances

  • 2026-10-13: position 19. Relevant to the recorded review topic (see source); Selected full-text review; not an exhaustive evaluation evidence
  • 2026-10-12: position 18. Relevant to the recorded review topic (see source); Selected full-text review; not an exhaustive evaluation evidence