龙明
龙明

Agent评估与CI验证34篇

2026-08-31 04:27:46

没有评测的 agent 迭代全靠感觉。这组 33 篇讲 evals 怎么落地:基准选择、LLM-as-judge 的坑、回归测试怎么进 CI、线上 shadow 评估,以及 SWE-bench 等基准的批判性分析。既有方法论,也有可以直接抄的流水线配置和真实团队的落地故事。想让 agent 质量可量化、每次改动有据可依的团队,从这组开始。
Demystifying Evals for AI Agen
Demystifying Evals for AI Agen
Anthropic评估
promptfoo/promptfoo
promptfoo/promptfoo
GitHub提示评估开源
THUDM/AgentBench
image
清华Agent基准GitHub
Testing Agent Skills Systemati
Testing Agent Skills Systemati
OpenAI技能测试
Agent Evaluation Readiness Che
Agent Evaluation Readiness Che
LangChain评估清单
Evaluating Skills
Evaluating Skills
LangChain技能评估
AgentAssay: Token-Efficient Re
AgentAssay: Token-Efficient Re
arXiv论文回归评估
Agentic Harness for Real-World
Agentic Harness for Real-World
arXiv论文真实场景评估
Eval-Driven Development: Build
Eval-Driven Development: Build
RedHat评估驱动开发
Agent Evaluation Framework 202
Agent Evaluation Framework 202
Galileo评估框架
The 2025 AI Agent Index: Docum
The 2025 AI Agent Index: Docum
arXiv论文Agent索引
sentrux/sentrux
sentrux/sentrux
GitHub评估工具
Driving the Agent Quality Flyw
Driving the Agent Quality Flyw
Google质量飞轮
vaquarkhan/mcp-test-harness
vaquarkhan/mcp-test-harness
GitHubMCP测试工具
confident-ai/deepeval
confident-ai/deepeval
GitHub评估框架开源
claw-eval/claw-eval
claw-eval/claw-eval
GitHub评估工具
SWE-bench
SWE-bench
SWE-bench编程基准
Accio-org/RealReplicaBench
Accio-org/RealReplicaBench
GitHub复现基准
UKGovernmentBEIS/inspect_ai
UKGovernmentBEIS/inspect_ai
GitHub评估框架英国政府
Quantifying Infrastructure Noi
Quantifying Infrastructure Noi
Anthropic基础设施噪声
AgentLens: Revealing The Lucky
AgentLens: Revealing The Lucky
arXiv论文运气偏差
StaminaBench: Stress-Testing C
StaminaBench: Stress-Testing C
arXiv论文压力测试
Harness-Bench: Measuring Harne
Harness-Bench: Measuring Harne
arXiv论文Harness基准
sierra-research/tau-bench
sierra-research/tau-bench
GitHub工具使用基准
Towards a Science of AI Agent
Towards a Science of AI Agent
arXiv论文评估科学
Characterizing Faults in Agent
Characterizing Faults in Agent
arXiv论文故障分析
VeRO: An Evaluation Harness fo
VeRO: An Evaluation Harness fo
arXiv论文评估框架
Eval Awareness in Claude Opus
Eval Awareness in Claude Opus
Anthropic评估意识
Designing AI-Resistant Technic
Designing AI-Resistant Technic
Anthropic技术面试
Amazon Bedrock AgentCore Evalu
Amazon Bedrock AgentCore Evalu
AWS评估托管服务
Live-SWE-agent: First Live Sof
Live-SWE-agent: First Live Sof
arXiv论文在线评测
OccuBench: Evaluating AI Agent
OccuBench: Evaluating AI Agent
arXiv论文职业能力基准
microsoft/STATE-Bench
microsoft/STATE-Bench
微软状态管理基准GitHub

Comments

chat_bubble_outline

No comments yet, be the first to comment