evalctl
Evaluation harnesses for LLM apps, as a command line.
brew install ozhiaki/tap/evalctlLocal-first evals for agents, not just prompts.
evalctl treats eval cases as files, runners as shell commands, and results as durable artifacts. It scores what agents actually do — files written, diffs produced, commands run — on your own machine, with no gateway, dashboard, or SaaS account.