Welcome to Spec27

Spec27 helps teams evaluate LLM-powered agents with shared Projects, Datasets, Agents, Judges, Specifications, Evals, and Results.

This documentation is written for product users and workspace admins. It focuses on the visible product workflows in the app, not the internal implementation.

Start here

• New to the product: read What Spec27 helps you do.
• Setting up your first workspace: read Create your first project.
• Ready to run something end to end: read Run your first evaluation.

How the product fits together

Most teams move through Spec27 in this order:

1. Join or create an organization.
2. Create a Project.
3. Add Datasets, Agents, and any required Secrets.
4. Create Judges if you need judge-based scoring.
5. Create a Specification to define the test inputs and attack methods.
6. Create an Eval that connects your Agents and Specifications.
7. Use Playground for quick runs or start a full eval run.
8. Review Results and iterate.

Recommended reading path

• Mental model
• Evaluation methods
• Projects
• Datasets
• Agents
• Specifications
• Evals
• Runs and results