AI & Machine Learning · Jev + Claude
Jev MCP with Claude: Structured Model Evaluations Through Natural Language
The short answer
Claude works with Jev through the Model Context Protocol (MCP), an open standard that gives Claude a secure adapter to Jev. Once connected with an API key, Claude can discover available models, run Noul evaluations, ask Choice and Score questions, and retrieve structured model state, all from natural-language requests.
What is Jev?
Jev is the System One API from TypeSafe AI. It helps teams discover which models are available and evaluate structured state using Noul, Choice and Score evaluations with strong typing. Typical users are AI product teams, data scientists, and engineering groups who build agents and chatbots that need reliable, structured answers about model behaviour. Jev can be used directly through its API or through an MCP server.
How Claude connects to Jev through MCP
The Model Context Protocol is an open standard that lets Claude securely use other software, reading information and taking actions, with permissions the organisation controls. Think of it as a universal adapter between Claude and Jev. Your team connects Jev once, and Claude can then work with it without custom code for every request.
Once connected, Claude can typically list the models available through Jev, run Noul evaluations, ask Choice and Score questions, and retrieve structured model state. A team member can ask, in plain English, for a comparison of two models on a set of criteria, and Claude carries out the matching Jev calls and returns the structured results.
Access is controlled by the API key you provide, which you can scope, rotate and revoke. Consequential actions can be set to require human approval before they run, and every request can be logged so your team can review what Claude did and why. People stay in charge of what Jev is asked to do.
What you can automate
Model discovery for product teams
Product managers can ask Claude which models are currently available through Jev and what each is suited to. Claude retrieves the list and summarises it in plain language, saving time spent searching documentation. The team gets a current view of options before choosing what to build with.
Structured evaluations for AI agents
Engineering teams can have Claude run Noul evaluations against a test set of agent responses. Results come back in a consistent, typed structure, which makes them easy to compare across releases. Teams can then spot regressions before a chatbot or agent reaches customers.
Choice and Score questions for quality reviews
Quality leads can ask Claude to run Choice and Score questions on a batch of outputs, such as which answer best meets a brief or how well it scores against set criteria. Claude gathers the results and presents them in a summary table. Reviewers can focus on the cases that need a human decision.
Retrieving model state for support teams
Support and operations staff often need to know the current state of a model before answering a customer or internal query. Claude can retrieve structured model state through Jev and explain it in everyday words. This reduces the back-and-forth between support and engineering.
Reporting for leadership
Department heads can ask Claude for a weekly digest of evaluation results, covering what changed, what passed, and what needs attention. Claude pulls the figures from Jev and drafts the summary. Leaders get consistent reporting without waiting on a manual report.
An end-to-end workflow: Weekly agent quality review
TriggerEvery Monday morning, a scheduled request asks Claude to run the weekly quality review for the customer support agent.
Claude lists the models currently used by the support agent through Jev.
Claude runs Noul evaluations on last week's sampled conversations and records the typed results.
Claude runs Choice and Score questions on flagged replies and checks them against the team's quality criteria.
Claude writes the summary into a shared document in Microsoft 365 and creates a ticket in the team's ticketing system for each regression found.
A team lead reviews the ticketed items and approves any change before it goes live.
OutcomeThe support team receives a consistent weekly quality report with issues already logged for follow-up, typically reducing the manual effort of the review and catching regressions earlier.
How we set it up
Free automation audit
We review your current agents, chatbots and evaluation work to identify the workflows where Jev and Claude will save the most time.
Connect Jev with an API key
We set up a secure connection using a Jev API key, scoped to the least access needed, and store the key in your organisation's secret management.
Design the end-to-end workflow
We map how Jev connects to your other systems, such as Microsoft 365, your CRM, ERP or ticketing tool, and agree which actions need human approval.
Test on real past cases
We run the workflow against past evaluation work and compare results with your team's existing judgements before anything goes live.
Launch, monitor and train
We go live with monitoring and audit logs in place, then train your team to use Claude with Jev confidently and to review its outputs.
Who benefits
- AI product manager
- Gets a clear, current view of available models and evaluation results without digging through documentation.
- Machine learning engineer
- Runs structured evaluations through plain-language requests and gets typed results that are easy to compare.
- Quality assurance lead
- Reviews agent outputs faster, with Choice and Score results summarised and flagged cases routed for decision.
- Operations or support manager
- Understands model state in everyday terms, so answers to customers and colleagues are quicker and more accurate.
Security and control
- Access uses an API key with least-privilege scope, which can be rotated or revoked at any time.
- Consequential actions can require approval from a named person before they run.
- Every request can be recorded in audit logs so your team can review what was done and why.
- Claude only reaches the parts of Jev and your other systems that you explicitly allow.
- We recommend regular reviews of permissions and evaluation results to keep the setup aligned with your policies.
For teams in the US, UK, India and the Middle East
Organisations in the United States and the United Kingdom can host the workflow in a region that meets their data requirements. Indian organisations are subject to the Digital Personal Data Protection Act, 2023, and we design around its obligations. Organisations in the UAE and Saudi Arabia must follow their local data protection laws, and we select a hosting region to meet those requirements. Hosting options cover India, the United States, the United Kingdom and the European Union.
Learn more about Claude and MCP
Anthropic's AI model and assistant.
Secure links between Claude and business tools.
How engineers build and change software with Claude.
Hosted agents that run multi-step work.
The open standard Claude uses to connect to tools.
Frequently asked questions
What does the Jev MCP server do?
The Jev MCP server acts as a secure bridge between Claude and Jev. It lets Claude discover available models, run Noul evaluations, ask Choice and Score questions, and retrieve structured model state. Your team can make these requests in plain English rather than writing an API call each time.
Is connecting Claude to Jev secure?
Security depends on how the connection is set up. We use an API key with the smallest access needed, store it in your secret management, and can require human approval for important actions. Every request can be logged for review. We also test the setup before launch so your team knows exactly what Claude can reach.
What can Claude do with Jev once connected?
Typically Claude can list available models, run Noul evaluations, ask Choice and Score questions, and retrieve structured model state. It then summarises the results and can pass them to other tools, such as a shared document or ticketing system. The exact actions available depend on the access you grant.
How long does setup take?
The timeline depends on how many workflows you want to connect and how complex your other systems are. A focused setup for one workflow often takes a few weeks, including testing and training. We confirm the timeline during the free automation audit, once we understand your needs.
Which plan or permissions do I need for Jev?
In general you need a Jev account that can issue an API key with the access your workflow requires. The exact plan and permission levels are set by Jev, so we confirm them with your account owner during the audit. We recommend creating a dedicated key for Claude so its access can be managed separately.