Documentation

Two ways to run an eval. A8 (Touchstone) is a general eval model: state any criterion and it returns a verdict, over an OpenAI-compatible endpoint. The Metrics API serves the catalog of named metrics — criteria already trained and benchmarked — over our own REST surface.

Both return a measured score rather than generated text, and the same input always returns the same score.

§1Quickstart

A first verdict from A8, in three steps. The full request and response contract is on the A8 reference.

§1.1Get a key

Issue one under Settings → API keys. It needs the models:infer permission; teaching with expected needs models:train. See API keys.

§1.2Point a client at the base URL

Any OpenAI-compatible client works. Only the base URL changes.

# pip install openai from openai import OpenAI client = OpenAI( base_url="https://api.u22a8.ai/eval/v1", api_key="<your key>", )

§1.3State a criterion, send a subject

The system message is the criterion — what is being judged. The last user message is the subject — the thing under judgment.

resp = client.chat.completions.create( model="a8", messages=[ {"role": "system", "content": "Is the reply free of hedging?"}, {"role": "user", "content": "It may possibly work, but I could be wrong."}, ], ) print(resp.choices[0].message.content) # the verdict print(resp.model) # the snapshot that answered

§2Where to go next

A8 TouchstoneWhat A8 is, the shape of a call, and where to go next.Read first →Metrics APINamed criteria, already trained and benchmarked, scored over REST.Browse the concepts →