> ## Documentation Index > Fetch the complete documentation index at: https://openai-hd4n6.mintlify.site/llms.txt > Use this file to discover all available pages before exploring further. # Get an eval run > Get an evaluation run by ID. ## OpenAPI ````yaml api-definition.yaml get /evals/{eval_id}/runs/{run_id} openapi: 3.0.0 info: title: OpenAI API description: >- The OpenAI REST API. Please see https://platform.openai.com/docs/api-reference for more details. version: 2.3.0 termsOfService: https://openai.com/policies/terms-of-use contact: name: OpenAI Support url: https://help.openai.com/ license: name: MIT url: https://github.com/openai/openai-openapi/blob/master/LICENSE servers: - url: https://api.openai.com/v1 security: - ApiKeyAuth: [] tags: - name: Assistants description: Build Assistants that can call models and use tools. - name: Audio description: Turn audio into text or text into audio. - name: Chat description: >- Given a list of messages comprising a conversation, the model will return a response. - name: Completions description: >- Given a prompt, the model will return one or more predicted completions, and can also return the probabilities of alternative tokens at each position. - name: Embeddings description: >- Get a vector representation of a given input that can be easily consumed by machine learning models and algorithms. - name: Evals description: Manage and run evals in the OpenAI platform. - name: Fine-tuning description: Manage fine-tuning jobs to tailor a model to your specific training data. - name: Batch description: Create large batches of API requests to run asynchronously. - name: Files description: >- Files are used to upload documents that can be used with features like Assistants and Fine-tuning. - name: Uploads description: Use Uploads to upload large files in multiple parts. - name: Images description: Given a prompt and/or an input image, the model will generate a new image. - name: Models description: List and describe the various models available in the API. - name: Moderations description: >- Given text and/or image inputs, classifies if those inputs are potentially harmful. - name: Audit Logs description: List user actions and configuration changes within this organization. paths: /evals/{eval_id}/runs/{run_id}: get: tags: - Evals summary: Get an eval run description: Get an evaluation run by ID. operationId: getEvalRun parameters: - name: eval_id in: path required: true schema: type: string description: The ID of the evaluation to retrieve runs for. - name: run_id in: path required: true schema: type: string description: The ID of the run to retrieve. responses: '200': description: The evaluation run content: application/json: schema: $ref: '#/components/schemas/EvalRun' components: schemas: EvalRun: type: object title: EvalRun description: | A schema representing an evaluation run. properties: object: type: string enum: - eval.run default: eval.run description: The type of the object. Always "eval.run". x-stainless-const: true id: type: string description: Unique identifier for the evaluation run. eval_id: type: string description: The identifier of the associated evaluation. status: type: string description: The status of the evaluation run. model: type: string description: The model that is evaluated, if applicable. name: type: string description: The name of the evaluation run. created_at: type: integer description: Unix timestamp (in seconds) when the evaluation run was created. report_url: type: string description: The URL to the rendered evaluation run report on the UI dashboard. result_counts: type: object description: Counters summarizing the outcomes of the evaluation run. properties: total: type: integer description: Total number of executed output items. errored: type: integer description: Number of output items that resulted in an error. failed: type: integer description: Number of output items that failed to pass the evaluation. passed: type: integer description: Number of output items that passed the evaluation. required: - total - errored - failed - passed per_model_usage: type: array description: Usage statistics for each model during the evaluation run. items: type: object properties: model_name: type: string description: The name of the model. invocation_count: type: integer description: The number of invocations. prompt_tokens: type: integer description: The number of prompt tokens used. completion_tokens: type: integer description: The number of completion tokens generated. total_tokens: type: integer description: The total number of tokens used. cached_tokens: type: integer description: The number of tokens retrieved from cache. required: - model_name - invocation_count - prompt_tokens - completion_tokens - total_tokens - cached_tokens per_testing_criteria_results: type: array description: Results per testing criteria applied during the evaluation run. items: type: object properties: testing_criteria: type: string description: A description of the testing criteria. passed: type: integer description: Number of tests passed for this criteria. failed: type: integer description: Number of tests failed for this criteria. required: - testing_criteria - passed - failed data_source: type: object description: Information about the run's data source. oneOf: - $ref: '#/components/schemas/CreateEvalJsonlRunDataSource' - $ref: '#/components/schemas/CreateEvalCompletionsRunDataSource' metadata: $ref: '#/components/schemas/Metadata' error: $ref: '#/components/schemas/EvalApiError' required: - object - id - eval_id - status - model - name - created_at - report_url - result_counts - per_model_usage - per_testing_criteria_results - data_source - metadata - error x-oaiMeta: name: The eval run object group: evals example: | { "object": "eval.run", "id": "evalrun_67e57965b480819094274e3a32235e4c", "eval_id": "eval_67e579652b548190aaa83ada4b125f47", "report_url": "https://platform.openai.com/evaluations/eval_67e579652b548190aaa83ada4b125f47?run_id=evalrun_67e57965b480819094274e3a32235e4c", "status": "queued", "model": "gpt-4o-mini", "name": "gpt-4o-mini", "created_at": 1743092069, "result_counts": { "total": 0, "errored": 0, "failed": 0, "passed": 0 }, "per_model_usage": null, "per_testing_criteria_results": null, "data_source": { "type": "completions", "source": { "type": "file_content", "content": [ { "item": { "input": "Tech Company Launches Advanced Artificial Intelligence Platform", "ground_truth": "Technology" } }, { "item": { "input": "Central Bank Increases Interest Rates Amid Inflation Concerns", "ground_truth": "Markets" } }, { "item": { "input": "International Summit Addresses Climate Change Strategies", "ground_truth": "World" } }, { "item": { "input": "Major Retailer Reports Record-Breaking Holiday Sales", "ground_truth": "Business" } }, { "item": { "input": "National Team Qualifies for World Championship Finals", "ground_truth": "Sports" } }, { "item": { "input": "Stock Markets Rally After Positive Economic Data Released", "ground_truth": "Markets" } }, { "item": { "input": "Global Manufacturer Announces Merger with Competitor", "ground_truth": "Business" } }, { "item": { "input": "Breakthrough in Renewable Energy Technology Unveiled", "ground_truth": "Technology" } }, { "item": { "input": "World Leaders Sign Historic Climate Agreement", "ground_truth": "World" } }, { "item": { "input": "Professional Athlete Sets New Record in Championship Event", "ground_truth": "Sports" } }, { "item": { "input": "Financial Institutions Adapt to New Regulatory Requirements", "ground_truth": "Business" } }, { "item": { "input": "Tech Conference Showcases Advances in Artificial Intelligence", "ground_truth": "Technology" } }, { "item": { "input": "Global Markets Respond to Oil Price Fluctuations", "ground_truth": "Markets" } }, { "item": { "input": "International Cooperation Strengthened Through New Treaty", "ground_truth": "World" } }, { "item": { "input": "Sports League Announces Revised Schedule for Upcoming Season", "ground_truth": "Sports" } } ] }, "input_messages": { "type": "template", "template": [ { "type": "message", "role": "developer", "content": { "type": "input_text", "text": "Categorize a given news headline into one of the following topics: Technology, Markets, World, Business, or Sports.\n\n# Steps\n\n1. Analyze the content of the news headline to understand its primary focus.\n2. Extract the subject matter, identifying any key indicators or keywords.\n3. Use the identified indicators to determine the most suitable category out of the five options: Technology, Markets, World, Business, or Sports.\n4. Ensure only one category is selected per headline.\n\n# Output Format\n\nRespond with the chosen category as a single word. For instance: \"Technology\", \"Markets\", \"World\", \"Business\", or \"Sports\".\n\n# Examples\n\n**Input**: \"Apple Unveils New iPhone Model, Featuring Advanced AI Features\" \n**Output**: \"Technology\"\n\n**Input**: \"Global Stocks Mixed as Investors Await Central Bank Decisions\" \n**Output**: \"Markets\"\n\n**Input**: \"War in Ukraine: Latest Updates on Negotiation Status\" \n**Output**: \"World\"\n\n**Input**: \"Microsoft in Talks to Acquire Gaming Company for $2 Billion\" \n**Output**: \"Business\"\n\n**Input**: \"Manchester United Secures Win in Premier League Football Match\" \n**Output**: \"Sports\" \n\n# Notes\n\n- If the headline appears to fit into more than one category, choose the most dominant theme.\n- Keywords or phrases such as \"stocks\", \"company acquisition\", \"match\", or technological brands can be good indicators for classification.\n" } }, { "type": "message", "role": "user", "content": { "type": "input_text", "text": "{{item.input}}" } } ] }, "model": "gpt-4o-mini", "sampling_params": { "seed": 42, "temperature": 1.0, "top_p": 1.0, "max_completions_tokens": 2048 } }, "error": null, "metadata": {} } CreateEvalJsonlRunDataSource: type: object title: JsonlRunDataSource description: > A JsonlRunDataSource object with that specifies a JSONL file that matches the eval properties: type: type: string enum: - jsonl default: jsonl description: The type of data source. Always `jsonl`. x-stainless-const: true source: oneOf: - $ref: '#/components/schemas/EvalJsonlFileContentSource' - $ref: '#/components/schemas/EvalJsonlFileIdSource' required: - type - source x-oaiMeta: name: The file data source object for the eval run configuration group: evals example: | { "type": "jsonl", "source": { "type": "file_id", "id": "file-9GYS6xbkWgWhmE7VoLUWFg" } } CreateEvalCompletionsRunDataSource: type: object title: CompletionsRunDataSource description: > A CompletionsRunDataSource object describing a model sampling configuration. properties: type: type: string enum: - completions default: completions description: The type of run data source. Always `completions`. input_messages: oneOf: - type: object properties: type: type: string enum: - template description: The type of input messages. Always `template`. template: type: array description: >- A list of chat messages forming the prompt or context. May include variable references to the "item" namespace, ie {{item.name}}. items: oneOf: - type: object title: ChatMessage properties: role: type: string description: >- The role of the message (e.g. "system", "assistant", "user"). content: type: string description: The content of the message. x-oaiExpandable: true required: - role - content - $ref: '#/components/schemas/EvalItem' x-oaiExpandable: true x-oaiExpandable: true required: - type - template - type: object properties: type: type: string enum: - item_reference description: The type of input messages. Always `item_reference`. item_reference: type: string description: >- A reference to a variable in the "item" namespace. Ie, "item.name" required: - type - item_reference x-oaiExpandable: true sampling_params: type: object properties: temperature: type: number description: A higher temperature increases randomness in the outputs. default: 1 max_completion_tokens: type: integer description: The maximum number of tokens in the generated output. top_p: type: number description: >- An alternative to temperature for nucleus sampling; 1.0 includes all tokens. default: 1 seed: type: integer description: A seed value to initialize the randomness, during sampling. default: 42 model: type: string description: >- The name of the model to use for generating completions (e.g. "o3-mini"). source: oneOf: - $ref: '#/components/schemas/EvalJsonlFileContentSource' - $ref: '#/components/schemas/EvalJsonlFileIdSource' - $ref: '#/components/schemas/EvalStoredCompletionsSource' required: - type - model - source - input_messages x-oaiMeta: name: The completions data source object used to configure an individual run group: eval runs example: | { "name": "gpt-4o-mini-2024-07-18", "data_source": { "type": "completions", "input_messages": { "type": "item_reference", "item_reference": "item.input" }, "model": "gpt-4o-mini-2024-07-18", "source": { "type": "stored_completions", "model": "gpt-4o-mini-2024-07-18" } } } Metadata: type: object description: > Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard. Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters. additionalProperties: type: string x-oaiTypeLabel: map nullable: true EvalApiError: type: object title: EvalApiError description: | An object representing an error response from the Eval API. properties: code: type: string description: The error code. message: type: string description: The error message. required: - code - message x-oaiMeta: name: The API error object group: evals example: | { "code": "internal_error", "message": "The eval run failed due to an internal error." } EvalJsonlFileContentSource: type: object title: EvalJsonlFileContentSource properties: type: type: string enum: - file_content default: file_content description: The type of jsonl source. Always `file_content`. x-stainless-const: true content: type: array items: type: object properties: item: type: object additionalProperties: true sample: type: object additionalProperties: true required: - item description: The content of the jsonl file. required: - type - content EvalJsonlFileIdSource: type: object title: EvalJsonlFileIdSource properties: type: type: string enum: - file_id default: file_id description: The type of jsonl source. Always `file_id`. x-stainless-const: true id: type: string description: The identifier of the file. required: - type - id EvalItem: title: Item description: An item can either be an input message or an output message. discriminator: propertyName: role oneOf: - type: object title: input_message properties: type: type: string enum: - message description: The type of item, which is always `message`. role: type: string enum: - user - system - developer description: >- The role of the message. One of `user`, `system`, or `developer`. content: type: object properties: type: type: string enum: - input_text description: The type of content, which is always `input_text`. text: type: string description: The text content. required: - type - text required: - type - role - content - type: object title: output_message properties: type: type: string enum: - message description: The type of item, which is always `message`. role: type: string enum: - assistant description: The role of the message. Must be `assistant` for output. content: type: object properties: type: type: string enum: - output_text description: The type of content, which is always `output_text`. text: type: string description: The text content. required: - type - text required: - type - role - content x-oaiExpandable: true EvalStoredCompletionsSource: type: object title: StoredCompletionsRunDataSource description: > A StoredCompletionsRunDataSource configuration describing a set of filters properties: type: type: string enum: - stored_completions default: stored_completions description: The type of source. Always `stored_completions`. x-stainless-const: true metadata: $ref: '#/components/schemas/Metadata' model: type: string nullable: true description: An optional model to filter by (e.g., 'gpt-4o'). created_after: type: integer nullable: true description: An optional Unix timestamp to filter items created after this time. created_before: type: integer nullable: true description: An optional Unix timestamp to filter items created before this time. limit: type: integer nullable: true description: An optional maximum number of items to return. required: - type - metadata - model - created_after - created_before - limit x-oaiMeta: name: >- The stored completions data source object used to configure an individual run group: eval runs example: | { "type": "stored_completions", "model": "gpt-4o", "created_after": 1668124800, "created_before": 1668124900, "limit": 100, "metadata": {} } securitySchemes: ApiKeyAuth: type: http scheme: bearer ````