> ## Documentation Index
> Fetch the complete documentation index at: https://openai-hd4n6.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Update an eval

> Update certain properties of an evaluation.



## OpenAPI

````yaml api-definition.yaml post /evals/{eval_id}
openapi: 3.0.0
info:
  title: OpenAI API
  description: >-
    The OpenAI REST API. Please see
    https://platform.openai.com/docs/api-reference for more details.
  version: 2.3.0
  termsOfService: https://openai.com/policies/terms-of-use
  contact:
    name: OpenAI Support
    url: https://help.openai.com/
  license:
    name: MIT
    url: https://github.com/openai/openai-openapi/blob/master/LICENSE
servers:
  - url: https://api.openai.com/v1
security:
  - ApiKeyAuth: []
tags:
  - name: Assistants
    description: Build Assistants that can call models and use tools.
  - name: Audio
    description: Turn audio into text or text into audio.
  - name: Chat
    description: >-
      Given a list of messages comprising a conversation, the model will return
      a response.
  - name: Completions
    description: >-
      Given a prompt, the model will return one or more predicted completions,
      and can also return the probabilities of alternative tokens at each
      position.
  - name: Embeddings
    description: >-
      Get a vector representation of a given input that can be easily consumed
      by machine learning models and algorithms.
  - name: Evals
    description: Manage and run evals in the OpenAI platform.
  - name: Fine-tuning
    description: Manage fine-tuning jobs to tailor a model to your specific training data.
  - name: Batch
    description: Create large batches of API requests to run asynchronously.
  - name: Files
    description: >-
      Files are used to upload documents that can be used with features like
      Assistants and Fine-tuning.
  - name: Uploads
    description: Use Uploads to upload large files in multiple parts.
  - name: Images
    description: Given a prompt and/or an input image, the model will generate a new image.
  - name: Models
    description: List and describe the various models available in the API.
  - name: Moderations
    description: >-
      Given text and/or image inputs, classifies if those inputs are potentially
      harmful.
  - name: Audit Logs
    description: List user actions and configuration changes within this organization.
paths:
  /evals/{eval_id}:
    post:
      tags:
        - Evals
      summary: Update an eval
      description: Update certain properties of an evaluation.
      operationId: updateEval
      parameters:
        - name: eval_id
          in: path
          required: true
          schema:
            type: string
          description: The ID of the evaluation to update.
      requestBody:
        description: Request to update an evaluation
        required: true
        content:
          application/json:
            schema:
              type: object
              properties:
                name:
                  type: string
                  description: Rename the evaluation.
                metadata:
                  $ref: '#/components/schemas/Metadata'
      responses:
        '200':
          description: The updated evaluation
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Eval'
components:
  schemas:
    Metadata:
      type: object
      description: >
        Set of 16 key-value pairs that can be attached to an object. This can be

        useful for storing additional information about the object in a
        structured

        format, and querying for objects via API or the dashboard. 


        Keys are strings with a maximum length of 64 characters. Values are
        strings

        with a maximum length of 512 characters.
      additionalProperties:
        type: string
      x-oaiTypeLabel: map
      nullable: true
    Eval:
      type: object
      title: Eval
      description: |
        An Eval object with a data source config and testing criteria.
        An Eval represents a task to be done for your LLM integration.
        Like:
         - Improve the quality of my chatbot
         - See how well my chatbot handles customer support
         - Check if o3-mini is better at my usecase than gpt-4o
      properties:
        object:
          type: string
          enum:
            - eval
          default: eval
          description: The object type.
          x-stainless-const: true
        id:
          type: string
          description: Unique identifier for the evaluation.
        name:
          type: string
          description: The name of the evaluation.
          example: Chatbot effectiveness Evaluation
        data_source_config:
          type: object
          description: Configuration of data sources used in runs of the evaluation.
          oneOf:
            - $ref: '#/components/schemas/EvalCustomDataSourceConfig'
            - $ref: '#/components/schemas/EvalStoredCompletionsDataSourceConfig'
          x-oaiExpandable: true
        testing_criteria:
          default: eval
          description: A list of testing criteria.
          type: array
          items:
            oneOf:
              - $ref: '#/components/schemas/EvalLabelModelGrader'
              - $ref: '#/components/schemas/EvalStringCheckGrader'
              - $ref: '#/components/schemas/EvalTextSimilarityGrader'
            x-oaiExpandable: true
        created_at:
          type: integer
          description: The Unix timestamp (in seconds) for when the eval was created.
        metadata:
          $ref: '#/components/schemas/Metadata'
        share_with_openai:
          type: boolean
          description: Indicates whether the evaluation is shared with OpenAI.
      required:
        - id
        - data_source_config
        - object
        - testing_criteria
        - name
        - created_at
        - metadata
        - share_with_openai
      x-oaiMeta:
        name: The eval object
        group: evals
        example: |
          {
            "object": "eval",
            "id": "eval_67abd54d9b0081909a86353f6fb9317a",
            "data_source_config": {
              "type": "custom",
              "item_schema": {
                "type": "object",
                "properties": {
                  "label": {"type": "string"},
                },
                "required": ["label"]
              },
              "include_sample_schema": true
            },
            "testing_criteria": [
              {
                "name": "My string check grader",
                "type": "string_check",
                "input": "{{sample.output_text}}",
                "reference": "{{item.label}}",
                "operation": "eq",
              }
            ],
            "name": "External Data Eval",
            "created_at": 1739314509,
            "metadata": {
              "test": "synthetics",
            },
            "share_with_openai": false
          }
    EvalCustomDataSourceConfig:
      type: object
      title: CustomDataSourceConfig
      description: >
        A CustomDataSourceConfig which specifies the schema of your `item` and
        optionally `sample` namespaces.

        The response schema defines the shape of the data that will be:

        - Used to define your testing criteria and

        - What data is required when creating a run
      properties:
        type:
          type: string
          enum:
            - custom
          default: custom
          description: The type of data source. Always `custom`.
          x-stainless-const: true
        schema:
          type: object
          description: |
            The json schema for the run data source items.
            Learn how to build JSON schemas [here](https://json-schema.org/).
          additionalProperties: true
          example: |
            {
              "type": "object",
              "properties": {
                "item": {
                  "type": "object",
                  "properties": {
                    "label": {"type": "string"},
                  },
                  "required": ["label"]
                }
              },
              "required": ["item"]
            }
      required:
        - type
        - schema
      x-oaiMeta:
        name: The eval custom data source config object
        group: evals
        example: |
          {
            "type": "custom",
            "schema": {
              "type": "object",
              "properties": {
                "item": {
                  "type": "object",
                  "properties": {
                    "label": {"type": "string"},
                  },
                  "required": ["label"]
                }
              },
              "required": ["item"]
            }
          }
    EvalStoredCompletionsDataSourceConfig:
      type: object
      title: StoredCompletionsDataSourceConfig
      description: >
        A StoredCompletionsDataSourceConfig which specifies the metadata
        property of your stored completions query.

        This is usually metadata like `usecase=chatbot` or `prompt-version=v2`,
        etc.

        The schema returned by this data source config is used to defined what
        variables are available in your evals.

        `item` and `sample` are both defined when using this data source config.
      properties:
        type:
          type: string
          enum:
            - stored_completions
          default: stored_completions
          description: The type of data source. Always `stored_completions`.
          x-stainless-const: true
        metadata:
          $ref: '#/components/schemas/Metadata'
        schema:
          type: object
          description: |
            The json schema for the run data source items.
            Learn how to build JSON schemas [here](https://json-schema.org/).
          additionalProperties: true
      required:
        - type
        - schema
      x-oaiMeta:
        name: The stored completions data source object for evals
        group: evals
        example: |
          {
            "type": "stored_completions",
            "metadata": {
              "language": "english"
            },
            "schema": {
              "type": "object",
              "properties": {
                "item": {
                  "type": "object"
                },
                "sample": {
                  "type": "object"
                }
              },
              "required": [
                "item",
                "sample"
              }
          }
    EvalLabelModelGrader:
      type: object
      title: LabelModelGrader
      description: >
        A LabelModelGrader object which uses a model to assign labels to each
        item

        in the evaluation.
      properties:
        type:
          description: The object type, which is always `label_model`.
          type: string
          enum:
            - label_model
          x-stainless-const: true
        name:
          type: string
          description: The name of the grader.
        model:
          type: string
          description: >-
            The model to use for the evaluation. Must support structured
            outputs.
        input:
          type: array
          items:
            $ref: '#/components/schemas/EvalItem'
        labels:
          type: array
          items:
            type: string
          description: The labels to assign to each item in the evaluation.
        passing_labels:
          type: array
          items:
            type: string
          description: >-
            The labels that indicate a passing result. Must be a subset of
            labels.
      required:
        - type
        - model
        - input
        - passing_labels
        - labels
        - name
      x-oaiMeta:
        name: The eval label model grader object
        group: evals
        example: |
          {
            "name": "First label grader",
            "type": "label_model",
            "model": "gpt-4o-2024-08-06",
            "input": [
              {
                "type": "message",
                "role": "system",
                "content": {
                  "type": "input_text",
                  "text": "Classify the sentiment of the following statement as one of positive, neutral, or negative"
                }
              },
              {
                "type": "message",
                "role": "user",
                "content": {
                  "type": "input_text",
                  "text": "Statement: {{item.response}}"
                }
              }
            ],
            "passing_labels": [
              "positive"
            ],
            "labels": [
              "positive",
              "neutral",
              "negative"
            ]
          }
    EvalStringCheckGrader:
      type: object
      title: StringCheckGrader
      description: >
        A StringCheckGrader object that performs a string comparison between
        input and reference using a specified operation.
      properties:
        type:
          type: string
          enum:
            - string_check
          description: The object type, which is always `string_check`.
          x-stainless-const: true
        name:
          type: string
          description: The name of the grader.
        input:
          type: string
          description: The input text. This may include template strings.
        reference:
          type: string
          description: The reference text. This may include template strings.
        operation:
          type: string
          enum:
            - eq
            - ne
            - like
            - ilike
          description: >-
            The string check operation to perform. One of `eq`, `ne`, `like`, or
            `ilike`.
      required:
        - type
        - name
        - input
        - reference
        - operation
      x-oaiMeta:
        name: The eval string check grader object
        group: evals
        example: |
          {
            "type": "string_check",
            "name": "Example string check grader",
            "input": "{{sample.output_text}}",
            "reference": "{{item.label}}",
            "operation": "eq"
          }
    EvalTextSimilarityGrader:
      type: object
      title: TextSimilarityGrader
      description: >
        A TextSimilarityGrader object which grades text based on similarity
        metrics.
      properties:
        type:
          type: string
          enum:
            - text_similarity
          default: text_similarity
          description: The type of grader.
          x-stainless-const: true
        name:
          type: string
          description: The name of the grader.
        input:
          type: string
          description: The text being graded.
        reference:
          type: string
          description: The text being graded against.
        pass_threshold:
          type: number
          description: >-
            A float score where a value greater than or equal indicates a
            passing grade.
        evaluation_metric:
          type: string
          enum:
            - fuzzy_match
            - bleu
            - gleu
            - meteor
            - rouge_1
            - rouge_2
            - rouge_3
            - rouge_4
            - rouge_5
            - rouge_l
            - cosine
          description: >-
            The evaluation metric to use. One of `cosine`, `fuzzy_match`,
            `bleu`, `gleu`, `meteor`, `rouge_1`, `rouge_2`, `rouge_3`,
            `rouge_4`, `rouge_5`, or `rouge_l`.
      required:
        - type
        - input
        - reference
        - pass_threshold
        - evaluation_metric
      x-oaiMeta:
        name: The eval text similarity grader object
        group: evals
        example: |
          {
            "type": "text_similarity",
            "name": "example text similarity grader",
            "input": "The graded text",
            "reference": "The reference text",
            "pass_threshold": 0.8,
            "evaluation_metric": "fuzzy_match"
          }
    EvalItem:
      title: Item
      description: An item can either be an input message or an output message.
      discriminator:
        propertyName: role
      oneOf:
        - type: object
          title: input_message
          properties:
            type:
              type: string
              enum:
                - message
              description: The type of item, which is always `message`.
            role:
              type: string
              enum:
                - user
                - system
                - developer
              description: >-
                The role of the message. One of `user`, `system`, or
                `developer`.
            content:
              type: object
              properties:
                type:
                  type: string
                  enum:
                    - input_text
                  description: The type of content, which is always `input_text`.
                text:
                  type: string
                  description: The text content.
              required:
                - type
                - text
          required:
            - type
            - role
            - content
        - type: object
          title: output_message
          properties:
            type:
              type: string
              enum:
                - message
              description: The type of item, which is always `message`.
            role:
              type: string
              enum:
                - assistant
              description: The role of the message. Must be `assistant` for output.
            content:
              type: object
              properties:
                type:
                  type: string
                  enum:
                    - output_text
                  description: The type of content, which is always `output_text`.
                text:
                  type: string
                  description: The text content.
              required:
                - type
                - text
          required:
            - type
            - role
            - content
      x-oaiExpandable: true
  securitySchemes:
    ApiKeyAuth:
      type: http
      scheme: bearer

````