Documentation

API Reference (v1)

RESTful API for managing prompts, prompt revisions, evaluations, evaluation templates, case runs, and more

Overview

Promposer provides a RESTful API for managing prompts, prompt revisions, evaluations, evaluation templates, case runs, and more. The API is designed for secure, programmatic access to all major features of the Promposer platform.

  • Authorization: All endpoints (except a few public ones) require authentication via API key. See the API Keys section for details.
  • HTTP Methods: The API supports standard REST methods: GET, POST, PUT, and DELETE.
  • Inputs/Outputs: All requests and responses use JSON. Input parameters are provided via JSON body (for POST/PUT), query parameters, or URL path as specified per endpoint. Responses are always JSON objects with a status field (ok or err), and either a data or message field.
  • Error Handling: On error, the API returns a JSON object with status: err, an error message, and a code.

API Keys

Get API key

  1. Sign up for a Promposer account at promposer.ai
  2. Navigate to API Keys in your dashboard
  3. Click "Create New API Key"
  4. Give your key a descriptive name (e.g., "Production App", "Development")
  5. Copy and securely store your API key - it won't be shown again!
⚠️ Security Warning: Never expose your API keys in client-side code, public repositories, or logs. Store them securely as environment variables.

Authentication

Promposer.AI supports two types of authentication headers:

  • Bearer Token: Send your API key in the Authorization header as Bearer <API_KEY>
  • HTTP Basic Auth: Use your API key as the username (password can be blank)

Example (Bearer):

bash 
curl -H "Authorization: Bearer <API_KEY>" https://promposer.ai/api/v1/prompts

Example (Basic):

bash 
curl -u <API_KEY>: https://promposer.ai/api/v1/prompts

Standard rate limits

  • Free Plan: 1,000 requests/hour, 3 API keys max
  • Pro Plan: 10,000 requests/hour, 25 API keys max

Cloud endpoints

The API supports POST, PUT, GET, and DELETE methods for most resources. See each section for details.

Prompts

A Prompt is a container for prompt revisions. The actual prompt content is stored in the associated PromptRevision objects. The Prompt object is used to group revisions together, manage metadata (name, description), and control access.

Prompt fields:

  • id: Unique identifier
  • name: Name of the prompt
  • description: Description
  • dateCreated: Creation timestamp

Create a prompt

POST /v1/prompts

Create a new prompt. Returns the prompt object and its first revision.

Parameters
prompt string required

The initial content of the prompt. This is the main text to be optimized and stored as the first revision.

name string optional

Name for the prompt. If omitted, a name will be generated automatically based on the prompt content.

description string optional

Description for the prompt. If omitted, a description will be generated automatically based on the prompt content.

mode enum (string) optional default: optimize-followup

Controls prompt optimization behavior. Possible values:

  • optimize-followup: Optimize the prompt and create a follow-up revision (default).
  • none: Store the prompt as-is without optimization.
Return
id string

Unique identifier for the prompt.

name string

Name of the prompt.

description string

Description of the prompt.

dateCreated integer

Creation timestamp (Unix epoch seconds).

latestRevision object

The most recent revision of the prompt:

  • id (string): Unique identifier for the revision.
  • content (string): The prompt text/content for this revision.
  • notes (string): Improvement notes for this revision.
  • remarks (string): Remarks or review notes (may be empty).
  • dateCreated (integer): Creation timestamp (Unix epoch seconds).
Example
bash 
curl -X POST https://promposer.ai/api/v1/prompts \ -H "Authorization: Bearer <API_KEY>" \ -H "Content-Type: application/json" \ -d '{"prompt": "Write a story about AI.", "name": "AI Story", "description": "A creative writing prompt."}'
Success Response
JSON 
{ "status": "ok", "data": { "id": "687514635616db903d042808", "name": "AI Story", "description": "A creative writing prompt.", "dateCreated": 1712345678, "latestRevision": { "id": "687514635616db903d04280a", "content": "Write a story about AI.", "notes": "", "remarks": "", "dateCreated": 1712345678 } } }
Error Response
JSON 
{ "status": "err", "message": "Prompt is required", "code": "missing_prompt" }

Update a prompt

PUT /v1/prompts/{id}

Update prompt metadata (name, description). Returns the updated prompt and its latest revision.

Parameters
name string required

Name for the prompt. Cannot be empty.

description string optional

Description for the prompt.

Return
id string

Unique identifier for the prompt.

name string

Name of the prompt.

description string

Description of the prompt.

dateCreated integer

Creation timestamp (Unix epoch seconds).

latestRevision object

The most recent revision of the prompt:

  • id (string): Unique identifier for the revision.
  • content (string): The prompt text/content for this revision.
  • notes (string): Improvement notes for this revision.
  • remarks (string): Remarks or review notes (may be empty).
  • dateCreated (integer): Creation timestamp (Unix epoch seconds).
Example
bash 
curl -X PUT https://promposer.ai/api/v1/prompts/687514635616db903d042808 \ -H "Authorization: Bearer <API_KEY>" \ -H "Content-Type: application/json" \ -d '{"name": "AI Story Updated", "description": "Updated description."}'
Success Response
JSON 
{ "status": "ok", "data": { "id": "687514635616db903d042808", "name": "AI Story Updated", "description": "Updated description.", "dateCreated": 1712345678, "latestRevision": { "id": "687514635616db903d04280a", "content": "Write a story about AI.", "notes": "", "remarks": "", "dateCreated": 1712345678 } } }
Error Response
JSON 
{ "status": "err", "message": "Name is required", "code": "missing_name" }

Retrieve a prompt

GET /v1/prompts/{id}

Retrieve a specific prompt and its revisions.

Parameters
id string required

Unique identifier for the prompt to retrieve (in URL path).

Return
id string

Unique identifier for the prompt.

name string

Name of the prompt.

description string

Description of the prompt.

dateCreated integer

Creation timestamp (Unix epoch seconds).

revisions array

All revisions for this prompt, each with:

  • id (string): Unique identifier for the revision.
  • content (string): The prompt text/content for this revision.
  • notes (string): Improvement notes for this revision.
  • remarks (string): Remarks or review notes (may be empty).
  • dateCreated (integer): Creation timestamp (Unix epoch seconds).
Example
bash 
curl -X GET https://promposer.ai/api/v1/prompts/687514635616db903d042808 \ -H "Authorization: Bearer <API_KEY>"
Success Response
JSON 
{ "status": "ok", "data": { "id": "687514635616db903d042808", "name": "AI Story", "description": "A creative writing prompt.", "dateCreated": 1712345678, "revisions": [ { "id": "687514635616db903d04280a", "content": "Write a story about AI.", "notes": "", "remarks": "", "dateCreated": 1712345678 } ] } }
Error Response
JSON 
{ "status": "err", "message": "Prompt not found", "code": "prompt_not_found" }

List all prompts

GET /v1/prompts

List all prompts for the authenticated user/team.

Parameters (query)
search string optional

Search in prompt name and description.

limit integer optional default: 50

Number of results to return.

offset integer optional default: 0

Pagination offset.

Return
prompts array

List of prompt summaries, each with:

  • id (string): Unique identifier for the prompt.
  • name (string): Name of the prompt.
  • description (string): Description of the prompt.
  • dateCreated (integer): Creation timestamp.
  • revisionCount (integer): Number of revisions for this prompt.
  • latestRevision (object or null): The most recent revision, if any.
total integer

Total number of prompts matching the query.

limit integer

The limit used for this query.

offset integer

The offset used for this query.

filters object

The filters applied to this query (e.g., search string).

Example
bash 
curl -X GET https://promposer.ai/api/v1/prompts \ -H "Authorization: Bearer <API_KEY>"
Success Response
JSON 
{ "status": "ok", "data": { "prompts": [ { "id": "687514635616db903d042808", "name": "AI Story", "description": "A creative writing prompt.", "dateCreated": 1712345678, "revisionCount": 2, "latestRevision": { "id": "687514635616db903d04280a", "dateCreated": 1712345678 } } ], "total": 1, "limit": 50, "offset": 0, "filters": {} } }
Error Response
JSON 
{ "status": "err", "message": "Unauthorized", "code": "unauthorized" }

Delete a prompt

DELETE /v1/prompts/{id}

Delete a prompt and all its revisions.

Parameters
id string required

Unique identifier for the prompt to delete (in URL path).

Return
message string

Success message.

id string

Unique identifier for the deleted prompt.

Example
bash 
curl -X DELETE https://promposer.ai/api/v1/prompts/687514635616db903d042808 \ -H "Authorization: Bearer <API_KEY>"
Success Response
JSON 
{ "status": "ok", "data": { "message": "Prompt and all revisions deleted successfully", "id": "687514635616db903d042808" } }
Error Response
JSON 
{ "status": "err", "message": "Prompt not found", "code": "prompt_not_found" }

Prompt Revisions

A PromptRevision represents a version of a prompt's content. Each prompt can have multiple revisions, which track changes and improvements over time. The actual prompt text is stored in the revision, not the prompt object.

PromptRevision fields:

  • id: Unique identifier
  • promptID: Associated prompt
  • content: The prompt text/content
  • notes: Notes for this revision
  • remarks: Remarks or review notes
  • dateCreated: Creation timestamp
  • dateUpdated: Last update timestamp

Create a prompt revision

POST /v1/prompt-revisions

Create a new revision for a prompt.

Parameters
promptID string required

Unique identifier for the prompt to which this revision belongs.

content string required

The prompt text/content for this revision.

notes string optional

Improvement notes for this revision.

Return
id string

Unique identifier for the revision.

promptID string

Unique identifier for the prompt.

content string

The prompt text/content for this revision.

notes string

Improvement notes for this revision.

dateCreated integer

Creation timestamp (Unix epoch seconds).

dateUpdated integer

Last update timestamp (Unix epoch seconds).

Example
bash 
curl -X POST https://promposer.ai/api/v1/prompt-revisions \ -H "Authorization: Bearer <API_KEY>" \ -H "Content-Type: application/json" \ -d '{"promptID": "687514635616db903d042808", "content": "Improved prompt text."}'
Success Response
JSON 
{ "status": "ok", "data": { "id": "687514635616db903d04280a", "promptID": "687514635616db903d042808", "content": "Improved prompt text.", "notes": "", "dateCreated": 1712345678, "dateUpdated": 1712345678 } }
Error Response
JSON 
{ "status": "err", "message": "Content is required", "code": "missing_content" }

Update a prompt revision

PUT /v1/prompt-revisions/{id}

Update an existing prompt revision.

Parameters
id string required

Unique identifier for the revision to update (in URL path).

content string required

The updated prompt text/content for this revision.

notes string optional

Updated notes for this revision.

Return
id string

Unique identifier for the revision.

promptID string

Unique identifier for the prompt.

content string

The updated prompt text/content for this revision.

notes string

Improvement notes for this revision.

dateCreated integer

Creation timestamp (Unix epoch seconds).

dateUpdated integer

Last update timestamp (Unix epoch seconds).

prompt object

The parent prompt:

  • id (string): Unique identifier for the prompt.
  • name (string): Name of the prompt.
  • description (string): Description of the prompt.
Example
bash 
curl -X PUT https://promposer.ai/api/v1/prompt-revisions/687514635616db903d04280a \ -H "Authorization: Bearer <API_KEY>" \ -H "Content-Type: application/json" \ -d '{"content": "Updated prompt text."}'
Success Response
JSON 
{ "status": "ok", "data": { "id": "687514635616db903d04280a", "promptID": "687514635616db903d042808", "content": "Updated prompt text.", "notes": "", "dateCreated": 1712345678, "dateUpdated": 1712345678, "prompt": { "id": "687514635616db903d042808", "name": "AI Story", "description": "A creative writing prompt." } } }
Error Response
JSON 
{ "status": "err", "message": "Revision not found", "code": "revision_not_found" }

Remark a prompt revision

POST /v1/remark

Given your remarks, this will create a new revision using AI.

Parameters
promptID string required

Unique identifier for the prompt to which this revision belongs.

revisionID string optional

Unique identifier for the revision to base the remark on. If omitted, the latest revision is used.

remarks string optional

Remarks or instructions for the AI to improve the prompt. If empty Promposer will run general optimization.

Return
id string

Unique identifier for the new revision.

promptID string

Unique identifier for the prompt.

content string

The improved prompt text/content for this revision.

notes string

Improvement notes for this revision.

dateCreated integer

Creation timestamp (Unix epoch seconds).

dateUpdated integer

Last update timestamp (Unix epoch seconds).

Example
bash 
curl -X POST https://promposer.ai/api/v1/remark \ -H "Authorization: Bearer <API_KEY>" \ -H "Content-Type: application/json" \ -d '{"promptID": "687514635616db903d042808", "remarks": "Make it more concise."}'
Success Response
JSON 
{ "status": "ok", "data": { "id": "687514635616db903d04280b", "promptID": "687514635616db903d042808", "content": "Concise prompt text.", "notes": "", "remarks": "Make it more concise.", "dateCreated": 1712345678, "dateUpdated": 1712345678 } }
Error Response
JSON 
{ "status": "err", "message": "Remarks are required", "code": "missing_remarks" }

Retrieve a prompt revision

GET /v1/prompt-revisions/{id}

Retrieve a specific prompt revision.

Parameters
id string required

Unique identifier for the revision to retrieve (in URL path).

Return
id string

Unique identifier for the revision.

promptID string

Unique identifier for the prompt.

content string

The prompt text/content for this revision.

notes string

Improvement notes for this revision.

dateCreated integer

Creation timestamp (Unix epoch seconds).

dateUpdated integer

Last update timestamp (Unix epoch seconds).

Example
bash 
curl -X GET https://promposer.ai/api/v1/prompt-revisions/687514635616db903d04280a \ -H "Authorization: Bearer <API_KEY>"
Success Response
JSON 
{ "status": "ok", "data": { "id": "687514635616db903d04280a", "promptID": "687514635616db903d042808", "content": "Improved prompt text.", "notes": "", "remarks": "", "dateCreated": 1712345678, "dateUpdated": 1712345678 } }
Error Response
JSON 
{ "status": "err", "message": "Revision not found", "code": "revision_not_found" }

List all revisions for a prompt

GET /v1/prompt-revisions?promptID={id}

List all revisions for a given prompt.

Parameters (query)
promptID string required

Unique identifier for the prompt whose revisions to list.

limit integer optional default: 50

Number of results to return.

offset integer optional default: 0

Pagination offset.

Return
revisions array

List of revisions for the prompt, each with:

  • id (string): Unique identifier for the revision.
  • promptID (string): Unique identifier for the prompt.
  • content (string): The prompt text/content for this revision.
  • notes (string): Notes for this revision.
  • dateCreated (integer): Creation timestamp.
  • dateUpdated (integer): Last update timestamp.
total integer

Total number of revisions for the prompt.

limit integer

The limit used for this query.

offset integer

The offset used for this query.

Example
bash 
curl -X GET "https://promposer.ai/api/v1/prompt-revisions?promptID=687514635616db903d042808" \ -H "Authorization: Bearer <API_KEY>"
Success Response
JSON 
{ "status": "ok", "data": { "revisions": [ { "id": "687514635616db903d04280a", "promptID": "687514635616db903d042808", "content": "Write a story about AI.", "notes": "", "dateCreated": 1712345678, "dateUpdated": 1712345678 } ], "total": 1, "limit": 50, "offset": 0 } }
Error Response
JSON 
{ "status": "err", "message": "Prompt not found", "code": "prompt_not_found" }

Delete a prompt revision

DELETE /v1/prompt-revisions/{id}

Delete a prompt revision. Cannot delete the last revision of a prompt.

Parameters
id string required

Unique identifier for the revision to delete (in URL path).

Return
message string

Success message.

id string

Unique identifier for the deleted revision.

promptID string

Unique identifier for the prompt.

Example
bash 
curl -X DELETE https://promposer.ai/api/v1/prompt-revisions/687514635616db903d04280a \ -H "Authorization: Bearer <API_KEY>"
Success Response
JSON 
{ "status": "ok", "data": { "message": "Prompt revision deleted successfully", "id": "687514635616db903d04280a", "promptID": "687514635616db903d042808" } }
Error Response
JSON 
{ "status": "err", "message": "Cannot delete the last revision of a prompt", "code": "last_revision_cannot_be_deleted" }

Evaluation Templates

An EvaluationTemplate defines a reusable template for running evaluations. It groups together a prompt, test cases, tools, and contexts, allowing you to quickly create and run evaluations with consistent settings.

EvaluationTemplate fields:

  • id: Unique identifier
  • name: Name of the template
  • promptID: Associated prompt
  • cases: Array of test case definitions
  • tools: Array of tool definitions
  • contexts: Array of context definitions
  • models: Array of model identifiers
  • accessMode: Access mode (e.g., private, team)
  • dateCreated: Creation timestamp
  • dateUpdated: Last update timestamp

Create an evaluation template

POST /v1/evaluation-templates

Create a new evaluation template.

Parameters
name string optional default: "Untitled evaluation"

Name of the template. Defaults to "Untitled evaluation" if omitted.

promptID string required

Unique identifier for the associated prompt.

cases array optional

Array of test case definitions (see CaseDefinition structure).

tools array optional

Array of tool definitions (see ToolDefinition structure).

contexts array optional

Array of context definitions (see ContextDefinition structure).

models array optional

Array of model identifiers to use for the evaluation.

accessMode enum (string) optional default: managed

Access mode for the template. Possible values: managed, private, team.

Return
id string

Unique identifier for the evaluation template.

name string

Name of the template.

promptID string

Unique identifier for the associated prompt.

cases array

Array of test case definitions (see CaseDefinition structure).

tools array

Array of tool definitions (see ToolDefinition structure).

contexts array

Array of context definitions (see ContextDefinition structure).

models array

Array of model identifiers.

accessMode string

Access mode for the template.

dateCreated integer

Creation timestamp (Unix epoch seconds).

dateUpdated integer

Last update timestamp (Unix epoch seconds).

Example
bash 
curl -X POST https://promposer.ai/api/v1/evaluation-templates \ -H "Authorization: Bearer <API_KEY>" \ -H "Content-Type: application/json" \ -d '{"name": "My Template", "promptID": "687514635616db903d042808", "cases": [], "tools": [], "contexts": [], "models": ["openai-gpt-4"], "accessMode": "private"}'
Success Response
JSON 
{ "status": "ok", "data": { "id": "687514635616db903d042810", "name": "My Template", "promptID": "687514635616db903d042808", "cases": [], "tools": [], "contexts": [], "models": ["openai-gpt-4"], "accessMode": "private", "dateCreated": 1712345678, "dateUpdated": 1712345678 } }
Error Response
JSON 
{ "status": "err", "message": "PromptID is required", "code": "missing_prompt_id" }

Update an evaluation template

PUT /v1/evaluation-templates/{id}

Update an existing evaluation template.

Parameters
id string required

Unique identifier for the evaluation template to update (in URL path).

name string optional

Updated name of the template.

promptID string optional

Updated associated prompt.

cases array optional

Updated array of test case definitions.

tools array optional

Updated array of tool definitions.

contexts array optional

Updated array of context definitions.

models array optional

Updated array of model identifiers.

accessMode enum (string) optional

Updated access mode. Possible values: managed, private, team.

Return
id string

Unique identifier for the evaluation template.

name string

Name of the template.

promptID string

Unique identifier for the associated prompt.

cases array

Array of test case definitions.

tools array

Array of tool definitions.

contexts array

Array of context definitions.

models array

Array of model identifiers.

accessMode string

Access mode for the template.

dateCreated integer

Creation timestamp (Unix epoch seconds).

dateUpdated integer

Last update timestamp (Unix epoch seconds).

Example
bash 
curl -X PUT https://promposer.ai/api/v1/evaluation-templates/687514635616db903d042810 \ -H "Authorization: Bearer <API_KEY>" \ -H "Content-Type: application/json" \ -d '{"name": "Updated Template", "cases": [], "tools": [], "contexts": [], "models": ["openai-gpt-4"], "accessMode": "private"}'
Success Response
JSON 
{ "status": "ok", "data": { "id": "687514635616db903d042810", "name": "Updated Template", "promptID": "687514635616db903d042808", "cases": [], "tools": [], "contexts": [], "models": ["openai-gpt-4"], "accessMode": "private", "dateCreated": 1712345678, "dateUpdated": 1712345678 } }
Error Response
JSON 
{ "status": "err", "message": "Evaluation template not found", "code": "template_not_found" }

Retrieve an evaluation template

GET /v1/evaluation-templates/{id}

Retrieve a specific evaluation template.

Parameters
id string required

Unique identifier for the evaluation template to retrieve (in URL path).

Return
id string

Unique identifier for the evaluation template.

name string

Name of the template.

promptID string

Unique identifier for the associated prompt.

cases array

Array of test case definitions.

tools array

Array of tool definitions.

contexts array

Array of context definitions.

models array

Array of model identifiers.

accessMode string

Access mode for the template.

dateCreated integer

Creation timestamp (Unix epoch seconds).

dateUpdated integer

Last update timestamp (Unix epoch seconds).

Example
bash 
curl -X GET https://promposer.ai/api/v1/evaluation-templates/687514635616db903d042810 \ -H "Authorization: Bearer <API_KEY>"
Success Response
JSON 
{ "status": "ok", "data": { "id": "687514635616db903d042810", "name": "My Template", "promptID": "687514635616db903d042808", "cases": [], "tools": [], "contexts": [], "models": ["openai-gpt-4"], "accessMode": "private", "dateCreated": 1712345678, "dateUpdated": 1712345678 } }
Error Response
JSON 
{ "status": "err", "message": "Evaluation template not found", "code": "template_not_found" }

List all evaluation templates

GET /v1/evaluation-templates

List all evaluation templates for the authenticated user/team.

Parameters (query)
search string optional

Search in prompt ID.

promptID string optional

Filter by associated prompt.

limit integer optional default: 50

Number of results to return.

offset integer optional default: 0

Pagination offset.

Return
evaluationTemplates array

List of evaluation template summaries, each with:

  • id (string): Unique identifier for the evaluation template.
  • name (string): Name of the template.
  • promptID (string): Unique identifier for the associated prompt.
  • cases (array): Array of test case definitions.
  • tools (array): Array of tool definitions.
  • contexts (array): Array of context definitions.
  • models (array): Array of model identifiers.
  • accessMode (string): Access mode for the template.
  • dateCreated (integer): Creation timestamp (Unix epoch seconds).
  • dateUpdated (integer): Last update timestamp (Unix epoch seconds).
total integer

Total number of evaluation templates matching the query.

limit integer

The limit used for this query.

offset integer

The offset used for this query.

filters object

The filters applied to this query (e.g., search string, promptID).

Example
bash 
curl -X GET https://promposer.ai/api/v1/evaluation-templates \ -H "Authorization: Bearer <API_KEY>"
Success Response
JSON 
{ "status": "ok", "data": { "evaluationTemplates": [ { "id": "687514635616db903d042810", "name": "My Template", "promptID": "687514635616db903d042808", "cases": [], "tools": [], "contexts": [], "models": ["openai-gpt-4"], "accessMode": "private", "dateCreated": 1712345678, "dateUpdated": 1712345678 } ], "total": 1, "limit": 50, "offset": 0, "filters": {} } }
Error Response
JSON 
{ "status": "err", "message": "Unauthorized", "code": "unauthorized" }

Delete an evaluation template

DELETE /v1/evaluation-templates/{id}

Delete an evaluation template.

Parameters
id string required

Unique identifier for the evaluation template to delete (in URL path).

Return
message string

Success message.

id string

Unique identifier for the deleted evaluation template.

Example
bash 
curl -X DELETE https://promposer.ai/api/v1/evaluation-templates/687514635616db903d042810 \ -H "Authorization: Bearer <API_KEY>"
Success Response
JSON 
{ "status": "ok", "data": { "message": "Evaluation template deleted successfully", "id": "687514635616db903d042810" } }
Error Response
JSON 
{ "status": "err", "message": "Evaluation template not found", "code": "template_not_found" }

Evaluations

An Evaluation represents a run of a prompt (using a template) against a set of test cases, tools, and contexts. Evaluations are created from templates and can be re-run or stopped. Each evaluation produces case runs and results.

Evaluation fields:

  • id: Unique identifier
  • evaluationTemplateID: Associated evaluation template
  • name: Evaluation name
  • promptContent: Prompt content
  • status: Status (e.g., pending, running, completed, aborted)
  • result: Result (e.g., success, failed)
  • caseRuns: Array of case run objects
  • dateCreated: Creation timestamp
  • dateUpdated: Last update timestamp

Create an evaluation

POST /v1/evaluations

Create a new evaluation from a template. Upon creation, the evaluation is automatically run for the first time.

Parameters
evaluationTemplateID string required

Unique identifier for the evaluation template to use.

revisionID string optional

Unique identifier for the prompt revision to use. If omitted, the latest revision is used.

Return
id string

Unique identifier for the evaluation.

promptID string

Unique identifier for the prompt.

revisionID string

Unique identifier for the prompt revision.

evaluationTemplateID string

Unique identifier for the evaluation template.

name string

Name of the evaluation.

status string

Status of the evaluation. Possible values: pending, running, completed, aborted.

result string

Result of the evaluation. Possible values: not_set, passed, partially_passed, failed.

dateCreated integer

Creation timestamp (Unix epoch seconds).

caseRunsCreated array

List of created case runs, each with:

  • id (string): Unique identifier for the case run.
  • model (string): Model identifier for the case run.
  • status (string): Status of the case run.
totalCaseRuns integer

Total number of case runs created.

Example
bash 
curl -X POST https://promposer.ai/api/v1/evaluations \ -H "Authorization: Bearer <API_KEY>" \ -H "Content-Type: application/json" \ -d '{"evaluationTemplateID": "687514635616db903d042810"}'
Success Response
JSON 
{ "status": "ok", "data": { "id": "687514635616db903d042820", "promptID": "687514635616db903d042808", "revisionID": "687514635616db903d04280a", "evaluationTemplateID": "687514635616db903d042810", "name": "Untitled Evaluation", "status": "pending", "result": "not_set", "dateCreated": 1712345678, "caseRunsCreated": [ { "id": "687514635616db903d042830", "model": "openai-gpt-4", "status": "pending" } ], "totalCaseRuns": 1 } }
Error Response
JSON 
{ "status": "err", "message": "Evaluation template not found", "code": "template_not_found" }

Update an evaluation

PUT /v1/evaluations/{id}

Update an evaluation. Used for aborting an evaluation in progress.

Parameters
id string required

Unique identifier for the evaluation to update (in URL path).

action enum (string) required

Action to perform. Only supported value: abort.

abortReason string optional

Reason for aborting the evaluation.

Return
message string

Success message.

id string

Unique identifier for the evaluation.

status string

Status of the evaluation after update.

abortReason string

Reason for aborting the evaluation.

Example
bash 
curl -X PUT https://promposer.ai/api/v1/evaluations/687514635616db903d042820 \ -H "Authorization: Bearer <API_KEY>" \ -H "Content-Type: application/json" \ -d '{"action": "abort", "abortReason": "User requested abort."}'
Success Response
JSON 
{ "status": "ok", "data": { "message": "Evaluation updated successfully", "id": "687514635616db903d042820", "status": "aborted", "abortReason": "User requested abort." } }
Error Response
JSON 
{ "status": "err", "message": "Evaluation not found", "code": "evaluation_not_found" }

Retrieve an evaluation

GET /v1/evaluations/{id}

Retrieve a specific evaluation and its case runs.

Parameters
id string required

Unique identifier for the evaluation to retrieve (in URL path).

Return
id string

Unique identifier for the evaluation.

promptID string

Unique identifier for the prompt.

revisionID string

Unique identifier for the prompt revision.

evaluationTemplateID string

Unique identifier for the evaluation template.

name string

Name of the evaluation.

promptContent string

Prompt content.

cases array

Array of test case definitions.

tools array

Array of tool definitions.

contexts array

Array of context definitions.

models array

Array of model identifiers.

accessMode string

Access mode for the evaluation.

status string

Status of the evaluation. Possible values: pending, running, completed, aborted.

result string

Result of the evaluation. Possible values: not_set, passed, partially_passed, failed.

insights string

Insights summary across all case run assessments.

caseRuns array

List of case runs for this evaluation (see case run structure for details).

dateCreated integer

Creation timestamp (Unix epoch seconds).

dateStarted integer or null

Start timestamp (Unix epoch seconds) or null if not started.

dateCompleted integer or null

Completion timestamp (Unix epoch seconds) or null if not completed.

Example
bash 
curl -X GET https://promposer.ai/api/v1/evaluations/687514635616db903d042820 \ -H "Authorization: Bearer <API_KEY>"
Success Response
JSON 
{ "status": "ok", "data": { "id": "687514635616db903d042820", "promptID": "687514635616db903d042808", "revisionID": "687514635616db903d04280a", "evaluationTemplateID": "687514635616db903d042810", "name": "My template name", "promptContent": "...", "cases": [], "tools": [], "contexts": [], "models": ["openai-gpt-4"], "accessMode": "private", "status": "completed", "result": "passed", "insights": "All test cases passed successfully.", "caseRuns": [], "dateCreated": 1712345678, "dateStarted": 1712345680, "dateCompleted": 1712345700 } }
Error Response
JSON 
{ "status": "err", "message": "Evaluation not found", "code": "evaluation_not_found" }

List all evaluations

GET /v1/evaluations

List all evaluations for the authenticated user/team.

Parameters (query)
status string optional

Filter by evaluation status (pending, running, completed, aborted).

limit integer optional default: 50

Number of results to return.

offset integer optional default: 0

Pagination offset.

Return
evaluations array

List of evaluation summaries, each with basic evaluation details and case run count.

total integer

Total number of evaluations matching the query.

limit integer

The limit used for this query.

offset integer

The offset used for this query.

filters object

The filters applied to this query (e.g., status).

Example
bash 
curl -X GET https://promposer.ai/api/v1/evaluations \ -H "Authorization: Bearer <API_KEY>"
Success Response
JSON 
{ "status": "ok", "data": { "evaluations": [ { "id": "687514635616db903d042820", "promptID": "687514635616db903d042808", "promptName": "AI Story", "revisionID": "687514635616db903d04280a", "evaluationTemplateID": "687514635616db903d042810", "name": "My template name", "status": "completed", "result": "passed", "insights": "All test cases passed successfully.", "dateCreated": 1712345678, "dateStarted": 1712345680, "dateCompleted": 1712345700, "caseRunCount": 3 } ], "total": 1, "limit": 50, "offset": 0, "filters": {} } }
Error Response
JSON 
{ "status": "err", "message": "Unauthorized", "code": "unauthorized" }

Delete an evaluation

DELETE /v1/evaluations/{id}

Delete an evaluation and all related data. Only allowed if the evaluation status is completed or aborted.

Parameters
id string required

Unique identifier for the evaluation to delete (in URL path).

Return
message string

Success message.

id string

Unique identifier for the deleted evaluation.

Example
bash 
curl -X DELETE https://promposer.ai/api/v1/evaluations/687514635616db903d042820 \ -H "Authorization: Bearer <API_KEY>"
Success Response
JSON 
{ "status": "ok", "data": { "message": "Evaluation and all related data deleted successfully", "id": "687514635616db903d042820" } }
Error Response
JSON 
{ "status": "err", "message": "Cannot delete evaluation that is still running", "code": "evaluation_in_progress" }

Stop evaluation

POST /v1/evaluation-stop

Stop a running evaluation by creating a stop signal.

Parameters
evaluationID string required

Unique identifier for the evaluation to stop.

Return
success boolean

Whether the operation was successful.

message string

Success message.

evaluationID string

Unique identifier for the evaluation.

status string

Updated status of the evaluation.

abortReason string

Reason for aborting the evaluation.

Example
bash 
curl -X POST https://promposer.ai/api/v1/evaluation-stop \ -H "Authorization: Bearer <API_KEY>" \ -H "Content-Type: application/json" \ -d '{"evaluationID": "687514635616db903d042820"}'
Success Response
JSON 
{ "status": "ok", "data": { "success": true, "message": "Stop signal created successfully", "evaluationID": "687514635616db903d042820", "status": "aborted", "abortReason": "User requested stop" } }
Error Response
JSON 
{ "status": "err", "message": "Evaluation not found or already completed", "code": "evaluation_not_found_or_completed" }

Run evaluation (re-run)

POST /v1/evaluation-run

Re-run an evaluation by creating/resetting evaluation jobs.

Parameters
evaluationID string required

Unique identifier for the evaluation to re-run.

Return
success boolean

Whether the operation was successful.

message string

Success message.

details object

Details about the re-run operation:

  • evaluationID (string): Unique identifier for the evaluation.
  • deletedCompletedJobs (integer): Number of completed jobs that were deleted.
  • deletedWorkingJobs (integer): Number of working/assessing jobs that were deleted.
  • jobAction (string): Description of the job action taken.
Example
bash 
curl -X POST https://promposer.ai/api/v1/evaluation-run \ -H "Authorization: Bearer <API_KEY>" \ -H "Content-Type: application/json" \ -d '{"evaluationID": "687514635616db903d042820"}'
Success Response
JSON 
{ "status": "ok", "data": { "success": true, "message": "Evaluation re-run started successfully", "details": { "evaluationID": "687514635616db903d042820", "deletedCompletedJobs": 2, "deletedWorkingJobs": 1, "jobAction": "Reset and restarted all evaluation jobs" } } }
Error Response
JSON 
{ "status": "err", "message": "Evaluation not found", "code": "evaluation_not_found" }

Case Runs

A CaseRun represents the execution of a single test case within an evaluation. Case runs are created automatically when an evaluation is started. Each case run contains the results and messages for a specific test case.

CaseRun fields:

  • id: Unique identifier
  • evaluationID: Associated evaluation
  • definition: Case definition object
  • model: Model identifier
  • status: Status. Possible values: pending, running, completed, aborted, error
  • result: Result. Possible values: not_set, passed, failed
  • assessment: Case assessment
  • recommendation: Improvement recommendation
  • attributes: Array of attribute strings
  • hallucinations: Hallucinations level. Possible values: none, low, high
  • avgResponseTime: Average response time in milliseconds
  • lastError: Last error message, if any
  • abortReason: Reason for aborting the case run
  • threadMessages: Array of case thread messages
  • dateCreated: Creation timestamp
  • dateStarted: Start timestamp or null
  • dateUpdated: Last update timestamp or null
  • dateCompleted: Completion timestamp or null

Retrieve a case run

GET /v1/case-runs/{id}

Retrieve a specific case run and its thread messages.

Parameters
id string required

Unique identifier for the case run to retrieve (in URL path).

Return
id string

Unique identifier for the case run.

evaluationID string

Unique identifier for the associated evaluation.

definition object

Case definition for this case run.

model string

Model identifier used for this case run.

status string

Status of the case run. Possible values: pending, running, completed, aborted, error.

result string

Result of the case run. Possible values: not_set, passed, failed.

assessment string

Assessment of the case run result.

recommendation string

Recommendation for improvements.

attributes array

Attributes identified for this case run (e.g., missing_tools, unclear_instructions).

hallucinations string

Hallucinations level. Possible values: none, low, high.

avgResponseTime integer

Average response time in milliseconds.

lastError string

Last error message, if any.

abortReason string

Reason for aborting the case run.

threadMessages array

List of thread messages for this case run (see thread message structure for details).

dateCreated integer

Creation timestamp (Unix epoch seconds).

dateStarted integer or null

Start timestamp (Unix epoch seconds) or null if not started.

dateUpdated integer or null

Last update timestamp (Unix epoch seconds) or null if not updated.

dateCompleted integer or null

Completion timestamp (Unix epoch seconds) or null if not completed.

Example
bash 
curl -X GET https://promposer.ai/api/v1/case-runs/687514635616db903d042830 \ -H "Authorization: Bearer <API_KEY>"
Success Response
JSON 
{ "status": "ok", "data": { "id": "687514635616db903d042830", "evaluationID": "687514635616db903d042820", "definition": { "title": "Test case 1", "scenario": "Test scenario" }, "model": "openai-gpt-4", "status": "completed", "result": "passed", "assessment": "The test case passed successfully", "recommendation": "Consider adding edge cases", "attributes": ["clear_instructions"], "hallucinations": "none", "threadMessages": [], "dateCreated": 1712345678, "dateCompleted": 1712345700 } }
Error Response
JSON 
{ "status": "err", "message": "Case run not found", "code": "case_run_not_found" }

List case runs for an evaluation

GET /v1/case-runs?evaluationID={id}

List all case runs for a given evaluation.

Parameters (query)
evaluationID string optional

Unique identifier for the evaluation to filter by.

status string optional

Filter by case run status (pending, running, completed, aborted, error).

limit integer optional default: 50

Number of results to return.

offset integer optional default: 0

Pagination offset.

Return
caseRuns array

List of case run summaries with basic information and execution details.

total integer

Total number of case runs matching the query.

limit integer

The limit used for this query.

offset integer

The offset used for this query.

filters object

The filters applied to this query (e.g., evaluationID, status).

Example
bash 
curl -X GET "https://promposer.ai/api/v1/case-runs?evaluationID=687514635616db903d042820" \ -H "Authorization: Bearer <API_KEY>"
Success Response
JSON 
{ "status": "ok", "data": { "caseRuns": [ { "id": "687514635616db903d042830", "evaluationID": "687514635616db903d042820", "definition": { "title": "Test case 1", "scenario": "Test scenario" }, "model": "openai-gpt-4", "status": "completed", "result": "passed", "assessment": "The test case passed successfully", "dateCreated": 1712345678, "dateCompleted": 1712345700, "threadMessageCount": 5 } ], "total": 1, "limit": 50, "offset": 0, "filters": { "evaluationID": "687514635616db903d042820" } } }
Error Response
JSON 
{ "status": "err", "message": "Unauthorized", "code": "unauthorized" }

Delete a case run

DELETE /v1/case-runs/{id}

Delete a case run and all related messages.

Parameters
id string required

Unique identifier for the case run to delete (in URL path).

Return
message string

Success message.

id string

Unique identifier for the deleted case run.

Example
bash 
curl -X DELETE https://promposer.ai/api/v1/case-runs/687514635616db903d042830 \ -H "Authorization: Bearer <API_KEY>"
Success Response
JSON 
{ "status": "ok", "data": { "message": "Case run deleted successfully", "id": "687514635616db903d042830" } }
Error Response
JSON 
{ "status": "err", "message": "Case run not found", "code": "case_run_not_found" }

Case Thread Messages

A CaseThreadMessage represents a message or step in the execution of a case run. These messages are created automatically during evaluation execution and provide detailed logs or results for each step.

CaseThreadMessage fields:

  • id: Unique identifier
  • caseRunID: Associated case run
  • role: Role (e.g., system, user, assistant)
  • type: Message type (e.g., input, output, error)
  • content: Message content
  • dateCreated: Creation timestamp

List all case thread messages for a case run

GET /v1/case-thread-messages?caseRunID={id}

List all thread messages for a given case run.

Parameters (query)
caseRunID string required

Unique identifier for the case run whose messages to list.

Return
messages array

List of thread messages for the case run, each with:

  • id (string): Unique identifier for the message.
  • caseRunID (string): Unique identifier for the associated case run.
  • role (string): Message role. Possible values: user, assistant, system, tool, result.
  • type (string): Message type. Possible values: text.
  • content (string): Message content.
  • dateCreated (integer): Creation timestamp (Unix epoch seconds).
total integer

Total number of messages for the case run.

caseRunID string

The case run ID that was queried.

Example
bash 
curl -X GET "https://promposer.ai/api/v1/case-thread-messages?caseRunID=687514635616db903d042830" \ -H "Authorization: Bearer <API_KEY>"
Success Response
JSON 
{ "status": "ok", "data": { "messages": [ { "id": "687514635616db903d042840", "caseRunID": "687514635616db903d042830", "role": "system", "type": "text", "content": "You are a helpful assistant.", "dateCreated": 1712345678 }, { "id": "687514635616db903d042841", "caseRunID": "687514635616db903d042830", "role": "user", "type": "text", "content": "Test message", "dateCreated": 1712345679 } ], "total": 2, "caseRunID": "687514635616db903d042830" } }
Error Response
JSON 
{ "status": "err", "message": "Case run not found", "code": "case_run_not_found" }

Models

The Models endpoint provides information about supported LLM models and their availability based on your account's billing plan. This endpoint returns a list of all supported vendors and their models, indicating which ones are available to your account.

Model fields:

  • id: Unique identifier (format: vendor-model)
  • vendorName: Name of the AI vendor
  • modelName: Name of the model
  • isPremium: Whether the model requires a premium plan
  • available: Whether the model is available to your account

List supported models

GET /v1/models

List all supported LLM models and vendors with their availability for the current account.

Parameters (query)
vendor string optional

Filter models by vendor ID (e.g., "openai", "anthropic").

Return
models array

List of supported models, each with:

  • id (string): Unique identifier for the model (format: vendor-model).
  • vendorName (string): Name of the AI vendor (e.g., "OpenAI", "Anthropic").
  • modelName (string): Name of the model (e.g., "GPT-4", "Claude-3").
  • isPremium (boolean): Whether the model is premium and requires a PRO plan.
  • available (boolean): Whether the model is available to your current account based on your billing plan.
Example
bash 
curl -X GET "https://promposer.ai/api/v1/models?vendor=openai" \ -H "Authorization: Bearer <API_KEY>"
Success Response
JSON 
{ "status": "ok", "data": { "models": [ { "id": "openai-gpt-4.1", "vendorName": "OpenAI", "modelName": "GPT-4.1", "isPremium": false, "available": true }, { "id": "openai-gpt-3.5-turbo", "vendorName": "OpenAI", "modelName": "GPT-3.5 Turbo", "isPremium": false, "available": true }, { "id": "openai-o1-pro", "vendorName": "OpenAI", "modelName": "o1-pro", "isPremium": true, "available": false } ] } }

Generative

These endpoints use AI to generate test cases, contexts, and tool definitions for prompts. They help automate the creation of evaluation assets.

Generate a test case

POST /v1/gen-testcase

Generate a new test case for a prompt using AI.

Parameters
promptID string required

Unique identifier for the prompt to generate a test case for.

revisionID string optional

Unique identifier for the prompt revision to use. If omitted, the latest revision is used.

caselist array optional

List of existing test case names or descriptions to consider.

sourceThread string optional

Source thread to use for generating the test case.

Return
title string

Generated title for the test case.

scenario string

Generated scenario description for the test case.

expectedResult string

Expected result for the test case.

responseThreshold integer

Response threshold in seconds (default: 20).

Example
bash 
curl -X POST https://promposer.ai/api/v1/gen-testcase \ -H "Authorization: Bearer <API_KEY>" \ -H "Content-Type: application/json" \ -d '{"promptID": "687514635616db903d042808", "caselist": ["Test case 1"]}'
Success Response
JSON 
{ "status": "ok", "data": { "title": "Complex AI reasoning test", "scenario": "Test the AI's ability to reason through complex multi-step problems", "expectedResult": "The AI should provide a logical, step-by-step solution", "responseThreshold": 20 } }
Error Response
JSON 
{ "status": "err", "message": "Prompt not found", "code": "prompt_not_found" }

Generate a context

POST /v1/gen-context

Generate a new context for a prompt using AI.

Parameters
promptID string required

Unique identifier for the prompt to generate a context for.

revisionID string optional

Unique identifier for the prompt revision to use. If omitted, the latest revision is used.

cases array optional

Existing test cases to consider.

tools array optional

Existing tools to consider.

contexts array optional

Existing contexts to consider.

Return
name string

Generated name for the context.

content string

Generated content for the context.

Example
bash 
curl -X POST https://promposer.ai/api/v1/gen-context \ -H "Authorization: Bearer <API_KEY>" \ -H "Content-Type: application/json" \ -d '{"promptID": "687514635616db903d042808", "contexts": ["Context 1"]}'
Success Response
JSON 
{ "status": "ok", "data": { "name": "Customer Service Context", "content": "You are a helpful customer service representative for an e-commerce company. Be professional, empathetic, and solution-oriented in your responses." } }
Error Response
JSON 
{ "status": "err", "message": "Prompt not found", "code": "prompt_not_found" }

Generate a tool

POST /v1/gen-tool

Generate a new tool definition for a prompt using AI.

Parameters
promptID string required

Unique identifier for the prompt to generate a tool for.

revisionID string optional

Unique identifier for the prompt revision to use. If omitted, the latest revision is used.

cases array optional

Existing test cases to consider.

tools array optional

Existing tools to consider.

Return
name string

Generated name for the tool.

description string

Generated description for the tool.

parameters object

Generated parameters schema for the tool (JSON object).

Example
bash 
curl -X POST https://promposer.ai/api/v1/gen-tool \ -H "Authorization: Bearer <API_KEY>" \ -H "Content-Type: application/json" \ -d '{"promptID": "687514635616db903d042808", "tools": ["Tool 1"]}'
Success Response
JSON 
{ "status": "ok", "data": { "name": "calculate_math", "description": "Performs mathematical calculations with support for basic arithmetic operations", "parameters": { "type": "object", "properties": { "expression": { "type": "string", "description": "Mathematical expression to evaluate" } }, "required": ["expression"] } } }
Error Response
JSON 
{ "status": "err", "message": "Prompt not found", "code": "prompt_not_found" }

Private endpoints

Endpoints for running server operations in privacy mode. These operations use message queues with short-lived temporary storage, retained only for the duration of the operation.

Thread reviews

Use thread reviews for real-time assessment of chats and agentic interactions.

Review a thread

POST /v1/private/thread-review

Send a full thread for review. Returns thread evaluation result and recommendations.

Note: For maximum accuracy we recommend sending the complete thread, including all instructions, context and chat messages, plus all tool calls and tool responses.

Available on all plans (usage credits required).

Parameters
thread string required

Full thread for review including instructions, context, chat messages, tool calls and tool responses.

expectedResult string optional

Expected outcome the agent was supposed to achieve.

responseThreshold integer optional default: 20

Maximum number of assistant responses (to the user, not tools) allowed to reach the expectedResult. If exceeded, the evaluation result is considered failed.

Return
result enum (string)

Whether the agent succeeded or failed. Possible values:

  • not_set: Couldn't be set due to incomplete chat.
  • failed: Expected result wasn't accomplished.
  • succeed: Expected result was accomplished exactly as described.
assesment string

Overall assessment of the thread.

recommendation string

Recommended improvements.

attributes array

Identified problems (e.g., missing_tools, missing_context, etc.).

hallucinations string

Hallucination level. Possible values: none, low, high.

Example
bash 
curl -X POST https://promposer.ai/api/v1/private/thread-review \ -H "Authorization: Bearer <API_KEY>" \ -H "Content-Type: application/json" \ -d '{ "thread": "System: You are a helpful assistant.\\nUser: Book a flight from NYC to SFO tomorrow.\\nAssistant: Which airline do you prefer?\\nTool(call): search_flights{\"from\":\"NYC\",\"to\":\"SFO\",\"date\":\"2025-08-22\"}\\nTool(result): {...}\\nAssistant: I found several options...", "expectedResult": "Provide at least two bookable flight options with total price and departure time", "responseThreshold": 5 }'
Success Response
JSON 
{ "status": "ok", "data": { "result": "failed", "attributes": ["missing_tools", "unclear_instructions"], "hallucinations": "low", "assesment": "The assistant asked clarifying questions but did not present concrete, bookable options.", "recommendation": "Provide explicit tool outputs (price, time) and ensure final options are summarized clearly." } }
Error Response
JSON 
{ "status": "err", "message": "Thread is required", "code": "thread_required" }

Prompts

Optimize LLM prompts in privacy mode. Processed ephemerally and not stored as cloud objects.

Optimize a prompt

POST /v1/private/optimize

Optimize a prompt and return the improved instructions.

Parameters
prompt string required

Prompt contents to optimize.

Return
prompt string

Optimized prompt text.

Example
bash 
curl -X POST https://promposer.ai/api/v1/private/optimize \ -H "Authorization: Bearer <API_KEY>" \ -H "Content-Type: application/json" \ -d '{"prompt": "You are a helpful assistant. Answer clearly and concisely."}'
Success Response
JSON 
{ "status": "ok", "data": { "prompt": "You are a helpful, concise assistant. Provide accurate answers with minimal verbosity." } }
Error Response
JSON 
{ "status": "err", "message": "Prompt is required", "code": "missing_prompt" }

Remark a prompt

POST /v1/private/remark

Apply your remarks to a prompt to produce a follow-up version with notes.

Parameters
prompt string required

Original prompt content.

remarks string optional

Remarks or instructions for how to improve the prompt. If empty Promposer will run general optimization - like an alias of /v1/private/optimize.

Return
prompt string

Improved instructions.

notes string

Improvement notes.

Example
bash 
curl -X POST https://promposer.ai/api/v1/private/remark \ -H "Authorization: Bearer <API_KEY>" \ -H "Content-Type: application/json" \ -d '{"prompt": "You are a helpful assistant.", "remarks": "Be more specific about formatting and length limits."}'
Success Response
JSON 
{ "status": "ok", "data": { "prompt": "You are a helpful assistant. Answer in 2-3 concise sentences and include bullet points when listing items.", "notes": "Clarified response length and formatting requirements." } }
Error Response
JSON 
{ "status": "err", "message": "Remarks are required", "code": "remarks_required" }

Evaluations

Run evaluations in privacy mode.

Process:

  1. Client sends POST request to /api/v1/private/evaluation-run with the prompt and optional definitions (cases, tools, contexts, models).
  2. The endpoint returns a sessionToken for accessing the private SSE queue (format: <evaluation_id>-<token>).
  3. Client connects to /api/v1/sse?queue=evaluation&sessionToken=[sessionToken] using the same API authorization key used in step 1.
  4. Server sends real-time events to the client.
  5. After the evaluation completes or is aborted, an SSE event of type end-stream is sent and the connection is closed. Temporary objects are then cleaned up.

Run evaluation

POST /v1/private/evaluation-run

Run evaluation. Returns sessionToken for accessing the SSE stream.

Parameters
prompt string required

Prompt contents.

cases array optional

Array of test case definitions (see CaseDefinition structure).

tools array optional

Array of tool definitions (see ToolDefinition structure).

contexts array optional

Array of context definitions (see ContextDefinition structure).

models array optional

Array of model identifiers to use for the evaluation.

accessMode enum (string) optional default: managed

Access mode for the template. Possible values: managed, private, team.

Return
message string

Status message.

sessionToken string

Session token for accessing the SSE stream. Use /api/v1/sse?queue=evaluation&sessionToken=[sessionToken] to get the evaluation progress and results.

Example
bash 
curl -X POST https://promposer.ai/api/v1/private/evaluation-run \ -H "Authorization: Bearer <API_KEY>" \ -H "Content-Type: application/json" \ -d '{ "prompt": "You are a helpful assistant. Answer user questions accurately and concisely.", "cases": [{ "title": "Summarize text", "scenario": "Given a long paragraph, produce a concise summary", "expectedResult": "Return a 2-3 sentence summary capturing key points", "responseThreshold": 5 }], "tools": [{ "name": "search_web", "description": "Search the web for recent information", "parameters": [{"name": "query", "type": "string", "required": true}] }], "contexts": [{ "name": "Company policy", "content": "Always avoid sharing internal IDs." }], "models": ["openai-gpt-4.1"], "accessMode": "managed" }'
Success Response
JSON 
{ "status": "ok", "data": { "message": "Evaluation run initiated successfully.", "sessionToken": "687514635616db903d042820-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" } }
Error Response
JSON 
{ "status": "err", "message": "Prompt is required", "code": "missing_prompt" }

Stop evaluation

POST /v1/private/evaluation-stop

Stop a running private evaluation using the sessionToken obtained from /v1/private/evaluation-run.

Parameters
sessionToken string required

Session token in the format <evaluation_id>-<token>.

Return
success boolean

Whether the operation was successful.

message string

Success message.

status string

Updated status of the evaluation.

abortReason string

Reason for aborting the evaluation.

Example
bash 
curl -X POST https://promposer.ai/api/v1/private/evaluation-stop \ -H "Authorization: Bearer <API_KEY>" \ -H "Content-Type: application/json" \ -d '{"sessionToken": "687514635616db903d042820-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}'
Success Response
JSON 
{ "status": "ok", "data": { "success": true, "message": "Evaluation stopped successfully", "status": "aborted", "abortReason": "Aborted by user." } }

Generative

These endpoints use AI to generate test cases, contexts, and tool definitions for prompts in privacy mode. They help automate the creation of evaluation assets.

Generate a test case

POST /v1/private/gen-testcase

Generate a new test case for a prompt using AI.

Parameters
prompt string required

Prompt contents.

caselist array optional

List of existing test case names or descriptions to consider.

transcript string optional

Transcript (conversation) to use for generating the test case.

Return
title string

Generated title for the test case.

scenario string

Generated scenario description for the test case.

expectedResult string

Expected result for the test case.

responseThreshold integer

Response threshold - maximum number of assistant responses to reach the expectedResult (default: 20).

Example
bash 
curl -X POST https://promposer.ai/api/v1/private/gen-testcase \ -H "Authorization: Bearer <API_KEY>" \ -H "Content-Type: application/json" \ -d '{ "prompt": "You are a helpful assistant that answers programming questions.", "caselist": ["Summarize an article", "Explain a code snippet"], "transcript": "User asked for an explanation of a recursive function, assistant replied with details." }'

Generate a context

POST /v1/private/gen-context

Generate a new context for a prompt using AI.

Parameters
prompt string required

Prompt contents.

cases array optional

Existing test cases to consider.

tools array optional

Existing tools to consider.

contexts array optional

Existing contexts to consider.

Return
name string

Generated name for the context.

content string

Generated content for the context.

Example
bash 
curl -X POST https://promposer.ai/api/v1/private/gen-context \ -H "Authorization: Bearer <API_KEY>" \ -H "Content-Type: application/json" \ -d '{ "prompt": "You are a financial assistant.", "cases": [{"title": "Calculate ROI", "scenario": "Given inputs, compute ROI", "expectedResult": "ROI% value"}], "tools": [{"name": "lookup_rates", "description": "Lookup current interest rates"}], "contexts": [{"name": "Compliance", "content": "Do not provide investment advice."}] }'

Generate a tool

POST /v1/private/gen-tool

Generate a new tool definition for a prompt using AI.

Parameters
prompt string required

Prompt contents.

cases array optional

Existing test cases to consider.

tools array optional

Existing tools to consider.

Return
name string

Generated name for the tool.

description string

Generated description for the tool.

parameters object

Generated parameters schema for the tool (JSON object).

Example
bash 
curl -X POST https://promposer.ai/api/v1/private/gen-tool \ -H "Authorization: Bearer <API_KEY>" \ -H "Content-Type: application/json" \ -d '{ "prompt": "You are a research assistant.", "cases": [{"title": "Find recent papers", "scenario": "Search for 3 recent ML papers"}], "tools": [] }'

Server sent events (SSE)

Listen for event notifications from the server. SSE allows clients to receive real-time updates by keeping an HTTP connection open. Users can connect using the same API authorization headers as for other endpoints.

  • Endpoint: /api/v1/sse
  • Auth: Use your API key as with other endpoints.
  • How it works: The server sends events as text streams. Each message is formatted as follows:
SSE Format 
id: 687514635616db903d042880 data: {"type": "update", "data": ["Prompt:687514635616db903d042808", "Evaluation:687514635616db903d042820"]}

Listen for server events

GET /v1/sse

Establish a Server-Sent Events connection to receive real-time updates.

Parameters
Authorization header required

API key authentication using Bearer token format.

Return
Stream text/event-stream

Continuous stream of server-sent events, each containing:

  • id: Event identifier (incremental).
  • data: JSON object with event data.
  • retry: Retry interval in milliseconds (optional).
Event Data object

Event data contains:

  • type (string): Event type (e.g., "update").
  • data (array): Array of object references in format "ObjectType:id".
Example
bash 
curl -N -H "Authorization: Bearer <API_KEY>" https://promposer.ai/api/v1/sse
Stream Format
SSE Stream 
id: 687514635616db903d042880 data: {"type": "update", "data": ["Prompt:687514635616db903d042808", "Evaluation:687514635616db903d042820"]} retry: 5000
Stream Example
SSE Stream 
id: 687514635616db903d042880 data: {"type": "update", "data": ["Prompt:687514635616db903d042808", "Evaluation:687514635616db903d042820"]} id: 687514635616db903d042881 data: {"type": "update", "data": ["Prompt:687514635616db903d042809"]}

Update Notifications

Update notifications are received when an object is updated. The notification type is update and data is an array of object strings in the format object:id.

Example cURL request:

bash 
curl -N -H "Authorization: Bearer <API_KEY>" https://promposer.ai/api/v1/sse

Example streamed response:

SSE Stream 
id: 687514635616db903d042880 data: {"type": "update", "data": ["Prompt:687514635616db903d042808", "Evaluation:687514635616db903d042820"]} id: 687514635616db903d042881 data: {"type": "update", "data": ["Prompt:687514635616db903d042809"]}

Object Notifications

Object notifications are received on private streams. The notification type is object and data is the object payload.

Example cURL request:

bash 
curl -N -H "Authorization: Bearer <API_KEY>" https://promposer.ai/api/v1/sse?queue=evaluation&sessionToken=[session_token]

Example streamed response:

SSE Stream 
id: 687514635616db903d042880 data: {"type": "object", "data": {"::": "Evaluation", "id": "687514635616db903d042820", "promptContent": "You are a helpful assistant...", "models": ["openai-gpt-4.1"], "status": "running", "result": "not_set", "dateCreated": 1712345678}} id: 687514635616db903d042881 data: {"type": "object", "data": {"::": "CaseRun", "id": "687514635616db903d042830", "model": "openai-gpt-4.1", "status": "running", "definition": {"title": "Summarize text"}}} id: 687514635616db903d042882 data: {"type": "object", "data": {"::": "CaseThreadMessage", "id": "687514635616db903d042840", "caseRunID": "687514635616db903d042830", "role": "assistant", "type": "text", "content": "Here is the summary..."}} id: 687514635616db903d042883 data: {"type": "end-stream"}

Queues

There are two queues for streaming events - general (default) and evaluation.

General queue

The general queue streams all cloud events related to the current user and team.

Endpoint:

  • https://promposer.ai/api/v1/sse
  • https://promposer.ai/api/v1/sse?queue=general
Evaluation queue

The evaluation queue streams events related to evaluations in privacy mode. It requires an additional argument sessionToken, which is acquired on evaluation run.

Endpoint:

  • https://promposer.ai/api/v1/sse?queue=evaluation&sessionToken=[session_token]

Need help? Check out our How to Use guide or contact support.