Sample readiness
The experiment finished and its verdict is available.
These checks apply to every experiment for conversation-recap, whatever the source model. Changes apply to new runs and start a fresh evidence pool, so earlier runs are not mixed with runs under different rules.
Added to the judge's instructions. A candidate that violates them is judged not equivalent.
Checked without the judge. A candidate that fails it counts as not equivalent and skips the judge call. If the original output fails it too, the schema is likely wrong, so the judge decides and the miss is only reported.
Test whether conversation-recap answers as well with a shorter prompt: less conversation history, or a shorter system prompt that the judge model proposes. TrAIce replays 30 captured requests on gpt-5.5 with the shorter prompt, and the judge compares each answer with the one your application got. Nothing changes in your traffic.
This demo is read-only. Create a workspace to run a trial.
Once a week, TrAIce groups up to 500 of this workload's sampled requests by what they ask for and shows what each group costs. The judge model on Google Vertex AI names each group from a few of its requests. Names are kept only as long as the samples they came from (at most 14 days); the counts and costs stay.
No analysis yet. The first one runs within a day or so.
These complete, unexpired prompt and response pairs can be selected for an experiment. Opening sample content is audited.
| Case | Captured | Prompt tokens | Response tokens | Production latency | Expires | Availability | Sample details |
|---|---|---|---|---|---|---|---|
| 1 | Nov 8, 08:57 PM |
| 45409 |
| 9605 |
| 478 ms |
| Nov 22, 08:58 PM |
| Used in latest experiment |
| View sample |
| 2 | Nov 8, 08:56 PM | 132117 | 22651 | 797 ms | Nov 22, 08:58 PM | Used in latest experiment | View sample |
| 3 | Nov 8, 08:55 PM | 221637 | 49144 | 1,119 ms | Nov 22, 08:58 PM | Used in latest experiment | View sample |
| 4 | Nov 8, 08:54 PM | 79573 | 14296 | 361 ms | Nov 22, 08:58 PM | Used in latest experiment | View sample |
| 5 | Nov 8, 08:53 PM | 25948 | 4825 | 108 ms | Nov 22, 08:58 PM | Used in latest experiment | Close |
Sample case 5 evidenceCaptured prompt and source response available for the next experiment. used Captured promptNorthstar support case 5: resolve the conversation-recap request with the account policy and cited help-center context. Original responsegpt-5.5 Resolved conversation-recap case 5 with the correct policy, next step, and customer-safe explanation. | |||||||
| 6 | Nov 8, 08:52 PM | 49733 | 9159 | 873 ms | Nov 22, 08:58 PM | Used in latest experiment | View sample |
| 7 | Nov 8, 08:51 PM | 112440 | 21489 | 1,588 ms | Nov 22, 08:58 PM | Used in latest experiment | View sample |
| 8 | Nov 8, 08:50 PM | 243260 | 46910 | 2,502 ms | Nov 22, 08:58 PM | Used in latest experiment | View sample |
| 9 | Nov 8, 08:49 PM | 86493 | 13581 | 954 ms | Nov 22, 08:58 PM | Used in latest experiment | View sample |
| 10 | Nov 8, 08:48 PM | 28543 | 5763 | 371 ms | Nov 22, 08:58 PM | Used in latest experiment | View sample |