AI judge

All reports. golden set passed (tuning 9/9 held-out 5/5), 0 new, 0 missed. Advisory: changes neither the run's result nor the probe gate. Why and how: ADR 0020. Raw: verdict.json golden-verdict.json

The run's evidence

Result: known. Model typesafe/jev-1.14, threshold 0.5.

problem_user · chromium, firefox, webkit

Answered by typesafe/jev-1.14-20261014 from evidence-chromium-desktop-1280x720-problem_user/checkout-problem-user-buys-59cd4-k-and-Sauce-Labs-Bike-Light-chromium-desktop-1280x720/receipt.png and the evidence; evidence 2459f84396065f59, request 55a8297da11a8790; 6187 input tokens.

ItemProbabilityFlaggedObservedExpectedKnown issue
receiptTotalscodeyes, critical{"itemTotal":79.96,"tax":6.4,"total":86.36}{"itemTotal":39.98,"tax":3.2,"total":43.18} (R-DONE-9, R-CO2-6, R-CO2-7, R-CO2-8)#2
receiptOrderNumber0.98yes, major{"dateLine":"Order Date October 21, 2026 at 1:50 AM","otherLines":[]}R-DONE-9: The order receipt PDF identifies the order: an order number or id the customer can quote, next to the date, the shipping details and the items.#13
receiptLayout0.06no"the attached image, 1108×1568 px"R-DONE-10: Page 1 of the order receipt PDF reads as a document: its content fits the page, no text is cut off or overlaps other text, and the shop's logo (the "Swag Labs" wordmark) heads it.
receiptDocument0.96yes, critical["Swag Labs","Order Receipt","O R D E R D E TA I L S","Order Date October 21, 2026 at 1:50 AM","S H I P TO","Hailee Muller","63731","I T E M S","DESCRIPTION PRICE","Sauce Labs Backpack $29.99","Sauce Labs Bike Light $9.99","Item total $79.96","Tax $6.40","Total $86.36","Thank you for your order! It has been dispatched, and will arrive just as fast as the pony can get there."]R-DONE-11: From the order receipt PDF alone the seller and the order can be told: the shop's name with a way to contact it (an address, e-mail or phone number), and what was bought.#15

standard_user · chromium

Answered by typesafe/jev-1.14-20261014 from evidence-chromium-desktop-1280x720-standard_user/checkout-standard-user-buy-38727-k-and-Sauce-Labs-Bike-Light-chromium-desktop-1280x720/receipt.png and the evidence; evidence d980aa5b6e73bf71, request 8a8c51900d3cf85a; 5949 input tokens.

ItemProbabilityFlaggedObservedExpectedKnown issue
receiptOrderNumber0.96yes, major{"dateLine":"Order Date October 21, 2026 at 1:49 AM","otherLines":[]}R-DONE-9: The order receipt PDF identifies the order: an order number or id the customer can quote, next to the date, the shipping details and the items.#13
receiptLayout0.04no"the attached image, 1108×1568 px"R-DONE-10: Page 1 of the order receipt PDF reads as a document: its content fits the page, no text is cut off or overlaps other text, and the shop's logo (the "Swag Labs" wordmark) heads it.
receiptDocument0.96yes, major["Swag Labs","Order Receipt","O R D E R D E TA I L S","Order Date October 21, 2026 at 1:49 AM","S H I P TO","Hailee Muller","63731","I T E M S","DESCRIPTION PRICE","Sauce Labs Backpack $29.99","Sauce Labs Bike Light $9.99","Item total $39.98","Tax $3.20","Total $43.18","Thank you for your order! It has been dispatched, and will arrive just as fast as the pony can get there."]R-DONE-11: From the order receipt PDF alone the seller and the order can be told: the shop's name with a way to contact it (an address, e-mail or phone number), and what was bought.#15

standard_user · firefox, webkit

Answered by typesafe/jev-1.14-20261014 from evidence-firefox-desktop-1920x1080-standard_user/checkout-standard-user-buy-38727-k-and-Sauce-Labs-Bike-Light-firefox-desktop-1920x1080/receipt.png and the evidence; evidence b17040f30baec7b3, request 5a7154419040f8c4; 5946 input tokens.

ItemProbabilityFlaggedObservedExpectedKnown issue
receiptOrderNumber0.97yes, major{"dateLine":"Order Date October 21, 2026 at 1:50 AM","otherLines":[]}R-DONE-9: The order receipt PDF identifies the order: an order number or id the customer can quote, next to the date, the shipping details and the items.#13
receiptLayout0.05no"the attached image, 1108×1568 px"R-DONE-10: Page 1 of the order receipt PDF reads as a document: its content fits the page, no text is cut off or overlaps other text, and the shop's logo (the "Swag Labs" wordmark) heads it.
receiptDocument0.96yes, major["Swag Labs","Order Receipt","O R D E R D E TA I L S","Order Date October 21, 2026 at 1:50 AM","S H I P TO","Hailee Muller","63731","I T E M S","DESCRIPTION PRICE","Sauce Labs Backpack $29.99","Sauce Labs Bike Light $9.99","Item total $39.98","Tax $3.20","Total $43.18","Thank you for your order! It has been dispatched, and will arrive just as fast as the pony can get there."]R-DONE-11: From the order receipt PDF alone the seller and the order can be told: the shop's name with a way to contact it (an address, e-mail or phone number), and what was bought.#15

Golden set

Golden set passed: tuning 9/9, held-out 5/5. 1 case(s) to look at.

Every case in detail: golden-verdict.json.