Accuracy
A tool that says “this is new damage” should tell you how often it’s right. Here’s exactly what we’ll publish and how we’ll measure it.
Status: no results yet. Our testing program is in progress. Until results are published here, we won’t quote a detection rate, false-alarm rate or size error anywhere.
What we’ll publish
One table per algorithm version, so a Condition Record can point to the numbers for the exact version that made it.
| Metric | Broken down by | Result |
|---|---|---|
| Recall: share of real damage found, with a 95% Wilson confidence interval | Damage type, size, surface and color, lighting, phone model | Not yet published |
| False alarms: flags that aren’t new damage | Per room, per car | Not yet published |
| Size error: measured vs. true size, in mm (in) | Size range, surface, lighting, phone model | Not yet published |
| Alignment rate: how often the second scan lines up with the first automatically | Home, car | Not yet published |
| Sample size: n for every cell | Every row above | Not yet published |
Small samples will be visible as small samples. Every number will carry its n and its confidence interval.
How we’ll measure
- Known damage. Calibration boards with nail holes, scuffs, a dent, a crack and a stain at measured sizes, scanned under several lighting setups, on several iPhones, at several distances.
- Blind tests in real places. Removable damage decals placed on real walls and cars by someone other than the person scanning.
- Controls. Scan pairs where nothing changed, to count false alarms honestly.
- Ruler truth. Every test defect is measured by hand, so size error is checked against a real number.
Also on this page, once available
- Example misses, with images, so you can see what the app doesn’t catch.
- A change log: what changed in each algorithm version and how it moved the numbers.
- Tamper test results: what happens when someone edits a pixel, swaps in a photo from another scan, or changes the phone’s clock.
What never changes a number
Results are published whether or not they look good. No sponsor or partner can change an accuracy number or a Fair Charge price.
Questions
Want the method in more detail, or to volunteer a room or car for testing? Email support@dinged.app.