← Back to the article: Understanding Jev and System 1 Models
Warning: AI-generated test run. An AI assistant wrote the scripts, ran them and built this page as a quick illustration of Jev's image support. Seven photos is not an evaluation, so don't read the scores as a benchmark.
AI-generated test run: Jev image tests
jev-1.14.0 with header TypeSafe-Danger-Zone-Images: true. Photos from Wikimedia Commons, credited at the bottom. Requests show base64 as <file>.
1. Coder vs executive (Choice with reference images)
Each option's description carries a reference image. One request per test photo.
| Input | Output |
|---|---|
{
"model": "jev-1.14.0",
"state": {
"photo": "a1b2-photo-t1"
},
"assets": {
"a1b2-photo-t1": {
"type": "image",
"format": "jpg",
"data_base64": "<t1.jpg>"
},
"c0d3-7a1e-ref-coder": {
"type": "image",
"format": "jpg",
"data_base64": "<ref-coder.jpg>"
},
"e8ec-41b9-ref-exec": {
"type": "image",
"format": "jpg",
"data_base64": "<ref-exec.jpg>"
}
},
"questions": {
"role": {
"type": "choice",
"instructions": "Which role does the person in `photo` most look like?",
"criteria": {
"coder": "Looks like this example: c0d3-7a1e-ref-coder (casual clothes, hoodie, at a computer)",
"executive": "Looks like this example: e8ec-41b9-ref-exec (suit, tie, formal)"
}
}
}
} | coder {
"model": "jev-1.14.0",
"answers": {
"role": {
"type": "choice",
"choice": "coder",
"confidence": 0.98,
"probabilities": {
"coder": 0.99,
"executive": 0.01
}
}
},
"usage": {
"input_tokens": 6748,
"output_tokens": 33
}
} |
{
"model": "jev-1.14.0",
"state": {
"photo": "a1b2-photo-t2"
},
"assets": {
"a1b2-photo-t2": {
"type": "image",
"format": "jpg",
"data_base64": "<t2.jpg>"
},
"c0d3-7a1e-ref-coder": {
"type": "image",
"format": "jpg",
"data_base64": "<ref-coder.jpg>"
},
"e8ec-41b9-ref-exec": {
"type": "image",
"format": "jpg",
"data_base64": "<ref-exec.jpg>"
}
},
"questions": {
"role": {
"type": "choice",
"instructions": "Which role does the person in `photo` most look like?",
"criteria": {
"coder": "Looks like this example: c0d3-7a1e-ref-coder (casual clothes, hoodie, at a computer)",
"executive": "Looks like this example: e8ec-41b9-ref-exec (suit, tie, formal)"
}
}
}
} | coder {
"model": "jev-1.14.0",
"answers": {
"role": {
"type": "choice",
"choice": "coder",
"confidence": 0.86,
"probabilities": {
"coder": 0.93,
"executive": 0.07
}
}
},
"usage": {
"input_tokens": 6600,
"output_tokens": 33
}
} |
{
"model": "jev-1.14.0",
"state": {
"photo": "a1b2-photo-t3"
},
"assets": {
"a1b2-photo-t3": {
"type": "image",
"format": "jpg",
"data_base64": "<t3.jpg>"
},
"c0d3-7a1e-ref-coder": {
"type": "image",
"format": "jpg",
"data_base64": "<ref-coder.jpg>"
},
"e8ec-41b9-ref-exec": {
"type": "image",
"format": "jpg",
"data_base64": "<ref-exec.jpg>"
}
},
"questions": {
"role": {
"type": "choice",
"instructions": "Which role does the person in `photo` most look like?",
"criteria": {
"coder": "Looks like this example: c0d3-7a1e-ref-coder (casual clothes, hoodie, at a computer)",
"executive": "Looks like this example: e8ec-41b9-ref-exec (suit, tie, formal)"
}
}
}
} | coder {
"model": "jev-1.14.0",
"answers": {
"role": {
"type": "choice",
"choice": "coder",
"confidence": 1.0,
"probabilities": {
"coder": 1.0,
"executive": 0.0
}
}
},
"usage": {
"input_tokens": 6378,
"output_tokens": 33
}
} |
{
"model": "jev-1.14.0",
"state": {
"photo": "a1b2-photo-t4"
},
"assets": {
"a1b2-photo-t4": {
"type": "image",
"format": "jpg",
"data_base64": "<t4.jpg>"
},
"c0d3-7a1e-ref-coder": {
"type": "image",
"format": "jpg",
"data_base64": "<ref-coder.jpg>"
},
"e8ec-41b9-ref-exec": {
"type": "image",
"format": "jpg",
"data_base64": "<ref-exec.jpg>"
}
},
"questions": {
"role": {
"type": "choice",
"instructions": "Which role does the person in `photo` most look like?",
"criteria": {
"coder": "Looks like this example: c0d3-7a1e-ref-coder (casual clothes, hoodie, at a computer)",
"executive": "Looks like this example: e8ec-41b9-ref-exec (suit, tie, formal)"
}
}
}
} | executive {
"model": "jev-1.14.0",
"answers": {
"role": {
"type": "choice",
"choice": "executive",
"confidence": 1.0,
"probabilities": {
"coder": 0.0,
"executive": 1.0
}
}
},
"usage": {
"input_tokens": 6156,
"output_tokens": 35
}
} |
{
"model": "jev-1.14.0",
"state": {
"photo": "a1b2-photo-t5"
},
"assets": {
"a1b2-photo-t5": {
"type": "image",
"format": "jpg",
"data_base64": "<t5.jpg>"
},
"c0d3-7a1e-ref-coder": {
"type": "image",
"format": "jpg",
"data_base64": "<ref-coder.jpg>"
},
"e8ec-41b9-ref-exec": {
"type": "image",
"format": "jpg",
"data_base64": "<ref-exec.jpg>"
}
},
"questions": {
"role": {
"type": "choice",
"instructions": "Which role does the person in `photo` most look like?",
"criteria": {
"coder": "Looks like this example: c0d3-7a1e-ref-coder (casual clothes, hoodie, at a computer)",
"executive": "Looks like this example: e8ec-41b9-ref-exec (suit, tie, formal)"
}
}
}
} | executive {
"model": "jev-1.14.0",
"answers": {
"role": {
"type": "choice",
"choice": "executive",
"confidence": 1.0,
"probabilities": {
"coder": 0.0,
"executive": 1.0
}
}
},
"usage": {
"input_tokens": 6526,
"output_tokens": 35
}
} |
{
"model": "jev-1.14.0",
"state": {
"photo": "a1b2-photo-t6"
},
"assets": {
"a1b2-photo-t6": {
"type": "image",
"format": "jpg",
"data_base64": "<t6.jpg>"
},
"c0d3-7a1e-ref-coder": {
"type": "image",
"format": "jpg",
"data_base64": "<ref-coder.jpg>"
},
"e8ec-41b9-ref-exec": {
"type": "image",
"format": "jpg",
"data_base64": "<ref-exec.jpg>"
}
},
"questions": {
"role": {
"type": "choice",
"instructions": "Which role does the person in `photo` most look like?",
"criteria": {
"coder": "Looks like this example: c0d3-7a1e-ref-coder (casual clothes, hoodie, at a computer)",
"executive": "Looks like this example: e8ec-41b9-ref-exec (suit, tie, formal)"
}
}
}
} | coder {
"model": "jev-1.14.0",
"answers": {
"role": {
"type": "choice",
"choice": "coder",
"confidence": 0.98,
"probabilities": {
"coder": 0.99,
"executive": 0.01
}
}
},
"usage": {
"input_tokens": 6378,
"output_tokens": 33
}
} |
{
"model": "jev-1.14.0",
"state": {
"photo": "a1b2-photo-t7"
},
"assets": {
"a1b2-photo-t7": {
"type": "image",
"format": "jpg",
"data_base64": "<t7.jpg>"
},
"c0d3-7a1e-ref-coder": {
"type": "image",
"format": "jpg",
"data_base64": "<ref-coder.jpg>"
},
"e8ec-41b9-ref-exec": {
"type": "image",
"format": "jpg",
"data_base64": "<ref-exec.jpg>"
}
},
"questions": {
"role": {
"type": "choice",
"instructions": "Which role does the person in `photo` most look like?",
"criteria": {
"coder": "Looks like this example: c0d3-7a1e-ref-coder (casual clothes, hoodie, at a computer)",
"executive": "Looks like this example: e8ec-41b9-ref-exec (suit, tie, formal)"
}
}
}
} | executive {
"model": "jev-1.14.0",
"answers": {
"role": {
"type": "choice",
"choice": "executive",
"confidence": 1.0,
"probabilities": {
"coder": 0.0,
"executive": 1.0
}
}
},
"usage": {
"input_tokens": 6748,
"output_tokens": 35
}
} |
2. Box
Box finds one named target. A Noul question checks the target exists, since Box always returns a box.
| Input | Output |
|---|---|
{
"model": "jev-1.14.0",
"state": {
"photo": "b0c5-9f2d-photo"
},
"assets": {
"b0c5-9f2d-photo": {
"type": "image",
"format": "jpg",
"data_base64": "<t1.jpg>"
}
},
"questions": {
"where": {
"type": "box",
"instructions": "Find the logo printed on the hoodie in `photo`."
},
"present": {
"type": "noul",
"instructions": "`photo` contains the logo printed on the hoodie."
}
}
} | 
{
"model": "jev-1.14.0",
"answers": {
"where": {
"type": "box",
"box": {
"top": 0.65,
"bottom": 0.82,
"left": 0.38,
"right": 0.61
},
"confidence": 0.88
},
"present": {
"type": "noul",
"noul": 0.97
}
},
"usage": {
"input_tokens": 2534,
"output_tokens": 36
}
} |
Photo credits
All from Wikimedia Commons. Photos are resized, and t1 is also shown with a box drawn on it.
ref-coder.jpgProgrammer writing code with Unit Tests by Joonspoon, CC BY-SA 4.0ref-exec.jpgPortrait of a man in a suit and tie by Edmund F. Arras, Public domaint1.jpgYoung man wearing a red hoodie by Hoodype, CC BY-SA 2.0t2.jpgHackathon participants by Ciell, CC BY-SA 4.0t3.jpgProgrammers by Lbronn, CC BY-SA 4.0t4.jpgMan in a suit by Edmund F. Arras, Public domaint5.jpgA businessman of Taiwan in the 1940s by Unknown author, Public domaint6.jpgMan Hoodie Salamanca by Comete El Coco, CC0t7.jpgPortrait of a man in a suit by Edmund F. Arras, Public domain
.jpg){kind=link}
.jpg){kind=link}
