Files
CSV opens in any spreadsheet; the two JSON files are what the site itself loads. Everything is regenerated together on every export, so the files always agree with each other and with the pages.
-
prompts.csv
One row per prompt: the request, task, required text.
-
models.csv
One row per model: maker, tier, credits, settings, link.
-
scores.csv
One row per prompt and model: the three automatic scores and the mean judge rank.
-
ratings.csv
One row per task and model (plus the pooled all-video ranking, task video-overall): Plackett–Luce strength, interval, rank, win probability.
-
compass.json
Everything the site shows in one file: models, tasks with rankings and recommendations, prompts with image paths.
-
router.json
What the browser router loads: 384-number embeddings of every prompt, per-task recommendation, keyword fallback, evaluation.
Images sit at the paths named in scores.csv (1024 px WebP with the Picsart AI label). The 512 px thumbnails live under img/thumbs/ at the same path. Raw judge sheets and both full rankings per prompt are available on request (see about).
Data dictionary
| File | Column | Meaning |
|---|---|---|
prompts.csv | id | Prompt id, `<task>-NN`. |
prompts.csv | task | Task id (see models and tasks below). |
prompts.csv | text | The exact request sent to every model. |
prompts.csv | needs_text / expected_text | Whether words must appear in the image, and which words (checked with OCR). |
prompts.csv | aspect_ratio | Aspect ratio requested. |
prompts.csv | n_outputs | Images we have for this prompt. |
models.csv | id | Model id as used in Picsart's AI Playground link. |
models.csv | name / provider | Name shown in Picsart, and the maker. |
models.csv | tier | premium, standard or fast, following Picsart's labels and price. |
models.csv | credits_per_image | Picsart's cost estimate at the benchmark settings, on the data date. |
models.csv | deep_link | AI Playground URL with the model preselected. |
models.csv | params | Extra generation settings sent on every call (JSON). |
scores.csv | clip_score | Prompt match: cosine similarity times 100. Compare only within one prompt. |
scores.csv | aesthetic | Aesthetic predictor, about 1 to 10. |
scores.csv | ocr_exact / ocr_similarity / ocr_text | Text tasks only: every letter right (true/false), 1 minus normalised edit distance, and what the reader saw. Empty for other tasks. |
scores.csv | judge_rank_mean | Mean position of this image across the judge's two shuffled runs (1 = best). |
scores.csv | image | Path of the 1024 px image on this site. |
ratings.csv | task | Task id. `video-overall` is the ranking over every video prompt pooled (each video task alone has 4 prompts, an early result). |
ratings.csv | strength | Plackett–Luce log-strength, centred on 0 within the task. |
ratings.csv | ci_low / ci_high | 95% bootstrap interval over prompts. |
ratings.csv | rank | Position by strength, 1 = best. |
ratings.csv | win_prob_vs_field | Probability of being ranked first among the task's models. |
ratings.csv | clip_mean / aesthetic_mean / ocr_accuracy | Per-model means of the automatic scores on this task; ocr_accuracy is the exact-match share. |
ratings.csv | judge_agreement_tau | Mean Kendall tau between the judge's two runs on this task. |
Task ids and their public names: product-photo (Product photo), logo (Logo with text), poster-flyer (Poster and flyer), portrait-headshot (Portrait and headshot), anime-illustration (Anime and illustration), interior-design (Interior and architecture), fantasy-concept-art (Fantasy and concept art), sticker-icon (Stickers and icons), product-video (Product video), cinematic-scene (Cinematic scene), character-animation (Animated character), people-in-motion (People in motion).
Empty cells mean "not measured" (for example OCR columns on tasks without text), never zero.
credits_per_image in models.csv is credits per output: per image for the image models and per clip for the video models, at the benchmark settings.
How the data was made
- 96 prompts in 12 tasks, fixed before generation (
prompts.csv). - Every prompt sent once to each of 17 models in Picsart's AI Playground, at the settings in
models.csv(image prompts to the 12 image models, video prompts to the 5 video models). One image or clip per prompt, no re-rolls. - Three programs scored every image: prompt match, aesthetic score, and OCR spelling for text tasks (
scores.csv). - An AI judge ranked each prompt's images twice in shuffled order without model names; the mean position is
judge_rank_mean. - Per task, the rankings were fitted with Plackett–Luce and bootstrapped 1,000 times over prompts (
ratings.csv). - Prompts were embedded with a small sentence model for the browser router (
router.json).
The full method, with every limit we know of, is on the method page.
Licence and attribution
The CSV and JSON files on this page are released under Creative Commons Attribution 4.0 International (CC BY 4.0). Copy, share, adapt and build on them, commercially too, as long as you credit the source. A line like this is enough:
Model Compass benchmark data by Vilya Poghosyan, https://vilyapoghosyan.github.io/model-compass/, licensed under CC BY 4.0.
The images and clips are not part of that release; they were generated in Picsart's AI Playground and are shown under Picsart's terms, as evidence for the scores, with the Picsart AI label. Model names belong to their makers. Nothing here is an official Picsart publication.