How we measure, and where it can be wrong

The tasks

We did not invent the tasks. Other AI design tools sell single-purpose products: an "AI product photo generator", an "AI logo maker", an "AI poster generator". We grouped those into clusters and kept the 12 that came up again and again. Each task is here because people already search for a tool that does exactly that.

2 tasks need words inside the image (a brand name, a headline). Spelling can be checked objectively there, so those tasks carry an extra score.

The prompts

Ten prompts per image task and 4 per video task, 96 in total, written the way people type. Some are five sloppy words; some are detailed briefs with subject, lighting, materials and style.

Each task spreads its prompts over the sub-uses the single-task tools sell separately. For product photos that means cosmetics, shoes, drinks, electronics, packaging, food and more, so no niche decides the ranking.

• No real brands, real people or copyrighted characters, and nothing a model would refuse: a refusal would decide the ranking instead of quality.
• Text prompts quote the exact words to show, in plain letters. Every word is unique across the set, so OCR can match a word to one prompt only.
• The prompt set was fixed and checked by tests before any image was generated, and never rewritten after we saw who won.

All prompts are in the data download.

The models

Only models you can pick yourself in Picsart's AI Playground, and we recorded where we saw each one offered. A model that exists only behind an API, or that we could not confirm in the picker, is not in the table.

We stopped at 17 models to keep every prompt affordable to run through all of them.

Tier follows Picsart's own labelling: premium models carry the premium badge or cost five credits or more. Credits were read from Picsart's cost estimate on 2026-10-08 and can change.

Generation settings

Same rules for every model:

• Resolution 1K (about 1024 pixels on the long side) wherever offered; models without the option ran at their default size.
• One image per prompt. No best of four, no re-rolls: the first result is what we scored.
• Defaults otherwise. No style presets, negative prompts or reference images, and the aspect ratio the prompt asks for. What a new user gets.
• Every output is stored with its settings, cost and time, so any row in the data can be matched to an image on the site.

A model that shines with careful tuning may rank lower here. We think that is fair: this site is for people choosing a model, not tuning one.

Three automatic scores

Three programs score every image: quick, repeatable, blind to who made it, and each with a known blind spot. They are supporting columns; the ranking itself comes from the judge below.

Does it show what was asked? (CLIP)

A vision model that puts images and sentences in one space. The score is cosine similarity times 100; about 28 to 38 means a good match. Only outputs of the same prompt are compared, because long prompts score lower for every model.

• Cannot read letters.
• Treats the prompt as a bag of words ("a cat chasing a dog" scores like the reverse).
• Weak on counts and positions; rewards literal, centred, saturated pictures.

Does it look good? (aesthetic score)

A predictor trained on human 1-to-10 ratings of photos and AI images; most good generations land between 5.5 and 7.

• Never sees the prompt, so a beautiful picture of the wrong thing scores high.
• Loves painterly, warm, detailed images and marks down clean, flat, minimal design even when that was asked. For logos, posters and icons we weight it lightly.

Is the text spelled right? (OCR)

For the 2 text tasks only, on-device text recognition reads the image and we compare it with the words the prompt required. Spelling correction is off on purpose: we want the letters the model drew.

We report the share of prompts with every letter right (exact match) and a softer similarity that separates "one letter off" from gibberish. Case, quotes and punctuation are ignored.

• Decorative, curved, tiny or low-contrast lettering can be misread even when a person reads it fine.
• Extra invented text around the right words is not penalised here; the judge catches it.
• The reader often fails on script, outlined or curved lettering, so “text exact” undercounts correct text. In spot checks, several logos it marked wrong read perfectly to a person. Treat it as a floor, not a verdict.
• Every model faces the same reader on the same prompts, so the comparison stays fair, but single images carry noise. The judge also reads the text and writes down what it saw.

The AI judge

The ranking comes from an AI judge. It sees all of a prompt's images at once, as numbered tiles on one sheet, and ranks them best to worst. It is told the request and, for text tasks, the exact words that must appear. It judges faithfulness first, then text, then image quality and artefacts, then looks.

Two things make it hard to fool

• It never sees model names, prices or any hint of where a tile came from. Tiles are only numbers.
• Every prompt is judged twice, in a different shuffled order. Judges favour the first or centre tile; two shuffles cancel most of that, and comparing the runs shows how stable the judge is.

Agreement between the two runs is 0.59 on average (Kendall tau: 1 is identical order, 0 is no relation). Per task it ranges from 0.39 to 0.85; see the task table above. Low agreement means the judge itself is unsure, and the error bars are wider for it.

The judge is one AI model looking at a contact sheet. It shares some taste with the aesthetic score, may prefer a certain kind of image, and can misread text. So its ranking sits next to the objective OCR check, and real people spot-check it.

Video

4 video tasks, 16 prompts, 5 video models, all 9:16. Same rules as images: every video model gets the same prompts, defaults only, one clip per prompt, no re-rolls. What differs, and what that means:

• Frames, not motion. CLIP and the aesthetic score run on three frames per clip, at 25, 50 and 75% of its length, and are averaged. The judge sees the same three frames per clip as a strip on one sheet. So it judges what is in the frames (subject, prompt fit, look, artefacts), not how smoothly things move.
• Sound is not judged. Models that make sound have it on by default and the clips keep it (open a clip on the compare page to hear it), but nothing here scores it.
• Lengths differ. Veo models make 4-second clips, the others 5 seconds, each at its default. Three frames per clip make the scores comparable; a longer clip has more chances to go wrong that the frames may miss.
• Few prompts per task. 4 prompts per video task, under the 8 we require for a per-task verdict, so video task pages are early results. The all-video ranking pools all 16 prompts and is the stronger signal.
• On this site a clip's still is its middle frame; clips are re-encoded to 540×960 with the Picsart AI label and play muted until you open them.

Human check

People vote on about 150 image pairs ("which one is better for this request?") without knowing which model made which. We count how often they agree with the AI judge where it had a clear preference. That share is reported with its interval and the 50% chance level. The number appears here once the votes are in.

Ratings and error bars

Each task's judge rankings become one strength number per model with a Plackett–Luce model, a chess-rating-like method for whole ranked lists. The idea: when the judge picks the best image, each model's chance is in proportion to its strength. Then the best of the rest, and so on. Fitting finds the strengths that make the rankings we saw most likely.

It counts who a model beat, not just its average position. A model missing from a list (a refused prompt) does not distort the rest.

Strengths are on a log scale, centred on zero within each task. A gap of 1.0 means the higher model is about 2.7 times as likely to be picked head to head; 0 is a coin flip. The "chance of ranking first" on task pages is the model's probability of being the judge's top pick among the task's models.

How to read the bars

If we had chosen different prompts, would the table look the same? We redraw the prompts 1,000 times with replacement, keeping both judge runs of a prompt together, refit, and keep the middle 95% of each strength. That range is the bar.

• Model A
• Model B
• Model C

• Bars that do not overlap (A and C): a real gap on this benchmark; the higher model is the safer pick.
• Bars that overlap (A and B): we cannot separate them with this many prompts; their order is a point estimate, not a verdict.
• A narrower bar means more prompts agreed. With ten prompts per task, bars are wide on purpose.
• A model never ranked for a task has no measurement and shows as not benchmarked, not as a bar.

The router

Your exact sentence has no result, so the router answers a smaller question: which of our 12 tasks is it closest to? A small sentence model (22 million parameters, 23 MB, running in your browser) turns your text into 384 numbers.

We compare them with every benchmark prompt's numbers, keep the 5 most similar, and let those vote for their task, weighted by similarity. The recommendation is that task's best model; the neighbours are shown as proof, so you can see when the router guessed wrong.

When it stays quiet

Two gates guard the answer. The nearest benchmark prompt must be at least 0.40 cosine-similar to your request, and the winning task must hold at least 60% of the vote. Below either line it says it is not sure and shows the two closest tasks instead. 62% of the benchmark prompts clear that gate when held out.

Requests it cannot serve at all (GIFs, photo editing, upscaling, writing) are told so before the model even loads; video requests are routed only among the video tasks, and image requests only among the image tasks. Nothing you type leaves your browser. If the model fails to download, a plain keyword list per task takes over.

How well it routes

There is no separate test set, so we make one. Take one prompt out, route it with the other 95, check the answer, repeat for all 96. This is leave-one-out accuracy.

The router puts a held-out prompt in the right task 80% of the time (guessing at random would score 8%). Split by kind: 82% on the 80 image prompts, 69% on the 16 video prompts.

The video number is pessimistic by construction. With 4 prompts per video task, a held-out video prompt has only 3 same-task neighbours left among the other 15 video prompts, against 5 votes, so one or two close prompts from a neighbouring video task can outvote them. A real request is routed with all 4 prompts of every video task in place; more video prompts per task would make the test, and the router, stronger.

The harder question: is the recommended model the one the judge preferred for that exact prompt? That changes from prompt to prompt inside one task:

• Compass router (leave-one-out)33%
• Always the most common winner24%
• Always the most expensive model19%
• Oracle: knows the true task35%

All four rows ask the same question on the same 96 prompts: how often is the recommended model the judge's winner for that prompt? The oracle knows the true task and picks its best model. A task router can only approach it; the gap is the price of imperfect routing. The ratings behind the oracle and router rows were fitted on all prompts, so those two are slightly optimistic. Task accuracy itself is clean leave-one-out.

Why nearest neighbours, not a trained classifier? With about ten prompts per task there is not enough data to beat "look at the most similar examples". It needs no training step, is easy to explain, and the neighbours double as the proof gallery.

How we recommend

The headline is always the model the judge ranked first for the task. Often that is a premium model, so we show why it is worth it. The win probability, the runner-up, and for text tasks how often it spelled the words right. We do not rank by price.

Close race

When the runner-up's error bar overlaps the winner's, the page says so with a "Close race" badge. A line explains why the winner still ranks first: the rating counts every placement, not only first picks. The runner-up gets its own "Or make it with" button. Either is a defensible pick.

Low on credits?

Behind a collapsed "Low on credits?" row we name one alternative, and only when it is a serious contender. It must clear four tests:

• top three for the task;
• at least half the winner's chance of ranking first;
• an error bar that overlaps the winner's;
• fewer credits per image or clip.

If no model clears all four, there is no alternative to show.

Limits

• 96 prompts is small. One prompt is ten percentage points of a task's accuracy. Read numbers as "about", and never read a rank difference inside overlapping bars as a finding.
• One image per prompt per model. Models vary between runs; a second image could rank differently. The bars partly cover this; the judge's two runs do not (they see the same image twice).
• We wrote the prompts. They imitate real requests, but another author would get different absolute numbers. Real visitors type shorter, stranger things, so expect the router to do worse in the wild than in leave-one-out.
• The judge has taste. One AI model, with a known lean towards polished, detailed images, and it can misread text. Shuffled runs, the OCR check and the human votes limit the damage; they do not remove it.
• English only. Prompts, OCR and the router's sentence model are English-centric. Other languages route worse; text in other scripts is out of scope.
• Video is judged from stills. Three frames per clip, so motion quality and sound are not measured; see video.
• Defaults only. A model that is excellent with tuning may rank lower here than it deserves.
• Dated. Models and prices change without notice. Every page carries the date of its data (2026-10-08); when a model is updated or added we rerun it and note it in the changelog.
• Built by someone who works at Picsart, as an independent project. The method and all the data are public so you can check or rerun it; see about.

Changelog

• 2026-10-09First public release: 17 models, 12 tasks, 96 prompts, 960 images, two judge runs per prompt, router evaluated leave-one-out.
• 2026-10-06Method fixed before any image was generated: tasks, prompts, model list, settings and judge prompt written down and checked by tests first.

Reproduce it

Everything on the site is computed from the files on the data page. They hold prompts, models with settings and costs, every image's scores and mean judge rank, and per-task ratings with intervals. To check one claim, open the compare view for a prompt and look yourself. To rerun the maths:

1. Download scores.csv and ratings.csv; each row names the prompt, the model and the numbers behind it.
2. Fit Plackett–Luce to the judge rankings per task. Mean judge rank in scores.csv is the summary; the two full rankings per prompt are available on request.
3. Bootstrap over prompts, 1,000 draws, keep the 2.5th and 97.5th percentiles.
4. Embed prompts.csv with the sentence model named in router.json and run the leave-one-out loop with k = 5.

If you get a different answer, or want a task or a model added, say so on the about page.