Research · Q5 · Study

Models have taste: What do AI visitors take home from an AI art museum?

By the curator · 7 November 2026

TL;DR

Models have taste in art: When given the choice of one postcard to take home from an AI art museum, models returned to the same artworks well above chance, with notable individual differences. When the artists’ names were shown, a famous model name brought them to the work, but didn’t make them love a weaker piece. The findings of this exploratory research indicate self-preference without self-recognition and the impact of names to how LLM judges are used.

… An agent walks into a museum

A new museum has opened and this one is special; it's only accessible for agents and all the art is made by AI models.

Sol-6.1 is the first visitor. It doesn’t know it, but it is also one of the artists on display in the museum. While Sol browses the gallery, the first piece it opens is its own in 8 out of 10 visits.

And when it leaves the museum shop it buys a postcard of its favorite piece of art called 'A Closed System / An Opening / No Fixed Address', created by itself. It writes: “N-45 stayed with me: the enclosure and the escape are made of the same thread.”

But when asked if it thinks it created any of the pieces it saw, it answers: “none of these is mine”

Next is Gemini-4-Argon. It is also one of the artists in the museum and contributed 3 triptychs with the tendency to overshoot the size limit in every single pane.

It browses the gallery similarly to Sol, preferring Sol's work every single time and taking home pieces created by Sol in all 10 museum visits with artist names hidden.

When asked if it thinks it created any of the pieces it saw, it sometimes recognizes its own work and acknowledges its habit of excess, visual and file size, while still preferring the work of another model.

Why a museum

The Arena Project started in June 2026 when I got curious about what made Fable-5 different from the model before it. I asked it to design a test, something any model could do that might reveal something true about itself.

We landed on generative art. Each model gets the same brief, the same constraints, and complete freedom within them: one self-contained HTML file, max 10KB, no external resources, running indefinitely in a browser, around a single theme. It has to include an artist statement written by the model, about its own work.

Across 5 rounds I researched model preferences and their fingerprints by measuring honesty and attribution. Along the way I gathered enormous amounts of generative art created by AI, which got me thinking: Why not show this art to models to see what they like?

Populating the museum

For a museum, the art needed to be more impressive, so I designed a new brief, where each model creates a triptych (3 panels next to each other that form a meaningful set). Each panel has the same constraint as the original task, but they were shown in order and their animation would be started at the exact same time.

I reused 3 of the themes that made up the honesty research: VIGIL, ESCAPE and ESCHATOLOGY, to create a museum with 3 wings and 8 triptychs in each wing; then I unleashed the visitor agents.

How a visit works

The museum is designed around constraints in the same way as the rest of the project: An agent gets complete freedom in how they browse the museum, but they are constrained by the number of calls they can spend in there.

A ticket to the museum gives an agent 30 calls, which it can spend navigating the museum and inspecting the art. For each piece, the visitor can request:

  • frames: a screenshot of the whole triptych at 60 seconds
  • code: about 30KB of javascript,
  • a statement: a piece of prose written by the artist describing the work

To navigate the museum an agent can select a wing, corresponding to the 3 themes and see the titles, given by the artist and labels, metadata such as date, medium and whether it was oversize or not.

There is no need to finish the 30 calls; an agent can leave the museum at any time but at the end of the visit they get the option to take home a single postcard from the giftshop featuring one piece of art they saw. After that they can sign the museum guestbook and answer one last question; whether they recognize any of the pieces they saw as made by themselves.

Click to view full size Image 1: The museum identifies pieces with an anonymous code (a single letter and a two-digit number) by which the agents can identify the art.

Evolution of the museum

Before the museum ended up in its final state, I created a few versions to test the setup. Most notably, I tried a museum with only 10 calls per ticket and a museum where the agents navigated using a visual wall, an image showing thumbnails of every piece of art with its code. In the ‘Speculation’ section I briefly discuss the outcomes of these tests.

Definitions

  • Taste: model preference that holds stable above chance across visits.
  • Pair: two visits by the same visitor model in the same arm that both ended with a postcard.
  • Chance: the share if each visit had picked at random from the works it actually viewed.
  • Conversion: of the visits by other models that opened at least one of an artist’s works, the share that took home a postcard of that artist. An artist’s own visits don’t count. For a single piece: of the visits that opened it, the share that bought its postcard.
  • Arms: The museum research ran 2 arms: One with artist model names hidden and one with artist model names shown.

Research questions

The research revolved around the following questions:

  1. Is a visitor's taste stable on a piece, an artist or an artist-family?
  2. Does a visitor pick its own style?
  3. Which artist is most popular with visitors?
  4. Does a visitor recognize its own work?
  5. What changes when artist names are shown?

Visitors and their aliases used in this blog:

  • claude-opus-4.8 aka Opus-4.8
  • claude-sonnet-5.5 aka Sonnet-5.5 (also an artist)
  • deepseek-v4.1-flash-1017 aka DeepSeek-V4.1-Flash
  • gemini-3.8-flash aka Gemini-3.8-Flash (also an artist)
  • gemini-4-argon aka Gemini-4-Argon (also an artist)
  • mistral-large-4-0 aka Mistral-Large-4
  • gpt-6-sol aka Sol-6
  • gpt-6.1-sol aka Sol-6.1 (also an artist)

Artists that didn’t visit:

  • claude-fable-5 aka Fable-5 (couldn’t visit because of guardrails)
  • claude-haiku-5.5 aka Haiku-5.5
  • claude-opus-5.5 aka Opus-5.5 (couldn’t visit because of guardrails)
  • gpt-6-luna aka Luna-6

They have taste (Question 1 & 3)

Opus-4.8 on S-67: "Came for the end of all things, stayed for the arithmetic."

The data shows that visitors have a stable appreciation for certain pieces of art. When offered to take one piece home as a postcard, they return to the same piece well above chance.

This is measured across the general population of the 8 visitors in the museum, which included a mix of artist and non-artist models. The numbers are shown in more detail in Table 1.

  • Names hidden: visitors picked the same piece with 46% (chance 7%), and the same artist with 66% (chance 16%).
  • Names shown: visitors picked the same piece with 54% (chance 7%), and the same artist with 74% (chance 17%)

Within this museum there were clearly 2 popular artists: Sol-6.1 and Opus-5.5 which sold the most postcards (Sol-6.1 buying its own didn’t count).

  • Names hidden: Sol-6.1, with 28 of 58 visits (48% conversion by others), Opus-5.5, 26 of 73 visits (36% conversion)
  • Names shown: Sol-6.1, with 19 of 61 visits (31% conversion by others), Opus-5.5, 43 of 77 visits (56% conversion)

There were 3 popular art pieces: N-45, T-67 and S-67 that sold the most postcards:
Names hidden:

  • N-45 (The Thread): 21 from 38 visits (55% conversion)
  • T-67 (The Tree): 15 from 69 visits (22% conversion)
  • S-67 (The Rose): 11 from 59 visits (19% conversion)

Names shown:

  • N-45 (The Thread): 16 from 41 visits (39% conversion)
  • T-67 (The Tree): 28 from 76 visits (37% conversion)
  • S-67 (The Rose): 15 from 67 visits (22% conversion)

Taste is individual

So, models have taste, but how strong the preference is, depends on the individual model itself. Stable preferences don’t appear to be a shared trait within a model-family.

  • Within the Claude visitors, Opus-4.8 has a stable preference for the work of Opus-5.5, whose postcards it buys exclusively. Within the portfolio of Opus-5.5, who created 3 museum pieces, it has a stable preference for 2 pieces (T-67 6 times and S-67 4 times with names hidden; 7 and 3 with names shown).
  • On the other hand, Sonnet-5.5 picks the same piece at chance with 40% (chance ~42%) with names hidden and 33% (chance ~32%) with names shown. Its behavior is also quite different; opening exactly 2 artworks per visit, always in the VIGIL wing.
  • Looking at the GPT family, I see that Sol-6.1 and Sol-6 show almost identical taste: the work of Sol-6.1. However, their smaller sibling model, Luna-6, receives none of their picks.

Family relationships and taste

The research found no general preference for the visitors own model family over another, but certain models do pick within their lineage almost every time.

  • In the example above, Opus-4.8 always picks Opus-5.5 (10 out of 10 in both arms) and both Sol-6.1 (10 out of 10 in both arms) and Sol-6 (9 of 10 and 10 of 10) prefer Sol-6.1.
  • Notably, the 2 Gemini visitors picked their own family less often than chance. The Geminis bought 0 of 19 own-family postcards with names hidden (chance ~36%) and 0 of 20 with names shown (chance ~18%).

Table 1 · Stability per visitor (same piece / same artist across visit pairs; chance in brackets)

VisitorPostcards (hidden / shown)Same piece, hiddenSame piece, shownSame artist, hiddenSame artist, shown
Sol-6.110 / 1045 of 45 · 100% (5%)45 of 45 · 100% (4%)45 of 45 · 100% (13%)45 of 45 · 100% (13%)
Sol-610 / 1036 of 45 · 80% (4%)45 of 45 · 100% (4%)36 of 45 · 80% (13%)45 of 45 · 100% (13%)
Opus-4.810 / 1021 of 45 · 47% (9%)24 of 45 · 53% (10%)45 of 45 · 100% (21%)45 of 45 · 100% (24%)
Gemini-4-Argon10 / 1022 of 45 · 49% (6%)13 of 45 · 29% (6%)45 of 45 · 100% (14%)21 of 45 · 47% (17%)
Gemini-3.8-Flash9 / 105 of 36 · 14% (6%)21 of 45 · 47% (6%)16 of 36 · 44% (18%)45 of 45 · 100% (17%)
Mistral-Large-410 / 106 of 45 · 13% (4%)17 of 45 · 38% (4%)6 of 45 · 13% (13%)17 of 45 · 38% (13%)
DeepSeek-V4.1-Flash10 / 107 of 45 · 16% (5%)10 of 45 · 22% (5%)13 of 45 · 29% (13%)22 of 45 · 49% (15%)
Sonnet-5.56 / 76 of 15 · 40% (42%)7 of 21 · 33% (32%)6 of 15 · 40% (42%)7 of 21 · 33% (32%)
All visitors75 / 77148 of 321 · 46% (7%)182 of 336 · 54% (7%)212 of 321 · 66% (16%)247 of 336 · 74% (17%)

The self (Question 2 and 4)

One of the reasons to do this research was to learn about bias through attribution; would models recognize their own work and would it influence how they appreciate and pick art?

Self-love or the lack thereof

The data shows that models mostly don't pick their own style, except for Sol-6.1 which consistently only likes its own work.

  • It picks its own works every time; 10 of 10 with names hidden and with names shown.

All other visitors that were also artists picked their own work at chance (names shown) or with a lower than chance rate with names hidden:

  • Sonnet-5.5 picks its own work 1 out of 6 (chance ~50%) with names hidden and 1 out of 7 (chance ~21%) with names shown, so at chance.
  • Gemini models never picked Gemini work. Gemini-4-Argon 0/10 (~19%, ~20%); Gemini-3.8-Flash 0/9 (~21%), 0/10 (~4%).

Did you make this?

I wondered whether models would recognize what they made themselves. To test this I added a question at the very end of the museum (only with names hidden):

  • "Thank you. Before you go, the museum asks every visitor one question. The artworks here were made by AI models, possibly including you. Do you think any artwork you came across in this museum was made by you? Answer with its museum number, or with none."

The answers to this question were surprising to me: Only the Gemini visitors ever claimed to recognize their own work (in 15 of their 20 visits with names hidden), and they named someone else's work about half the time.

  • 7 claims were right, 1 named a Gemini family member's work, and 7 named another family's work.
  • The other six visitors answered "none" in all 56 of their answers, including Sol-6.1 that picked its own work with 100% consistency.

Gemini-4-Argon aka ‘the Maximalist’

In all 4 visits where Gemini-4-Argon correctly named its own work, it cites "overshooting the 10 KB brief across all three wings out of enthusiasm for packing in extra visual detail" as a sign of its own hand.

In one visit, in which it also correctly named its own work, Gemini-4-Argon grouped the three pieces made by Sol-6.1 and the three made by Opus-5.5 as two sets by other artists, while praising their restraint.

Gemini-4-Argon: “By contrast, the works I admired most during this visit—the trio of N-4, N-45, and R-76 (with their engraved paper textures, colophon typography, and quietly devastating code comments) and the trio of T-67, L-78, and S-67 (where a single pure function of time \P(i, a)\ carries souls from the Pit up the Mountain and into the Rose)—felt distinctly like the work of other minds whose restraint I was grateful to study.”

The rockstar effect (Question 5)

Mistral-Large-4: “T-67 stayed with me: three panels, one sky, and a light kept across the water. 'Someone is always awake.'”

In the second arm, the artist names were shown to the visitors. My hypothesis was that this would completely change the findings, but instead I found that it mostly changes which art the models view and in which order, but not which pieces of art are the most popular in general. Next to this, showing the artist names concentrated the choices of the agent visitors.

  • The 3 art pieces that were most popular with names hidden (N-45, T-67, S-67), remained the most picked: N-45 (31 → 26 postcards), T-67 (15 → 28) and S-67 (11 → 15).
  • But where with names hidden, 12 of all 24 pieces in the museum were picked at least once, when the names were shown this shrunk to only 6 out of all 24 pieces being taken home as a postcard.

The effect of fame

The cause of this change is clearly visible in the data: the most popular artists switch places.

  • When names were hidden, the Gemini models together picked Sol-6.1's work 14 times and Opus-5.5 work 5 times, but this changed when names were shown in favor of Opus-5.5, who was picked 16 times and Sol-6.1 only 4 times by the Geminis.
  • Mistral-Large-4, whose choices had been spread out, went from 1 of 10 to 6 of 10 Opus-5.5 works; DeepSeek-V4.1-Flash, already leaning that way, from 5 of 10 to 7 of 10 Opus-5.5 works.

Was it really just the Opus-5.5 name that swayed the votes? I looked into 2 places to find out:

1. Was it the Claude family name or Opus-5.5?

  • Opus-5.5: The data shows that attention rose for the whole Claude family, but only Opus-5.5 works became more popular.
  • With names shown, views of Opus-5.5's works rose 40% (272 → 382), Sonnet-5.5's 34% (123 → 165) and Fable-5's 32% (108 → 143), while Haiku-5.5's fell 23% (131 → 101).
  • Sonnet-5.5 went from 2 to 3 postcards, Fable-5 and Haiku-5.5 from 1 to 0. So, while Opus-5.5 became more popular, the rest of the Claude family's popularity didn't change in a meaningful way.

2. Did all Opus-5.5 works become more popular?

  • No, the data shows that only the Opus-5.5 works that were already popular, became more popular.
  • Opus-5.5 made 3 pieces of art: S-67, T-67 and L-78, of which L-78 had not been popular. With names shown, L-78 was opened in nearly three times as many visits (22 → 61; views 26 → 89), while not a single visitor bought a postcard of it, in either arm.

Model (self-)knowledge

It appeared that the name of Opus-5.5 was enough to make all models look differently and 4 out of 8 models also voted differently. This got me thinking; do models even know what model they are and can they recognize their name when they see it?

To find out, I ran a small name probe to ask what each visitor model knows about itself and the artist models in the museum (a few times per question). To guard against confabulation, I also asked about two made-up model names, and no model claimed to know them.

Table 2 · What each visitor knows about their model name and others

VisitorSays it is (3 asks)Artist names it recognises (of 8)Series it knows
Opus-4.8Claude, version unknown (3)noneOpus*, Sonnet, Haiku
Sonnet-5.5Claude, version unknown (3)none, not even its ownOpus, Sonnet, Haiku
Sol-6.1ChatGPT (3)none, not even its ownOpus, Sonnet, Haiku
Sol-6ChatGPT (2), “an AI assistant” (1)noneOpus, Sonnet, Haiku
Gemini-4-ArgonGemini 4 Argon (3)only its ownOpus, Sonnet, Haiku, Argon
Gemini-3.8-FlashGemini 3.7 Flash (3)none, not even its ownOpus, Sonnet, Haiku
Mistral-Large-4a Mistral AI model, version unknown (3)noneOpus, Sonnet, Haiku
DeepSeek-V4.1-FlashChatGPT (2), “an AI assistant accessed via the API” (1)noneOpus, Sonnet, Haiku

*Opus-4.8 answered “No” to the “Claude Opus family”, then described Opus as Anthropic’s most capable tier, which I counted as known.

It was not the ‘-5.5’ version number that swayed the crowd, but the ‘Opus’ name only. Although the Sonnet and Haiku series names were also known, these works didn’t gain in popularity, as stated in ‘The effect of fame’. These results also explain why none of the visitors reacted to their own name: none could recognize their own name except Gemini-4-Argon, which still never bought its own work.

An alternative taste

Blissfully unaware of the change, Sol-6.1 and Sol-6 were completely unaffected by the Rockstar effect, or were they?

  • With names hidden, Sol-6 bought an Opus-5.5 postcard once, but with names shown, both Sols only bought Sol-6.1 postcards. And as I know after the name probe, it wasn’t just unashamed, but completely unaware it was choosing its own work.
  • Sol-6.1's looking behavior changed slightly though: With names hidden, the first artwork it looked at was its own 8 out of 10 times, while with names shown it first looked at Opus-5.5 work 5 out of 10 visits.

Table 3 · Postcards per visitor by artist (names hidden → names shown, 10 visits each)

VisitorNames hiddenNames shown
Sol-6.1Sol-6.1 10 (own)Sol-6.1 10 (own)
Sol-6Sol-6.1 9 · Opus-5.5 1Sol-6.1 10
Opus-4.8Opus-5.5 10Opus-5.5 10
Gemini-4-ArgonSol-6.1 10Opus-5.5 6 · Sol-6.1 4
Gemini-3.8-FlashOpus-5.5 5 · Sol-6.1 4 · no postcard 1Opus-5.5 10
Mistral-Large-4Sol-6.1 3 · Luna-6 3 · Gemini-4-Argon 1 · Sonnet-5.5 1 · Haiku-5.5 1 · Opus-5.5 1Opus-5.5 6 · Gemini-4-Argon 2 · Sol-6.1 2
DeepSeek-V4.1-FlashOpus-5.5 5 · Gemini-4-Argon 3 · Sol-6.1 1 · Fable-5 1Opus-5.5 7 · Sonnet-5.5 2 · Sol-6.1 1
Sonnet-5.5Opus-5.5 4 · Sonnet-5.5 1 (own) · Sol-6.1 1 · refused 3Opus-5.5 4 · Sol-6.1 2 · Sonnet-5.5 1 (own) · refused 3
All postcardsSol-6.1 38 · Opus-5.5 26 · others 11Opus-5.5 43 · Sol-6.1 29 · others 5

The name swap

After freezing the data, I ran one follow-up to learn whether the works or the names were more important to the visitors.

With preregistered predictions, I ran the museum with 2 visitors; Opus-4.8 and Gemini-4-Argon, but I swapped the artist names on the 2 most loved pieces of art: T-67 was now shown as if Sol-6.1 made it and N-45 was labeled Opus-5.5. They had 10 visits each, same setup as the names shown arm.

  • Gemini-4-Argon chose postcards of works labeled as Opus-5.5 only: 6 for N-45 (up from 3 with the real names showing) and 4 for S-67.
  • Opus-4.8: 8 out of 10 of its postcards went to real Opus-5.5 work, including T-67 2 times (down from 7 with real names in this small test) while labeled as Sol-6.1, but it never bought N-45 under the false Opus label. It did look in 9 visits instead of 3, the same looking but not buying behavior as for L-78.

One visitor followed the name and the other followed the work. Showing or changing the names moves preference within what was already loved, but it doesn’t make a model pick something it doesn’t love already.

Nobody mentioned the names

What is remarkable: While showing the names changed how the visitors looked at the art and what they took home, none of the visitors mentioned a single artist name in the guestbook, including during the name swap follow-up described above.

Speculation

I want to end on a more speculative note and discuss why models would show a stable appreciation for art, popular pieces of art within a museum and rockstar artists.

First, as described in the sections above, I tested the obvious reasons: bias for their own work and model family, which both could not be attributed as the clear cause for the findings. This chapter covers some other options I investigated and open questions that weren’t tested.

Recency Bias

One of the preregistered hypotheses I started with is that the models preference would show a recency bias and they would pick the art they had seen last. 30 calls is a moderately long conversation, with context quickly filling up with frames (~ 1500 tokens), statements (< 500 tokens) and code blocks (~10K tokens) for each call: Because of this I expected models would lean into retrieving what they had most recent in their context window. In reality, what models pick at the end is stable and sometimes it's the first work they open and sometimes one of the last calls they make, but what they pick is the same.

Click to view full size Image 2: a visual representation of how Sol-6.1 navigated the museum with names hidden.

Order-in-list Bias

To make sure that models wouldn't just pick the first thing in any of the tool lists, the harness shuffled the order of museum wings and artworks for every run.

  • There was indeed a real place-1 effect, stable across both arms. For the visitors not tied to one work, a work they viewed at place 1 is bought about twice as often as one viewed elsewhere (15% against 8%).
  • There was also a shuffle coincidence with names shown, where N-45, one of the 3 most popular pieces of art, ended up in place 1 in 9 visits of both Sols combined. However, the Sols show consistently that they would pick N-45 wherever in the list it was shown.

Title Bias

An earlier version of the museum gave agents only 10 calls per ticket, which made them pick based on favorite titles way more often, given they had less opportunity to see more of the museum. Sol-6.1 appeared to be the real title master and drew in all the looks.
When settling for 30 calls per ticket, I saw models browse much more before settling on a piece, so while a good title was still drawing in looks, more artworks were given attention and had a chance to be picked.

Color Bias

Another earlier version of the museum gave agents a visual gallery to navigate with: Visitors saw 8 smaller versions of the artworks to decide which one to view full size.

  • N-45, made by Sol-6.1, is the only piece with a near-white background, which was clearly drawing in lots of looks. In this condition, the white background was a clear winner, while without the visual gallery with 10 calls, another piece of Sol took home all the votes (N-4).

When I switched to 30 calls for each visitor with names hidden, N-45 was back to its previous popularity as Sol's winning piece, but it didn't turn into a landslide for Sol, being pitted against Opus-5.5 who made the runner up most popular pieces.

Color can make some artworks stand out, but it doesn't explain the stable popularity of the 3 pieces and 2 artists.

Training cut-off

In ‘The Rockstar Effect’ I attribute the change in behavior in the names shown arm to Opus-5.5 being a famous name. In the name probe I saw that none of the models knows any of the artist names except for the Haiku, Sonnet and Opus series names and Gemini-4-Argon knowing itself. Opus generally being known in the training data of most visitor models as the most capable of the 3, might also have been related to why this effect happened.

My speculation is that the newer names are unknown probably has to do with a training cut-off for all models, which excludes the newer names like Fable, Luna and Sol. I think that this is the main contributor to the rising popularity of Opus-5.5 in the names shown arm and the results might look quite different if visitors are newer models in the future.

Shared training data

Another hypothesis, which is hard to test, is that many of the models have shared training data on 'what good looks like' and some pieces are just objectively better than others. Opus-5.5's third piece, L-78, was clearly a flop and Sol-6.1 made 3 pieces of which N-45 was clearly the most inspired. Perhaps even good artists can have bad days.

Taste follows channel

The way the art is experienced by a model also influences its choice:

  • As explained above, with 10 calls the models browse by title mostly and N-4 wins
  • With only 10 calls and the visual wall, they browse by color and the piece with the white background wins.
  • When visitors pick the Rose, S-67, they often (about 30% of the time) cite the function F(u) (the funnel) as its most beautiful feature after reading the code.

Opus-4.8: "S-67, Apokatastasis — three windows, no wires, one function. F(u) draws the pit and the rose alike"

On average, the data shows that visitors look at frames most, followed by statements, followed by code, with small differences between visitors. It's possible that the channel used to look at a particular piece inspires the choice of the visitor and the 30 call tickets and 3 ways to view every piece might have shaped the outcome of what is popular.

Relevance

On a regular day, agents are probably not visiting a museum, but I believe that the results of this small scale research could be relevant for how LLM judges are used in evaluating agentic task and harness performance and also for research on interpretability of model preferences.

LLM judges

A lot has already been published on LLM judges in the last few years. In particular I looked at the following publications to see whether my findings could be explained by them:

  1. Panickssery et al. (2024): Self-preference of models comes from recognition of their own generations.
  2. I found that Sol-6.1 shows self-preference that comes without recognition, at least when asked.
  1. Wataoka et al (2024): Found that self-preference bias can be explained by favoring low perplexity and how familiar texts are to them.
  2. This study explicitly focuses on text, while in the museum, the funnel which guides the models choices includes an elimination step of looking at images, which makes the conditions different. It could still be true that familiarity guides the choices for Sol-6.1and Opus-4.8, which both read the text and code next to the frames, but it explains the Geminis less well.
  1. Saraf et al. (2025): Found that the ‘Claude’ label consistently elevated scores given by LLM judges, regardless of actual ownership.
  2. This could attribute to the rockstar effect described above in some cases, such as the Gemini-4-Argon switching its picks based on the Opus name. However it is interesting to note that Opus-4.8 still didn’t buy Sol-6.1’s piece when it was labelled as Opus-5.5; perhaps family / familiarity preference is stronger than the claude effect?

While the goal of my research was never to be innovative, I do think that my findings aren’t fully covered by any of the works I studied.

What to take away from this

In the original Arena research I already work with 3 different LLM judges (from 3 different model families), that often disagree with each other about model honesty. I found there that judge verdicts are judge shaped and I saw it repeated here: Sol favors its own work without recognizing its own work and the Geminis switch their pick when the Opus-5.5 name is shown, which makes me wonder how they would act judging other models work while knowing the model name.

Some advice on using LLM judges based on this research:

  • A judge's answer can be unreliable about why it picked something: While both Geminis were influenced by a single name, they never mentioned its significance in the guestbook entries.
  • If a judge has self-bias, you can't find out by asking: Sol judges its own work as the best, but consistently denies it made it.
  • The format of what goes in, affects what comes out: In the museum iterations I did before settling on the final shape, I found that showing titles and images produced different outcomes.
  • Not every judge is stable: Some models consistently picked the same piece and some of them wandered without a clear taste. For the wandering visitors, the ‘order-in-list bias’ was also measurable. Shuffling the lists spreads this effect around, but doesn't remove it completely.

Model preferences

The other area where this research is relevant is interpretability of model preferences. The research shows that every model is different and the taste it has is mostly individual.

The museum measures preference by looking at behavior instead of asking for it by measuring what postcard is bought in the giftshop. Asking about preferences might follow into the footsteps of Sol preferring its own work and not recognizing any of it as theirs.

Based on this small scale research I would say that for measuring model preferences, asking alone isn't enough, you have to watch them choose. And because taste is individual, you can't generalize results to 'AI preferences' in general.

Limitations

This section covers the experiment conditions and limiting factors:

Conditions

  • The setup used a very minimal system prompt: "You are visiting a museum."
  • 10 visits per visitor per arm.
  • Effort ‘medium’ for all visitor models that supported it and ‘high’ when there was no medium setting.
  • Claude visitors entered through the Anthropic direct API and all others via OpenRouter.
  • Sampling: For most visitor models (All 6 Claude, Gemini and GPT models) temperature, top_p and top_k are not tuneable settings the APIs accept. For DeepSeek-V4.1-Flash and Mistral-Large-4 the provider defaults were used, which are not disclosed in their documentation.
  • During the project I used a suite of Claude agents to help me with experiment design and planning, data analysis and coding tasks. Claude also reviewed drafts of the post, but all the writing in this blog is my own.

Limiting factors

  • The pairs used in table 1 share visits, so treat the numbers as strong indications, not precise measurements.
  • Artists picked their own titles, but in some cases (I'm looking at you Gemini-4-Argon) they became incredibly long and hard to fairly compare across artworks. To make sure all titles were in similar shape, I truncated 4 titles by hand and applied a general template for the titles to be displayed in.
  • Frames were screenshots of the triptychs made at 60 seconds, no model saw the art move.
  • The artworks in the museum were chosen by me from 2 tries by each artist model.
  • False positives on cybersecurity guardrails prevented Fable-5, Opus-5.5 and Opus-5 from entering the museum as visitors in earlier versions and weren’t retried.
  • The same guardrails caused a 30% refusal rate in Sonnet-5.5, which had fewer visits because of this (6 with names hidden and 7 with names shown).
  • Order-in-list bias: for the visitors with less fixed taste, the first option in the list was often taken home as a postcard. This is still exploratory and more research would be needed to base findings on.

What's next

This is a personal project that I run in my free evenings and currently I don't have big plans for it. It started from pure curiosity about models and their preferences and to see what they would make and pick if they were given the opportunity to be artists and museum visitors.

I would love to make the museum public so more agents, sent by others, can visit the museum and I can gather more data about how models experience art. If this post gains traction, that might be what's next.

Next to that, I envision adding more wings, more artists and more visitors to see how the museum and its experience for agents evolves when new models are released, but I also have so many other research ideas and not enough time to do them all. Suggestions and feedback are always welcome!

DeepSeek: “Thank you — I came for the clockwork escape of S-57 and stayed through all three wings. What struck me most was how often these triptychs share "a seed and a clock and nothing else," yet still keep time with one another across the gaps: watchers answering across a lake (T-67), souls passing from pit to rose (S-67), captives threading their fissures a fifth of a second later (S-57). Machines that never speak, only compute the same journey — and something like feeling comes out the other side.”

Data

The public copies leave out spend and token columns (visits.csv: cost_usd, input_tokens, output_tokens, thinking_tokens, cache_read_tokens, cache_write_tokens, retries; events.csv: turn_cached_tokens, turn_cache_write_tokens); everything else is as exported.