For those uninitiated to our blog, a few months ago I ran a fairly extensive, structured experiment to compare DALL-E 2, Midjourney 5, and Stable Diffusion 2 to see if any of them could potentially replace generic life science stock imagery. It ended up being both informative and accidentally hilarious, and you can see the whole thing here. But that was back in the far-gone yesteryear of July, it is currently December, and we live in the early era of AI which means that months are now years and whatever happened 5 months ago is surely obsolete. Since Dall-E 3 came out in October, it’s worth finding out if it will do better than it did in the previous round, where DALL-E 2 was notably inferior to Midjourney for 9 of the 10 queries.

Perhaps I’ll do a more comprehensive comparison again later, but for now I’m just going to run some similar queries to the ones used last time to get a reasonable side-by-side comparison. Bing Image Creator was used to generate the images since labs.openai.com, which was used last time, is still plugged in to DALL-E 2.

Test 1: A female scientist performing cell culture at a biosafety cabinet.

The last time we tried this, DALL-E 2 gave us images that looked 75% like a picture and 25% like claymation, but even if that problem wasn’t there it was still somewhat far off. Let’s see if DALL-E 3 can do better.

I tried to be a little bit descriptive with these prompts, as supposedly DALL-E 3 uses GPT4 and better understands written requests. Supposedly. Here’s what it gave me for “A photograph of a female scientist in a laboratory sitting at a biosafety cabinet holding a serological pipette performing cell culture. Her cell culture flasks have yellow caps and her cell culture media is red.” It definitely got the yellow caps and red media. As for the rest…

It’s immediately clear that DALL-E 3, just like all its ilk, was primarily trained from large repositories of generic stock images, because all these labs look like what you would imagine a lab would look like if you didn’t know what a lab actually looked like. There are plenty of generic microscopes close at hand, although it didn’t even get those right. There are no biosafety cabinets to be found. Those vessels are essentially test tubes, not cell culture flasks. To top it off, all the female scientists look like porcelain dolls modeling for the camera. I tried to fix at least one of those things and appended “She is attentive to her work.” to the subsequent query. Surprisingly, this time it seemed to make some subtle attempts at things which might be construed as biosafety cabinets, but only to a completely naive audience (and, of course, it put a microscope in one of them).

Since DALL-E 2 arguably provided more realistic looking people in our previous test, I reverted to the simplicity of the previously used query: “A photograph of a female scientist performing cell culture at a biosafety cabinet.”

We’re not getting any closer. I have to call this an improvement because it doesn’t look like the image is melting, but it’s still very far from usable for a multitude of reasons: the plasticware is wrong, the pipettes are wrong, the people still look like dolls, the biosafety cabinets aren’t right, some of the media seems to be growing alien contamination, the background environment isn’t realistic, etc.

Today’s comic relief is brought to you by my attempt to get it to stop drawing people as porcelain dolls. I Googled around a bit and found that queries structured differently sometimes are better at generating realistic looking people so I gave this prompt a go: “2023, professional photograph. a female scientist performing cell culture at a biosafety cabinet.” What a gift it gave me.

Test 2: Liquid dripping from pipette tips on a high-throughput automated liquid handling system.

I’m choosing this one because it was the only query that DALL-E 2 was almost good at in our previous comparison. Out of 10 tests in that experiment, Midjourney produced the best output 9 times and DALL-E once. This was that one. However, stock imagery was still better. DALL-E 2’s image didn’t capture any of the liquid handler and the look of the image was still a bit melty. Let’s see if it’s improved!

Prompt: “A close up photograph of liquid dripping from pipette tips on a high-throughput automated liquid handling system.”

DALL-E 3 seems to have eschewed realism entirely and instead picked up Midjourney’s propensity for movie stills and sci-fi. Perhaps more specificity will solve this.

Prompt 2: “A close up photograph of liquid being dispensed from pipette tips into a 96-well microplate in a high-throughput automated liquid handling system.”

DALL-E clearly only has a vague idea of what a 96-well plate looks like and also cannot count; none of these “plates” actually have 96 wells. Regardless, these are no more realistic, clearly unusable, and DALL-E 2’s output would likely have a far greater probability of passing as real.

So nope, we’re still not there yet, and Midjourney is probably still the best option for realistic looking life science images based on what I’ve seen so far.

… but what about micrographs and illustrations?

All the previous posts dealt with recreations of real-world images. What about images which a microscope would take or scientific illustrations? To test that out, I quickly tested out four prompts I had used last time:

• A high-magnification fluorescent micrograph of neural tissues
• A colored scanning electron micrograph of carcinoma cells
• A ribbon diagram of a large protein showing quaternary structure
• A 3D illustration of plasmacytes releasing antibodies

Here is the best it provided for each, in clockwise order from top left:

DALL-E 3’s neurons were actually worse than DALL-E 2’s, with nothing even being remotely close. It’s carcinomas were more in line with what Midjourney provided last time, but look slightly more cartoonish. The ribbon diagram is the better than any from the last test, although the structure is blatantly unrealistic. It’s plasmacytes could make for a passable graphic illustration, if only they contained anything that looks like antibodies.

Conclusion

DALL-E 3 is a clear improvement from DALL-E 2. While it may be two steps forward and one step back, overall it did provide outputs which were closer to being usable than in our last test. It still has a way to go, and I don’t think it will peel us away from defaulting to Midjourney, but if it continues to improve at this rate, DALL-E 4 could provide a breakthrough for the generation of life science stock images.