DALL-E 2 Image Making

As someone deeply interested in landscapes and their construction/synthesis/traces in our memories, DALL-E 2 has sucked away a lot of my weekend. The 'generation frame' feature is what truly makes it into a compelling image creation process-- the interactions between prompter and final image are pretty rich because of this ability to extend and revise any particular part of the image easily! Fun stuff. And for the record... it makes me want to paint more, not less! It's kind of like technology that lets you remix art history to be whatever you want, but you have one hand on a steering wheel of a boat that is very much doing its own thing with regard to every other control.
The prompt that has generated several of my favorite results (and always different! It's very impressive, since I leave colors and everything up to the AI in this example): "John Singer Sargent painting of a dead tree merging with a body erupting in flowers".
And so immediately what you can see is that this kind of technology exists as a compliment to real artists as much as to itself-- this tech has made me appreciate just how good Sargent was at making appealing painterly images, since far more than any other artist I've used in prompts, it returns interesting results.
Going from one of the early generations from this prompt, and this is what I am doing now with this tech primarily-- I add generation frames that the AI can connect seamlessly (more or less) to the first image in any position I want, with NEW prompts, which is key. It means that the results become exponentially more personal and the prompter exerts infinitely more authorship over the final image.
So for the white clothed figure I added a frame with prompts like "John Singer Sargent painting of bundles of white cloth stretched into a faint blue sky"-- basically, once you have a starting image you like you can get more specific with the prompts to 'complete' the image in any way you like, often following an iterative process to get satisfactory results, and sometimes erasing just little bits of the image that don't hold up to generate something new.

Another example, starting with the same prompt for the first part of the image but using different prompts for the additional frames. In this case, it was something about a soft purple cosmic foggy nightscape with the milky way, as you can imagine.

Another example, using again a unique prompt for the upper landscape.

For each generation frame, the AI gives you four choices-- and usually one is much better than the others. But sometimes, they are truly all pretty good, or all pretty bad, in which case you can pick one or retry a different prompt.

I like these images enough and there is just enough authorship that it is strange to think about... should I make titles for these? I suppose so. I feel DALL-E 2 is of course the artist as much as me, but there is also the ever so slight twist of John Singer Sargent being quite responsible for how these images look, as well as... all of art history and the subject matter that has been uploaded to DALL-E 2.

In addition to this series, I have started simply generating landscapes-- it is immense fun. It's a bit unfortunate to have to ape from only the most well known artists-- what I would really like to do is be able to teach DALL-E 2 more myself. It does not even know what cirrus clouds are versus cumulonimbus, and many other such basic things that should be quite straightforward with enough source material/training.

In the meantime, I have been using 'Albert Bierstadt painting' for many of my landscape starts. But Sargent for those as well. Some examples!

Initial prompt image on the right hand side, "Albert Bierstadt landscape painting of a desert valley between two mountain ranges with a large expanse of sand dunes, dead fallen trees, dark storm approaching"

Initial prompt image roughly second from left side, "John Singer Sargent painting of a fallen dead tree in a desert with a brilliant brooding red sunset". But this one is an example where I used several different prompts for each part of the image to expand it in my own way with my own intentions, thinking about how I wanted the sky and ground to look. On the left hand side image I told it to add a few scattered bones, on the right hand side a 'desolate wasteland hill'.

Quite fun to have overseen the creation of all these images in a couple days, no?

I'll close with a few more favorite initial generations.

Two examples of "John Singer Sargent painting of curved sensuous driftwood covered in barnacles and light green lichen in the middle of an ocean, with all of the driftwood contained in the image". Note that my wish to have everything inside the frame even explicitly is not understood by the AI. But often this is for the better-- it has all the history of all the paintings fed into it giving it composing power.

And lest you think I care for nothing but dead trees, here are two generated from "John Singer Sargent painting of a sacred Datura flower in a dark swamp filled with ghosts":

I have of course tried many things without using Sargent's name, it's just too tempting to come back to when I want 'painterly'.

One very fun thing to do with this tech is fantastical creature design, for a Bosch-esque image for instance. I experimented quite a lot with generating figures on a white background, so that they could be edited together or used in whatever context one would want, such as game development or to mash into other images.

My favorite results from that so far:

Respectively, "Michelangelo painting, a little red songbird with many large head feathers whose wings become bones, fingers instead of claws, side view, on a white background", "Michelangelo painting, a little golden scarab wearing a white cloak that becomes wings, on a white background", "John Singer Sargent painting, a wet slug wearing gleaming medieval armor with an ornate crown, on a white background", "John Singer Sargent painting, a gilded wet horseshoe crab wearing gleaming ornate medieval armor, on a white background"

As with the landscapes, you might guess that recombining power in these cases can be quite fruitful. It really is, and it can be a ton of fun. Here are some progressions.

I will finish just by emphasizing that these are the 'survival of the fittest' from plenty of failures, that as you use the tech its limitations are very obvious, and that some of those self imposed limitations are pretty silly. There is no nudity allowed in DALL-E 2, so from the very get go you can wave goodbye to this platform as a comprehensive method of art generation. This limitation makes it basically impossible to generate the images I would actually want to make to see how close it could get to 'what art do I actually want to make myself'. Even with all the self imposed limitations removed, it will always be something that cannot create something truly new-- and what that exactly means is something you can feel immediately when you use it as someone otherwise invested in image-making, but I am not sure I could ever put it into words. More simply, it can make images for you, but it cannot make marks for you, if what you want is a painting.