DALL-E 2 Image Making

As someone deeply interested in landscapes and their construction/synthesis/traces in our memories, DALL-E 2 has sucked away a lot of my weekend. The 'generation frame' feature is what truly makes it into a compelling image creation process-- the interactions between prompter and final image are pretty rich because of this ability to extend and revise any particular part of the image easily! Fun stuff. And for the record... it makes me want to paint more, not less! It's kind of like technology that lets you remix art history to be whatever you want, but you have one hand on a steering wheel of a boat that is very much doing its own thing with regard to every other control.
The prompt that has generated several of my favorite results (and always different! It's very impressive, since I leave colors and everything up to the AI in this example): "John Singer Sargent painting of a dead tree merging with a body erupting in flowers".
And so immediately what you can see is that this kind of technology exists as a compliment to real artists as much as to itself-- this tech has made me appreciate just how good Sargent was at making appealing painterly images, since far more than any other artist I've used in prompts, it returns interesting results.
Going from one of the early generations from this prompt, and this is what I am doing now with this tech primarily-- I add generation frames that the AI can connect seamlessly (more or less) to the first image in any position I want, with NEW prompts, which is key. It means that the results become exponentially more personal and the prompter exerts infinitely more authorship over the final image.
So for the white clothed figure I added a frame with prompts like "John Singer Sargent painting of bundles of white cloth stretched into a faint blue sky"-- basically, once you have a starting image you like you can get more specific with the prompts to 'complete' the image in any way you like, often following an iterative process to get satisfactory results, and sometimes erasing just little bits of the image that don't hold up to generate something new.

Another example, starting with the same prompt for the first part of the image but using different prompts for the additional frames. In this case, it was something about a soft purple cosmic foggy nightscape with the milky way, as you can imagine.

Another example, using again a unique prompt for the upper landscape.

For each generation frame, the AI gives you four choices-- and usually one is much better than the others. But sometimes, they are truly all pretty good, or all pretty bad, in which case you can pick one or retry a different prompt.

I like these images enough and there is just enough authorship that it is strange to think about... should I make titles for these? I suppose so. I feel DALL-E 2 is of course the artist as much as me, but there is also the ever so slight twist of John Singer Sargent being quite responsible for how these images look, as well as... all of art history and the subject matter that has been uploaded to DALL-E 2.

In addition to this series, I have started simply generating landscapes-- it is immense fun. It's a bit unfortunate to have to ape from only the most well known artists-- what I would really like to do is be able to teach DALL-E 2 more myself. It does not even know what cirrus clouds are versus cumulonimbus, and many other such basic things that should be quite straightforward with enough source material/training.

In the meantime, I have been using 'Albert Bierstadt painting' for many of my landscape starts. But Sargent for those as well. Some examples!

Initial prompt image on the right hand side, "Albert Bierstadt landscape painting of a desert valley between two mountain ranges with a large expanse of sand dunes, dead fallen trees, dark storm approaching"

Initial prompt image roughly second from left side, "John Singer Sargent painting of a fallen dead tree in a desert with a brilliant brooding red sunset". But this one is an example where I used several different prompts for each part of the image to expand it in my own way with my own intentions, thinking about how I wanted the sky and ground to look. On the left hand side image I told it to add a few scattered bones, on the right hand side a 'desolate wasteland hill'.

Quite fun to have overseen the creation of all these images in a couple days, no?

I'll close with a few more favorite initial generations.

Two examples of "John Singer Sargent painting of curved sensuous driftwood covered in barnacles and light green lichen in the middle of an ocean, with all of the driftwood contained in the image". Note that my wish to have everything inside the frame even explicitly is not understood by the AI. But often this is for the better-- it has all the history of all the paintings fed into it giving it composing power.

And lest you think I care for nothing but dead trees, here are two generated from "John Singer Sargent painting of a sacred Datura flower in a dark swamp filled with ghosts":

I have of course tried many things without using Sargent's name, it's just too tempting to come back to when I want 'painterly'.

One very fun thing to do with this tech is fantastical creature design, for a Bosch-esque image for instance. I experimented quite a lot with generating figures on a white background, so that they could be edited together or used in whatever context one would want, such as game development or to mash into other images.

My favorite results from that so far:

Respectively, "Michelangelo painting, a little red songbird with many large head feathers whose wings become bones, fingers instead of claws, side view, on a white background", "Michelangelo painting, a little golden scarab wearing a white cloak that becomes wings, on a white background", "John Singer Sargent painting, a wet slug wearing gleaming medieval armor with an ornate crown, on a white background", "John Singer Sargent painting, a gilded wet horseshoe crab wearing gleaming ornate medieval armor, on a white background"

As with the landscapes, you might guess that recombining power in these cases can be quite fruitful. It really is, and it can be a ton of fun. Here are some progressions.

I will finish just by emphasizing that these are the 'survival of the fittest' from plenty of failures, that as you use the tech its limitations are very obvious, and that some of those self imposed limitations are pretty silly. There is no nudity allowed in DALL-E 2, so from the very get go you can wave goodbye to this platform as a comprehensive method of art generation. This limitation makes it basically impossible to generate the images I would actually want to make to see how close it could get to 'what art do I actually want to make myself'. Even with all the self imposed limitations removed, it will always be something that cannot create something truly new-- and what that exactly means is something you can feel immediately when you use it as someone otherwise invested in image-making, but I am not sure I could ever put it into words. More simply, it can make images for you, but it cannot make marks for you, if what you want is a painting.

Game Development Update

 It has been three years since I finished a traditional painting-- what have I been doing then?

Many times I do have a strong desire to throw everything out and get back to oil and pigment and gesso and canvas. It would be celebrating a thing I built for myself and out of myself with the most consequential years of my life (by which I mean the investment of full-time focus for 3 years which would be extremely expensive to re-create). But other days I am actually quite happy with where my practices have taken me.

My first figurative sculpture, the "Alizarin Man", or rather, its face. The face of one of the playable characters in the game project (this is an in-game screenshot from Unity).

Of course, I have been working on a game development project for several years now. But really it has been a series of ideas, newly learned disciplines, re-thinking and refinements. What is encouraging is that the fundamental part of it-- digital sculpting-- is a practice that has not shifted or degraded over time. The sculptures I made in the first couple years of digital sculpting may well find their way into my current work eventually. Many of the things I've worked on will not, and that is the natural result of learning an entirely different medium and artistic context. Just describing game development as a medium comparable to painting is not really accurate at all. It is a whole different kind of circumstance in which and for which art is made.

This is a stitched screenshot of all my commits to my project in Github. Swear words warning. Game development is hard.

A few years ago, I discovered a method of coloring digital sculptures that suits my practice as a digital sculptor perfectly. Since then, I have been learning and discovering through countless trials how to solve other problems related to game development in ways that suit my practice as well: specifically, things like how to create landscapes of large scale in relationship to a character. There are so many technical challenges with the attempted translation of abstract expressionist painting into three dimensions and player agency with free movement being the mode of the final experience. Composition is completely different, and what you need to focus on and not focus on is extremely consequential to having any chance of completing something ambitious.

This is "Blue Voice", my second figurative sculpture for the project that will be a fully rigged humanoid character.

And unfortunately my project is ambitious. I have consoled myself in recent weeks by quickly setting up a roughed out version of what a more reasonable take on my current project would be. And by doing this, I confirmed that I could indeed do that instead, and get it done much sooner than I will finish my project the other way. But doing that would mean throwing out (or at least throwing out the intentionality behind the sculptures in terms of how the player is supposed to relate to them) a good deal of work. And I like building more than anything-- for me it is critical that my process be one of organically building something deeper and higher and with more intrigue as I go. 

I color my sculptures in Substance Painter with countless layers of 'cast light' that catch the details in the high detail version of the sculpture from different angles, and of course a little bit of direct digital painting of areas. Like all of my sculptures, my figures are completely asymmetrical.

So instead of flattening out a tumultuous landscape, I have instead committed to building a suitable barrier within it to create a reasonable final limit to my project. 

In my mind, a 'suitable barrier' is a string of sculptures that are interesting enough to be a destination-- so that the fact that they block you will not be (as) disappointing. 

A new sculpture with no color that is the beginning of the boundary wall I am sculpting. Another such un-colored sculpture can be seen in the model preview window.

I want to finish this current project either on my own, or in collaboration with my partner in order to feature her writing and music. Given that, a huge consideration has always been audio. That is another thing I have been doing-- I started off 2021 by learning how to create my own music via an intensive month long class. Specifically, Andrew Huang's music production course on Monthly. It went well for me, and the first thing I've spent money on in a long time is some audio gear and software.

Making my own music has been immensely satisfying and exciting. For the game project, of course I will aim to make some music, but more important perhaps for that is making ambient soundscapes and the sounds of various objects and interactions. Learning to make sounds like this has been a lot of fun. I bought a H5 Zoom and the shotgun mic capsule and have recorded Canadian geese, an elephant seal at Point Reyes National Seashore, and countless other sounds that present atmospheric possibilities for my game project.

Canyonlands National Park... and Roads

 'Ad Infinitum' jokes abounded between my partner and I on our (my first, her second) excursion to Canyonlands National Park in Utah. 

From Island in the Sky, you get a good sense of the repetitions of erosion that have incised the uplifted terrain stretching as far as you can see. It turns out that wandering through this difficult to parse landscape is just as you might imagine from above: confusing, and overwhelming. 

The bloated crumbling differently eroded surfaces all wash over you in a haze that makes it difficult to pick out particular places to remember. Being surrounded by boundaries that block the sky means you have little to orient yourself. On our second day, we hiked from Elephant Hill to the Joint, passing through Chesler Park on the way and heading a different way on the way back.

Chesler Park and the Joint stand clear in my memory, but the rest of the rises and falls through gaps and onto more views of an endless grid twisted into cacophony blend together differently than I had yet experienced on a hike. The layers of rock themselves were distinctive in shape and color but the endless parade of their manifestations are impossible to keep track of, as you clamber up and down the slickrock, through gaps, onto flats, into slots, over and under and through. It's a sandstone overload, and at any given moment you are more secluded, or perhaps simply more occluded from the rest of the world than on most kinds of walks.

Just as systems full of similar repetitions knit the fabric of our skin and teeth, so our lives fit neatly in the grooves carved by erosion, between deposited mounds that resisted it, and into a chain of cycles...

The Canyonlands are a strange place because there is no clear way to survive there other than by bringing in water with your car-- we saw no large mammals, no petroglyphs, and yet it is a landscape created largely by water. In that way it is like a ghost landscape, one that makes sense only with an awareness of vast and distant change. I believe there are a few springs in the area somewhere, but only a few if any.

Newspaper Rock is a site situated many miles from there which lies in a landscape more alive, where the water continues today, and clearly is not new-- where the rock was marked in passing and celebration with images of the forms that pass through, or that live nearby, in countless repetitions.

It is a strange privilege to explore a place like the Needles District on foot, having driven there across huge distances in a short time-- and a stranger privilege still to know that deeper access to places like this across America are the purview of only those who own a high clearance four wheel drive vehicle. I don't think any of those roads should be paved, but I also don't know that they should exist in the first place. 

This infrastructure is inherently inequitable. It is impossible not to notice when you are on foot and have hiked all day in 100 degrees: the disposition and casual entitlement of people driving in with jeeps and marking their names on the rocks, declaring both that they were in this place and are proud of it. Names are written for others to see. Those with access are proud of it. Most people who come here do not write their names, and often I am sure they do respect the place they are in as something greater than them. And of course, what they do when they write their names we have categorized as illegal graffiti in this kind of place, a national park.

Regardless of a name etched pathetically onto the rocks, we cannot be at home in this land. It is a shallow cross-section of the deep time on which we float, and it will consume all of our marks with complete indifference. That is not vindictiveness, that is grace. It is worth treating some surfaces as sacred. Do you feel you are your best person behind the wheel?

Roads are marks like names, declaring our sense of ownership of the land. Did the people here before us think they owned the land in the same way? The petroglyphs imply to my layman white eye that they thought the forms around them that allowed them to survive were more important than their names to record on the rocks. You can see a bit of this same spirit twisted in the marks left by white settlers: they marked their own technology as what was important in allowing them to survive: the wagon wheel. It makes perfect sense then, that the perpetuation of respect for technology and not food or life would result in our culture of roads and vehicles. Our culture and laws of ownership over animals, water, and the land that makes our life possible. You have to be able to at least get around and look at something in order to feel like you own it.

America wants to say: "Build yourself a wagon and you can do anything, and go anywhere, and own anything". But the lands are already divided up and controlled by the hundreds of millions of us that have spread across them. There is no more vast sense of freedom to steal from the people that were there before us. That was already taken. What is left is lots, acres, units, and parcels. Our consolation prize is the vast network of public roads available to us if we participate in the culture. At least there are more options. I think access in a national park being tied to the ownership of something wholly unnecessary for life is against the supposed purpose and spirit of the national parks. What am I going to suggest next, public transportation to the parks from the cities, as well? Anyways, the way it is done in Denali feels much better-- at least there, there is a bus that takes you down that dirt road and lets you get off at any point.