Artificial Intelligence Art: Questions Answered

July 29, 2021

Video Transcript

In this video, I'll answer some of the questions I get asked more as an artist than a researcher and engineer. I got some of these questions in conversations I had in person, some on my social media accounts, and others in some interviews I gave. Most of them are related to AI art and computational creativity and a few of them are related to my background. Alright, without further due, let's start with the first question.

Q1: What is AI Art? How do artists use AI in their work?

AI art refers to any kind of artwork that is created or assisted by Artificial Intelligence systems.

There are many ways artists can use AI in their artwork. Some use general-purpose generative models. Some use multi-modal systems to experiment with multiple modalities like images, text, and audio. Some use data visualization tools. Some use computer graphics techniques, and some use sequence models to draw and paint.

A commonly used family of generative models is called generative adversarial networks. Generative adversarial networks refer to neural network setups that involve two models that compete against each other. One of those models acts as a learning artist and the other acts as an art critic. Both models improve over time and once they reach an equilibrium we use the first model to create images that look similar to a given set of images, but not identical to any of them. If you are interested in learning more about how this works you can check out my earlier videos.

Q2: How do you use AI in your art?

I usually come up with a concept first, such as flowers having heartbeats or meadows turning into cities, and vice versa. Then, I use generative models to approximate what I had in mind. Currently, I use a custom generative model that gives me all the flexibility to tune the knobs to get the results I like.

I usually use a combination of artificial intelligence and traditional algorithms to create art that pops out. I optimize multiple aspects of the generated images, such as how well they reflect the ideas I had in mind, how well they match the audio, and how beautiful they look. And I use mathematical formulas to do so.

You may ask: wait, how can one mathematically optimize how beautiful images look? It’s all subjective, isn’t it? Although image quality and aesthetics are generally regarded as subjective, it's actually something we can optimize for objectively, to some extent. Images that we perceive as good quality tend to have some statistical properties. Using those properties, we can constrain the outputs of a model only to beautiful images.

Perceptual image quality is a fairly old yet active area of research. My former research advisor Alan Bovik is one of the pioneers in this field. You can check out his work if you are interested in learning more about it.

Other algorithms that I use in my art include onset detection algorithms, which detect the beginning and the peak of musical notes and beats. They help match the video to audio. I also experiment with some other algorithms to achieve certain effects in my videos.

Q3: Can you explain your process of creating art with an example?

Sure. Let’s take a look at the Heartbeats of a Flower, which I recently tokenized as an NFT on SuperRare, which is a marketplace for digital art.

Heartbeats of a Flower started with a question: if machines could think, what would they think of plants? Would they see them as inanimate objects or as living beings? Can machines really perceive things the way we do? Do even humans perceive things the way other people do?

All those questions led to the concept of flowers having a heartbeat. It was an approximation of how I imagined the perception of plants through the eyes of a machine, in an exaggerated way.

Q4: What is an NFT, and what is NFT art or crypto art?

I’m not an expert in blockchain technology but NFT stands for non-fungible token. They are unique records on the blockchain that can serve as proof of ownership and authenticity. They can be used to track the identity, authenticity, and ownership history of digital assets.

NFTs can be tied to all kinds of things, like art, news articles, and even tweets. NFT art or crypto art usually refers to pieces of art that are tokenized as NFTs and sold, transferred, or stored in that form.

Q5: How do you generate high-resolution and vibrant images using neural networks?

First, let me explain the high-resolution part. I designed a resolution-independent generative model. It’s a model that can generate high-resolution videos while being trained on lower resolution images. I did this by using continuous-valued, sinusoidal waves to encode the global structure of the images.

Theoretically, there is no limit to the output resolution of my model but the relative size of the visual elements in the images gets smaller as I increase the resolution. So, I usually render images natively up to 4k resolution. I can also upscale images up to 16x after that using a separate super-resolution model. I made a video on how AI upscaling works earlier. You can check it out if you want to learn more about that.

As for the vibrance of the colors, I used a colorfulness metric to optimize alongside the other loss functions. You can think of this colorfulness metric as a soft constraint on two-dimensional chrominance histograms.

Q6: Have you published any papers about your AI art techniques?

I haven't published any technical papers about my art but I’ll put a list of papers that I found inspiring in the description below. You can check them out if you are interested in the technical details.

Q7: When did you become interested in art?

I have been interested in art since my early childhood. I wanted to do a lot of things as a child. I wanted to become a computer engineer, scientist, teacher, astronaut, and artist. Obviously, I didn’t become all of them but I think I got pretty close. I work as an engineer. I published some research articles in academic journals as a scientist. I never became a teacher but I make videos in the form of mini-lectures here on YouTube. As for the astronaut part, the closest I got was working on images sent from low earth orbit during my PhD research.

As for becoming an artist, I didn’t have the skills to draw and paint by hand well, I admit that. That was one of the reasons why I shifted my focus to digital art. My first encounter with the concept of creative coding was at demoscene parties we had in college. It was liberating to see that I could use my coding skills to create something expressive.

In 2011, I decided to do a Master’s project that involved both art and technology. First, I had the idea of using machine learning models to generate paintings. But at that time, it was a very challenging problem, since we had very limited computing power, no deep learning frameworks, and very little prior work to look into. Then I decided to pause my efforts on the ‘generation’ problem and focus on the ‘detection’ problem. So, my research at that time focused on detecting whether certain visual elements of a digital artwork were apparent in other art. I built machine learning models to detect images that share common visual elements.

I revisited the idea of AI-generated art in early 2020 and started experimenting with ideas that I had been thinking about for a while.

Q8: When did you become interested in AI? What fascinates you about it?

My interest in AI also goes all the way back to my childhood. I remember the first time I heard about artificial neural networks when I was 11. I saw it in an article about optical character recognition. The software they developed was basically parsing text in scans of documents. It's very trivial to do today but it was a very new technology back then. I was fascinated by the kind of things that can be done using machine learning, which is a sub-field of AI that focuses on teaching computers to do things by showing them examples.

That's one of the aspects of AI that fascinates me. With machine learning, we don't have to fully understand the underlying mechanics of a problem to solve it. AI may figure it out for us, given a set of examples. That doesn’t mean we need no understanding of the problems we are attempting to solve though. We still need to have a substantial amount of understanding so that we can properly formulate the problem. So, there's still an ample amount of human involvement in today's AI systems.

Q9: Do you see AI as a kind of a creative partner, a medium of expression, or a tool for artists?

I see code as a medium of expression. I consider writing code and building AI art models analogous to using brushes, paint, paper, and wood in the process of creating art.

I think coding, in general, is a very versatile medium for art. We can use it to build functional applications as well as to express our creativity. We can even use code to implement learning algorithms to build models that learn and evolve over time.

As for AI art models, I see them as more of a tool. Arguably, they can express their own creativity to some extent but they usually work better as interpolators than extrapolators. What I mean by that is that they can successfully generate images that are in-between the samples in the dataset. But they are not good at coming up with something entirely new, something outside the boundaries of the dataset. Their output may be unique but they still remain within the boundaries of their datasets.

Until we develop models that can extrapolate well and come up with a metric of creativity that AI can optimize for, neural networks will remain as interpolators without any inherent creativity.

Q10: Can machines be creative? Do you think AI systems producing autonomous art could one day deserve to be called artists in their own right?

Very good question. Whether machines can be creative is a philosophical question. The answer depends on your stance on the possibility of computational creativity. My stance as of now is more on the side of AI as a tool in art, rather than AI being the artist, but I keep an open mind.

One day perhaps we can call AI systems non-human artists. But today, they are far from being fully autonomous when it comes to creative tasks. As I mentioned earlier, what generative AI models do today doesn't go beyond imitation and interpolation.

As a toy example, we can teach an AI model to divide one number by another by showing it some examples. Division operation is actually not so easy for neural networks to learn. Still, the model would be able to approximate it as long as the inputs are within the range of the examples in the dataset. But it would most likely fall apart if we ask it to divide numbers that are out of range of the samples in the dataset unless we employ some special tricks, like operating in the logarithmic space where division becomes subtraction.

To build truly creative models, we need them to go beyond what they see. AI systems need metrics to optimize. If we mathematically quantify creativity and manage to build models that can extrapolate beyond what's in the dataset, then I think we may call those AI models artists in their own right.

Q11: Do you feel you are depending on AI to be creative?

I have experimented with different forms of computational art in the past. To create some of them, I didn't use AI at all. But as far as AI art goes, by definition, I do depend on AI art models. I don't perceive this as something negative or restrictive though. I have control over pretty much all aspects of my AI art models, including the model architecture, datasets, and the other algorithms I use in pre and post-processing steps.

Q12: Some people fear that AI may render human artists unnecessary in the future. What do you think?

At some point in history, I'm sure some people also feared that photography would spell the end of art as we know it. But, that didn't happen. We just got another form of art in addition to the ones that we already had. I think the same is happening with AI art now. AI art may never become as popular as photography but it already provides some tools for those who want to experiment with it.

I don't think AI art is something artists should fear. AI-powered art tools don’t replace artists, they empower artists.

Many people are indeed predicted to lose their jobs as AI and automation systems get better. For example, autonomous driving systems can render cab and truck drivers unnecessary in the future but I don't see that happening for artists. There's also no financial incentive to render artists unnecessary. Trucking companies may want to fully automate their fleets to cut costs. But I'm not aware of any large-scale, highly profitable art companies that may want to get rid of their artists to become more profitable.

Q13: If someone was to question your role as an artist because you are not mechanically creating the artworks yourself, how would you respond?

I actually don't mind if people question my role as an artist. I have a background in computer science and engineering. I'm aware that I’m kind of an outlier in the art world.

I agree that the role of an artist in AI art is different from the role of a painter. I see AI art similar to photography and cinematography in that sense. Photographers choose their subjects, scenes, and lighting, but their art is not created by hand pixel by pixel. I think the same applies to AI art. AI artists guide their production using different forms of AI systems, rather than using cameras and image editing software. They build compositions using the tools they have to convey a message or to evoke emotions, similar to a photographer or a cinematography artist.

Q14: What is an artist’s role in AI art?

An artist’s role in AI art or any kind of art I think is to create something new, something that has never been done before. Artists always find ways to innovate with the tools they have. Researchers create new tools and artists discover new ways to interact with those tools. Some people do both at the same time. Inventing their own tools and using them. Without innovation, all AI art would start looking the same, and wouldn’t be called art at that point.

Q15: Would you compare AI art with concept art, where the artist's idea is more important than the actual artwork?

Yes, that would be a good comparison. Humans are good at coming up with concepts whereas AI is good at filling in the gaps. I think ideas are what makes art 'art,' but I care about aesthetics too. In that sense, I think filling in the gaps beautifully is also important.

Q16: What do you think about the perception of AI art in the art community? Do you think AI art will change people's perception of art?

I am happy to see that AI art, and creative coding in general, are getting more and more accepted in the art community. I don’t think AI systems will change people’s perception of art more than cameras did in the late 1800s. Sure, there will be some changes. Some art techniques will fall out of favor, some will be transformed, and some will be invented. But I think the meaning of ‘art’ will stay more or less the same.

Q17: What opportunities do you see about using code and AI systems in art?

I see limitless creative opportunities for art and culture, using AI and creative coding.

Code is a very versatile medium for art and creative coding allows for crossovers between seemingly unrelated fields. Mathematicians and artists can collaborate on art projects, using code as the shared medium.

Sometimes, crossovers between art and other fields happen accidentally. For example, during my studies at the University of Texas at Austin, I worked on image processing and machine learning methods to analyze satellite imagery, as a part of my doctoral research. One day, tuning some parameters in my code outside their expected range resulted in this piece of art.

I think AI art pushes analytical people to be creative and creative people to be analytical, expanding the overlap between art and science.

In today's world doing one thing, and doing that only thing very well is valued more than doing a lot of things. Someone who may be praised as a polymath in Renaissance could be called a "jack of all trades" today, in a somewhat derogatory way. This perception may change as people start exploring what's a little outside their focus.

I see AI art, and creative coding in general, as an opportunity for both artists, researchers, and engineers to step outside their comfort zone.

Q18: Do you also see problems with AI art that may arise in the future?

At some point, AI models may reach a level where they can create imagery that looks nothing like anything else, yet looks beautiful. They may generate art that maximizes certain human emotions. Pictures that make you happy or angry when you look at them without knowing why.

Today, we already have AI models that emulate human judgments of image quality and aesthetics. We use those models to assess and improve image quality. What if we had the same for quantifying and maximizing certain emotions? That could be exploited in unexpected ways. For example, some shady businesses could use some wall art that evokes the feeling of trust to manipulate their customers. Fast food companies could use imagery that makes you feel hungry and eat more than you need. Politicians could use them to get people to trust them and be angry at others.

This is pure speculation though. Even image quality assessment models are not that robust today. Hacking into people's brains using visual stimuli is something on a whole different level. I don't think we are anywhere near that kind of technology. By the time we get there, if we ever do, we will probably have measures and protection mechanisms against it too. So, I don't see it as something to worry about too much.

Alright, that was pretty much it. I hope you liked it. Check out my artwork on Instagram and SuperRare. Thanks for watching and see you next time.

My Art on Social Media

Recommended Technical Papers