Blizzard patent will generate AI images to flesh out games

:

Techniques are described for a multiple-phase process that uses machine learning (ML) models to produce a texturized version of an input image. During a first phase, using a pix2pix-based ML model, an automatically-generated image that depicts structured texture is generated based on an input image that visually identifies a plurality of image areas for the structured texture. During a second phase, a neural style transfer-based ML model is used to apply the style of a style image (e.g., a target image from the training dataset of the pix2pix-based ML model) to the texture image generated at the first phase (the content image) to produce a modified texture image. According to an embodiment, during a third phase, the generated texture image produced at the first phase and the modified texture image produced at the second phase are combined to produce a structured texture image with a moderated amount of detail.

While this is a very technical and complicated description, what it boils down to is using an existing image for the A.I.'s machine learning to be trained on. It will then emulate the style and generate, via the described process, a "new" image. Most A.I. generators today such as Stable Diffusion, Midjourney, Dall-E, and others follow this fundamental process, albeit on a larger scale by using billions of sourced datasets through a diffusion and denoising system.

Going further into the details of Blizzard Entertainment's patent, the reasoning for the development can be found in the background section:

Art-based products, such as video games, animated movies, and other products, require large quantities of high-quality artwork in order to be marketable. For example, art-based products that are in a hand-painted style include particularly high-quality artwork, which require highly-skilled and trained artists a significant amount of time to produce. As an art-based product becomes more intricate, the scope of the required artwork rapidly increases. This problem of scope is exacerbated when it comes to interactive media, such as video games, where high-quality artwork is required to flesh out interactive digital worlds, and the quality of artwork is directly related to marketability of the interactive media-based product. Many times, these art-based products are subject to release deadlines, and it can be difficult for a team of artists to produce all of the artwork required to meet such deadlines.

The gist of it is that this patented system will theoretically save time and effort in generating common textures for game assets. But there are potential issues regarding this use of A.I. generated imagery.

The U.S. Copyright Office has already made it clear that images generated by A.I.

cannot be protected by copyrights

without substantive human authorship involved. Another major issue is that the main bulk of datasets used by A.I. image generators are from proprietary content from artists, designers, photographers, and other creative without consent nor compensation. There are already

major lawsuits

filed against A.I. image generators concerning this very issue.

The second issue mentioned above might be circumvented by Blizzard if the only datasets the company uses in the process are all Blizzard's own art and graphics. However, the current technology A.I. art generators only function as well as they do due to the benefit of having access to billions of sources. Anything less would potentially generate very low quality output from the diffusion and denoising process.

Will we be seeing the use of this technology in upcoming Blizzard releases, such as World of Warcraft expansions,

Hearthstone

sets, or this year's Diablo IV? Possibly, but does Blizzard have enough proprietary art and images to accomplish an ethical output for their image generation of game texture assets with sufficient quality? Furthermore, the Federal Trade Commission (FTC) and other agencies have mandated guidance on A.I. powered products and services. With the

recent issues

concerning parent company Activision Blizzard (NASDAQ: ATVI) and the Microsoft buyout, the use of potentially controversial A.I. in their video games may potentially be ill-timed.

Blizzard President Mikey Ybarra responded to the news report stating:

Blizzard will always strive to maintain Blizzard quality. You're trying to associate recent AI advances (generative AI) to something completely unrelated. Our approach at Blizzard is to use machine learning and AI in ways that are additive, empathic, and allow our talented teams to spend more time on the highest quality creative thinking and tasks.

https://twitter.com/Qwik/status/1654542974276206609

Furthermore, Ybarra references the response of Brenden Sewell, Blizzard Design Manager:

I suspect a lot of this would be cleared up had the patent the article was about actually been linked in the article. This isn't AI art. The ethics considerations of this are about the same as the ethics of a photoshop filter. It's a tool to save time creating variant textures.

https://twitter.com/BrendenSewell/status/1653977425141956608

However, it should be noted that the links to the patent were indeed provided and the relevant text quoted directly in the report to avoid such confusion.
https://www.youtube.com/watch?v=KsNDMHvz98M

Blizzard Entertainment's Diablo IV is expected to be released on June 6, 2023, with a pre-purchase available right now.

{ "@context": "https://schema.org", "@type": "FAQPage", "mainEntity": [{ "@type": "Question", "name": "What is the new Blizzard Entertainment patent for?", "acceptedAnswer": { "@type": "Answer", "text": "The Blizzard Entertainment patent is for 'Machine learning-based 2D structured image generation,' which establishes a process to generate A.I. art for assets in their video games." } }, { "@type": "Question", "name": "Why did Blizzard Entertainment develop this A.I. system?", "acceptedAnswer": { "@type": "Answer", "text": "According to the background section of the patent, this A.I. system will theoretically save time and effort in generating common textures for game assets, which require highly-skilled and trained artists a significant amount of time to produce." } }, { "@type": "Question", "name": "What are the potential issues with this use of A.I. generated imagery?", "acceptedAnswer": { "@type": "Answer", "text": "There are potential ethical concerns, including the fact that images generated by A.I. cannot be protected by copyrights without substantive human authorship involved, and that the main bulk of datasets used by A.I. image generators are from proprietary content from artists, designers, photographers, and other creative without consent nor compensation." } }, { "@type": "Question", "name": "Will we be seeing the use of this technology in upcoming Blizzard releases?", "acceptedAnswer": { "@type": "Answer", "text": "It's possible that we'll see the use of this technology in upcoming Blizzard releases, but Blizzard may need to ensure that they have enough proprietary art and images to accomplish an ethical output for their image generation of game texture assets with sufficient quality." } }] }