Roblox unveiled a new set of AI tools on Monday at the Game Developers Conference that will enable the company’s millions of player-creators to create playable game code and in-game 2D surfaces using nothing more complex than text descriptions.

The release is a significant step toward “democratizing” game creation, taking it out of “the hands of the skilled few” and giving it out to people “who were blocked by some technical limitations but had a great idea” that they were previously unable to express without any specialized skills, Roblox Studio head Stef Corazza told a packed audience at the conference.

The beta version of Roblox Code Assist, released on Monday morning, has the potential to make it easy for users to create basic code snippets. A user can instruct the system to “make orb turn red and destroy after 0.3 seconds when the player touches it,” according to Corazza’s example at the conference. Based on an earlier-provided coder-defined orb object, the system generates a seven-line Lua function that accomplishes this.

Similar to the previous example, another prompt for a function to “create a 3 by 3 grid of orbs around orb” generates a few lines of code to insert the requested objects into the game’s scene.

Roblox AI Game Creation Tools Plan

According to Corazza, it needed to be clarified whether this tool would function well enough for a public release today just four months ago. However, Roblox has benefited from improvements in natural language code generation that have only recently become available.

But it was by tweaking that standard model with code from the Roblox platform itself that the company’s Code Generator Beta could produce usable results. This important context “significantly increases the quality of output,” he claimed.

According to Corazza, the requirement for context also applies to coders who use the tool. A knowledge expert would be asked to take a test “in a completely white room where you didn’t hear the question completely” if one asked the AI to generate code on an empty document. However, Corazza claimed that starting the AI tool with just three lines of sample code in internal testing increased the “acceptance rate” for the tool’s recommendations over attempts with no such “context” code as a starting point by 50%.

According to Corazza, the primary goal of the Code Generator Beta is to “help automate basic coding tasks so you can focus on creative work” and to free up experienced coders from having to “work on simple stuff.” But in the future, according to Corazza, he envisions a more chatbot-style user interface that can serve as a teaching tool, outlining how code operates and providing documentation for those just learning the fundamentals.

In addition to the AI code generator, Roblox has released a Material Generator that streamlines the laborious process of producing flat art assets that are layered on top of the numerous 2D surfaces in a game world.

This goes above, and beyond the type of fundamental image generation, you’d get from a program like Stable Diffusion. Along with other “maps” for attributes like albedo, roughness, and “metalness,” Roblox’s tool also automatically adds a fake 3D normal map to the surface. The game engine can then use those characteristics to generate accurate lighting reflections and reactions from other objects.

But according to Corazza, this is only “step one” of the company’s plans to generate AI assets. The next step is an AI system that can create an entire “specific geometry” that can completely retexture a full 3D model or character beyond flat surfaces. Due to the requirement to understand the full context of the object itself (such as where different body parts go on a living character), this is “a very hard problem to crack.” Still, Corazza said the team has seen “some early breakthroughs,” and he’s “confident it will land” eventually.

But the “holy grail” for this kind of tool can simultaneously imitate a “certain game style,” Corazza continued. The idea is to use a concept artist’s few sketches as a starting point and then have the AI instantly create a whole set of assets that are both stylistically and functionally cohesive.

Someone might be able to type “Scene with a forest, a river, and a large rock” and receive a fully interactive, realistic 3D world that corresponds to the prompt, according to Corazza’s “extreme example” of a potential use case. He predicted that it would feel like nuclear fusion. “Two years [until it is ready],” I will say.

A Distance To Travel

Corazza noted that a platform like Roblox is a natural setting to experiment with these kinds of early, flawed generative test cases despite Roblox’s warning that its AI system still doesn’t always “suggest perfect code.” The “bar is a little bit lower” for AI-generated Roblox code and surfaces compared to self-driving cars, where any AI mistakes could have “massive consequences,” he said. If the generation is bad, nothing terrible will happen; just press the button to start a new one.

However, Roblox is already preparing for the challenge of policing what the company anticipates will be an enormous flood of AI-generated content in the future. To instantly moderate “things created at runtime by tens or hundreds of millions of players,” Corazza said, the company will likely need to create more automated tools.

Despite these difficulties, Corazza was enthusiastic about the coming waves of AI-powered game development tools that will eventually “converge” around the generation of all of a game’s assets—materials, code, 3D assets, terrain, audio, avatars, 3D scene, and images—from a single text prompt. Those future tools will be built more directly around capturing the user’s ” intent,” unlike today’s focus on fine, granular control at the code/vertex level. If and when that occurs, the highly technical position of “game developer” may not look like it now.

Well, there you have it! Keep an eye out for the latest content Nexus of Gaming will be posting soon! Stay tuned.