Traditionally, bringing a writer’s vision to life in a game engine has been a slow, painstaking process. Scripting believable character behaviors and interactions requires extensive manual work, leading to potential mismatches between the creative script and the final in-game execution.
This was the challenge at the heart of a Business Finland-funded AI R&D project for a Finnish startup. Vertti Luostarinen, a researcher & developer at CasvuGen, led the team in developing Scene2Plan, a novel framework to bridge the gap between creative writing and game development.
The solution was a powerful AI tool built to orchestrate scenes using Large Language Model (LLM) agents. By creating a purpose-built “scaffolding” system, Vertti and his team enabled LLMs to read a dramatic screenplay and translate it into a series of executable animations within the Unreal Engine. Instead of manually scripting every character movement and interaction, the system could generate an action plan automatically. The framework was tested with several models, including Llama-3.1 and Qwen3, demonstrating a flexible and robust approach.
The results were impressive. The research showed that even with relatively small, resource-efficient LLMs, the framework could generate feasible, executable plans with a surprisingly strong grasp of a scene’s three-dimensional space. By designing the right scaffolding, the team was able to significantly reduce the performance gap between huge, powerful models and smaller ones. This demonstrates that a clever approach to problem-solving is often more important than raw computational power.
While the research revealed that LLMs still have some way to go in reasoning about nuanced non-verbal communication or long-term character motivations, the project proved the immense potential of AI to streamline game development. This technology has the power to act as an intelligent assistant, dramatically speeding up production while keeping the creative human element—the writer’s vision—firmly in the driver’s seat.