We all know TVC directors love to find any excuse to fiddle with shiny new toys. As they should. But, most volume studios aren’t necessarily brimming with TV commercials. And maybe that’s not entirely surprising… Is the hesitancy around virtual production about to change? Will AI help make digital pre-production costs and lead-times less prohibitive?
Enjoy this brief yet somehow rambling blog about virtual production!
JUMPLINKS
Virtual Production: Quick Guides
As a quick primer, the first video linked below gives a reasonably rough but informative overview of virtual production. This includes lighting and digital environment creation.
Next up, a video from Netflix Production Services: a slick and in-depth BTS focusing on the use of virtual cinematography on 1899.
If you want to be hip to the lingo, here is an incredibly well-structured glossary of virtual production techniques, roles, and terms.
Additionally, if you’d like to learn more about virtual production workflows, you can check out this article to view the lessons learned from The Mandalorian and other projects.
Here is an article that covers the basics of virtual production, including pixel pitch and the restrictions of the workflow. Likewise, this article has a list of six useful tips for shooting on a volume stage. Notably, the current tech doesn’t love fast camera movements. It’s also common to get a moire effect when focusing on the screens.
Lastly, here’s a comprehensive directory of virtual production stages in the US via ProductionHub.
Virtual Production in Advertising: Key Take Aways
Liberating…yet Restrictive
There are tons of creative possibilities inherent to the virtual production workflow. Plenty of potential pratfalls too. At present, the technique seems to excel with specific types of scenes and settings. In-camera visual effects (ICVFX) are great for epic or fantastical shows with plenty of resources and lead times. This doesn’t always suit the seat-of-your pants approach to TVC production, although agencies are starting to see the benefits and economies of scale XR production.
Thus, to meet the demand, more and more standing stages are popping up. As result, this is cutting the costs of needing to assemble a bespoke volume studio for each project. With writers and actors on strikes, this glut of XR stages is making it more worthwhile for these studios to more actively court advertising work. But notably, even big shows like The Mandalorian, which pioneered the use of virtual production, have opted for a hybrid of location and XR stage production.
Saves Costs and Time…sometimes…
Good or bad, shooting in XR has a profound effect on the entire filmmaking workflow. Lead times can be tricky, since the content needs to be created prior to shooting. This is not always well-suited to the typical fast turnarounds in advertising which don’t benefit from the economies of scale of film & TV workflows. It’s also not easy for producers how to know how to safely budget for this type of pre-production visual effects work.
Trends We are Seeing
As ghostwriters for director’s treatments, we’ve noticed plenty of initial interest in XR stage workflows for certain projects, followed by a quick reality check once the numbers come through.
We are also noticing bigger agencies starting to create their own XR systems and studios, knowing that it allows them to generate much more content at scale.
Key Types of Virtual Production
LED Volume: This technique involves rendering CGI (and sometimes filmed) elements in real-time during filming. It allows actors to interact with virtual objects or environments while seeing them in real-time through monitors or headsets. This necessarily involves extensive asset creation to generate the digital environments.
Virtual cameras: Virtual cameras are used to capture scenes within entirely digital environments. They provide filmmakers with the freedom to move and explore virtual spaces, mimicking traditional camera movements.
Motion capture: Motion capture uses motion tracking to capture performance data from actors.
Projection-based XR: Same general concept as LED volume production, although the use of projectors has unique restrictions.
Virtual Production: An Overview
The Mandalorian ushered in a new era of cinematic virtual production, revolutionizing the filmmaking process by combining technology from the worlds of moviemaking, live broadcasting, VR, and gaming.
Volume studios usually consist of large LED walls, although sometimes the effect is achieved with projectors. These Extended Reality Stages create an immersive ‘wrap around’ environment. This allows you to see the live action and virtual worlds combined, in real-time and in-camera. The latency is minimal, meaning that the digital projections can also adapt to the way the camera moves, as it happens.
One distinct advantage of virtual production is the added control over lighting, reflections, and interactive elements. It also gives an added sense of immediacy, for both the actors and the filmmakers.
But that’s not all…Virtual Production and Green Screen Hybrids
By shooting at high speed (along with equally high refresh rates on the screens) filmmakers can also capture a wealth of extra data. This is done by harvesting (and feeding back to the render engine) data from the ‘in-between’ frames that would usually be deleted when you speed the footage back to a normal frame rate. Thus, you can shoot ‘layers’ with green screen, tracking markers, and XR. All on the same sensor, and with minimal time offset. Ghostframe is an interesting example of this technology.
Real-time Rendering
A game-changing aspect of virtual production is real-time rendering technology that essentially digitally composites multiple elements in-camera during filming. This real-time feedback loop facilitates better decision-making and enhances collaboration between the director, cinematographer, and visual effects team.
Lighting For Virtual Production
Enjoy these articles about lighting and pixel mapping for virtual production.
This video explains the basics of Pixel Mapping, a technique that syncs LED panels via DMX to the imagery on the volume.
This article from British Cinematographer speaks to lighting for XR stages. Also note, there is brief mention of the restrictions of realtime rendering (ie. issues when panning the camera too quickly for the system to catch up).
Virtual Production: Pros and Cons
Advantages of Virtual Production
- Immersive environments with precise control over lighting and reflections
- Cost efficiencies and time savings at scale
- Extensive assets and libraries within Unreal Engine workspace
- Creative flexibility and real-time collaboration
- Ability to sync lights with the content on the volume
- Sustainable practices
- Versatile applications
- Integration with CGI and motion capture workflows
- Possibility of capturing a hybrid of chroma key/tracking data/XR footage
Disadvantages of Virtual Production
- Not suitable for all workflows and storytelling types
- Cost of setup, particularly when combined with traditional location or stage-based shooting
- Need for extensive pre-production planning and asset creation
- Limitations of realism, scale, and physical interaction
- Not well suited to certain lighting conditions
Hybrid Green Screen & XR
Filmmakers can capture both green screen and XR content at the same time. By utilizing high frame rates on the camera, in sync with high refresh rates on screens and projectors, the footage can include both green screen and ‘pre-comped’ data.
Since advertising agencies are notoriously prone to pixel-fudging in post, it’s likely that workflows that allow for traditional vfx approaches alongside XR will be increasingly popular.
Virtual Production in Advertising 2023
Slow initial uptake, except in automotive
Virtual production is fast becoming an advertising industry trend, although it seems that its adoption has been relatively slow. With the exception of the automotive sector, which has always been open to shooting driving scenes in a ‘process studio’ setup.
Agencies looking for partnerships with XR studios, some creating their own
Recognizing the transformative power of virtual production, advertising agencies are increasingly seeking partnerships with volume stage studios and post houses who can offer hybrid solutions on quick turnarounds.
Smaller scale solutions, including mobile LED volumes increasingly popular
While large-scale virtual production setups may be cost-prohibitive for some advertisers, smaller-scale solutions are gaining popularity, including bespoke and mobile LED volumes.
Examples of Virtual Production in Advertising
Here is an automotive TVC example seamlessly mixing XR stage with live action cinematography. Currently, one of the dead giveaways of XR footage is the hyper shallow focus look, but this may change when realtime rendering systems become better able to pull focus on the content displayed on the LED walls.
And here’s another Car Ad example, shot on a volume stage. In general, epic landscapes are well suited to XR production. Note the use of in-camera elements (including snow flurries) to create less of an A over B look to the footage.
Set transitions are also ideally suited to volume stage production, as seen in this TVC for Hawaiian Airlines.
Creating XR Content
New roles in creating XR content
The rapidly evolving field of virtual production has given rise to new roles and responsibilities in the creation of XR content. Traditional filmmaking and video game development have merged with cutting-edge technologies, resulting in a demand for specialized professionals who can bridge the gap between the physical and virtual worlds.
One such role is that of the Virtual Production Supervisor, who oversees the entire process of creating XR content. This individual acts as a liaison between directors, cinematographers, and technical teams, ensuring seamless integration of real-world elements with digital environments. Their expertise lies in understanding both traditional filmmaking techniques and the intricacies of virtual reality.
Another emerging role is that of the XR Content Designer. These designers possess a deep understanding of interactive storytelling and user experience design. They craft immersive narratives that engage users within virtual environments, blending elements from film, gaming, and animation to create captivating experiences.
AI and Virtual Production
Emerging Possibilities: Generative AI in Virtual Production
The advent of generative artificial intelligence (AI) has opened up exciting possibilities for creating XR content. Generative AI algorithms can autonomously generate vast amounts of diverse content based on predefined parameters or training data. In virtual production, this technology offers immense potential for automating certain aspects of content creation while maintaining creativity and uniqueness.
Artificial intelligence is expected to play a significant role in enhancing virtual production workflows, revolutionizing the way films, television shows, and other media content are created.
Conclusion: Wrapping Up Virtual Production
Since the tech is evolving rapidly, and bespoke and DIY solutions are emerging just as quickly. As ad agencies start adopting XR and AI into their workflows, expect a plethora of ICVFX content—good, bad, and truly dodgy—on your screens in the near future.