Mural Mapping Submission
LUMA Festival 2023
Preface
Each year, the LUMA Projection Arts Festival hosts a community mural mapping feature (now called the Peg Johnston Living Lights Project), which allows local artists to submit still pieces to be displayed on a selected building during the festival, through a digital or physically printed canvas template with an outline of the building features. This is a massive opportunity for artists to dip their toes into the incredibly difficult-to-enter field of architectural projection mapping—in addition to having their work seen by tens of thousands of people over the festival weekend each September.
I typically serve as a Technical Director at LUMA each year, but I decided to take on this extra project as a means to expand my skills in 3D through my first major project in Blender.
Tools: Blender, Photoshop, Spark AR
Location: Atomic Tom’s, Binghamton, NY
The provided template outline for the building.
Goals and Ideation
Prior to starting, I spent some time studying successful projections and taking notes on the aspects that were visually appealing as well as particular projects that were successful on the colorful building I would be projecting onto (adds extra complexity compared to monochromatic stone).
For this project, I primarily wanted to focus on (1) creating a 3D model in Blender. This led me to ideate concepts that could incorporate minimal visual design elements so that I could focus on using the window shapes of the building as actual windows to peer into a room. In order to do this, I decided that I could try (2) experimenting with particle effect tools to create realistic piles of objects, particularly using just one premade 3D model of a rubber duck. From there, my concept became a building full of thousands of tiny life-sized rubber ducks with one giant “boss” duck in the center, partially in homage to my first ever AR hackathon project back in 2018, a multi-player AR game to defeat a giant debugging duck!
“Tracing” the Template in 3D
To get started, I imported the blank template as a reference image into Blender. This allowed me to essentially “trace” along the contours of the template when creating my 3D components. This was straightforward for straight shapes like rectangular columns, but I ended up experimenting with several YouTube tutorials (e.g., 1, 2, 3) in an attempt to shape the arches, which was initially unsuccessful. However, I eventually achieved the correct arch shape by using Boolean modifiers to cut and paste together simple shapes like a cylinder into a half cylinder and then a curve, for example.
Creating the Building 3D Model
Creating 3D Rooms
In order to try and create a sense of depth, I created 3D rooms behind the windows where I would eventually place the piles of ducks. This was as simple as creating a box behind the building façade and deleting the front face of it. From there, I added lighting to try and cast shadows along the contours of the building (and its eventual contents) in order to emphasize its shape.
Experimenting with Particle Effects
Creating a Pile of Objects with Particle Effects
In order to create the pile of ducks, I used a technique that involves creating a particle emitter, converting the particles to objects, adding physics, applying those modifications to all the objects, then tweaking the result until I’m happy with the shape and size of the pile using this tutorial as a reference.
Since the rubber duck model I was using had over 14k triangles, initial attempts would freeze up Blender and crash at different stages of the process. To solve this, I used the Meta Spark mesh simplifier tool that I already had installed to reduce the triangles to 1/3 the size. This caused small distortions in the texture of the model, but the individual ducks in the piles would end up being so small that it didn’t ultimately matter. After that I was able to successfully create the pile, even piling them over a lump to essentially create a “shell” of a duck pile and reduce the overall size of the resulting 3D object.
Incorporating the Ducks into the Building Model
After I finally achieved the duck pile I was looking for, I exported the pile as its own 3D object and imported it into the building project. I then made multiple copies of the pile, varying the rotation and position so they would not look overly uniform. In the final render, the gaps in the piles are obvious but I figured it would be less noticeable when actually placed on the building (which was true).
The final touch was adding a giant “boss/king” duck (which, in hindsight, would have been more fun with a crown or something) on top of the piles.
Final Rendering vs. Projection Result
The final rendering was done from straight-on, as the template required a flat image to project. This perspective ended up showing very minimal contours of the building model I created despite my amateur attempts to achieve some definition through lighting. I had initially attempted to do this by adding bevels to the edges of my shapes, but I discovered various hidden geometry issues that prevented successful application of that technique.
Because of the colored bricks on the building itself, some aspects of the final projection lost that extra sense of immersion, as compared to how it would appear on a monochromatic stone building. However, the bigger problem, in my opinion, is that I used too bright of a material on my building model for it to create that illusion of depth while peering into the building windows that I was aiming to achieve. In fact, it might have been better if I didn’t texture the outside of the building at all and left it black in order to highlight the depth of the inner rooms! I recognize that achieving the ideal look in the projection medium is ultimately about experimentation and experience to see what works!
All in all, onlookers considered the result successful and whimsical, despite my areas for future improvement.
Planned AR Effect
While creating this project, I toyed around with the idea of augmenting the experience through an AR effect with Spark AR. I had two concepts, (1) ducks jumping off the building and floating to the ground on parachutes or (2) a water-filled environment effect with ducks floating along the water. The latter seemed a bit easier to implement (and more of a wow-factor) after finding a water patch, so I decided to go on-site to test it out with a sample duck to judge scale.
Since my priority was my role as Technical Director, I didn’t find enough time to finish the effect before the projection was shown. But it was fun, nonetheless!
Final Thoughts
Lessons Learned
Optimization in the early parts of the project is really important to save time and prevent crashes. My application crashed multiple times when trying to create big particle effects with the duck model as-is. Since the particles were intended to be small anyway, I should have simplified the model first before attempting it (which I did end up doing later on). Similarly, I accidentally created some geometry issues due to my very limited experience with Blender, which made creating the final render that much more difficult.
As far as the projected result, light does not behave in the ways we’d expect it to (at least, for anyone not as familiar with creating art for projection)! I realized that even the medium-gray hues were too bright, illuminating the building too much and creating an overall flatter look to my piece. Next time, I’d like to focus on creating a darker building so the inner contents pop and glow through the edges and create a more convincing illusion of depth.
Future Goals
Understand how to better optimize various aspects of the models before getting too far into the project
Spend more time learning how to join shapes properly so I can use bevel tools and other techniques to create a more defined edge
Test a combination of 2D and 3D portions of the final render (e.g., creating a contrasted 2D outline of the outside of the building if it renders too flat) rather than doing it all in 3D, especially because the template guidelines don’t exactly follow the topography of the building
Learn more about complex patches in Spark AR, especially the downstream inputs and outputs
That's all for now!
That's all for now!
Eager for more?
Click the button below to check out some other projects!