Role: Development of the experience, in association with Future Colossal
To accompany the marketing strategy associated with the launch of the fragrancy GOLD by JAY Z, the decision was to develop an experience that included a simulation of liquid gold with which the users could interact, affecting its behaviour in real time.
To keep in line with the brand, the experience was required to have a component of spectacularity. In order to do this, it was installed in a storefront, with a 4K resolution screen that allowed users to interact with the liquid gold at the best possible resolution; when users played with the installation, the virtual gold was absorbed by their silhouettes, giving way to the exhibition of one of several videos associated with the brand.
Simulating gold in realtime, in 4K
At the beginning of the project, the gold simulation was implemented using a particle based fluid. However, the results weren’t of the expected quality, and so the approach was changed, using instead a fluid simulation that used waves, and solving the associated equations using the Fourier transform. This allowed the creation of a surface that simulated the behaviour of a fluid, with the expected quality.
With the mathematical problems of the simulation solved, it was necessary to solve the challenges associated with the representation of the metal at the resolution and with the performance needed. To get the simulation running in real time (more than 30 frames per second) and at so high a resolution required multiple optimizations and the use of advanced computer graphics techniques, such as Screen Space Rendering and Cube Mapping.
Using the silhouettes to absorb the gold
The simulation of the gold absorption by the users’ silhouettes was implemented using Image processing techniques and the silhouettes generated by the depth camera of the Kinect sensor.
Integrating brand elements with the simulation
To include brand elements (such as videos) in the experience, it was necessary to use strategies that would made credible and transparent the integration. For this, the objects generated by the simulation were modified, using height maps defined for each one of the shown brand elements, in order to make them appear part of the gold and the whole simulation.
Passersby recognition when sunlight affected the performance of the Kinect sensors
The depth camera in the Kinect sensors is affected when it’s directly exposed to sunlight, due to its use of infrared rays. In this cases it was not possible to get accurate information from the sensors in order to control the interaction with the user’s silhouette, and so it was necessary to use an independent control strategy using a web camera which output was processed applying facial recognition techniques and motion detection to recognize the users and control their interaction with the liquid gold.
Video produced by Future Colossal
Kinect. To detect passersby and control their interactions with the installation.
Facial Recognition. As an alternative strategy to detect users when the Kinect was affected by the lighting conditions.
Simulation. To generate the liquid gold that was the conceptual center of the experience.
During the time the installation was active, thousands of passersby interacted with it. The success was such, that even though the experience was supposed to be active for 2 weeks, the client decided to keep it running for a total of 6 weeks, generating thousands of tweets about it and the brand.