Deepening NBA fan engagement through cutting-edge interactive gaming and immersive technology.
With the NBA All-Star game's return to the classic and much beloved East vs. West format, we designed an experiential activation to celebrate the rivalry in the crossroads of America—Indianapolis. AT&T and the NBA partnered to create an immersive fan experience that transcended the traditional boundaries of fan experiences, featuring three hero activations that melded sports, technology, and art into a cohesive narrative. At the heart of this activation was the integration of real-time body and basketball tracking technology, transforming participants from passive spectators into dynamic avatars within a virtual basketball universe.
Dribble Town
Dribble Town offered a unique challenge, blending elements of classic arcade games with the physical act of dribbling a basketball. Set against a backdrop inspired by East Coast cities like NYC, Miami, and Boston, players navigated their avatars through obstacles to collect stars and rings, enhancing the competitive edge with a "Final Frenzy" mode where the pace intensified, obstacles and rewards surged, and points potentially skyrocketed. The objective? To make your way to All-Star Weekend with as many rings as possible.
Splashville
Splashville transported players to a vibrant West Coast environment for a collaborative pop-a-shot style game with an objective of earning trophies. The LA-inspired setting was brought to life with ball and environment shaders, as well as a dynamic scoring system. It all culminated in a "Final Frenzy" where hot spots moved and every basket made caused visual feedback to radiate across the screen. Western conference mascots and custom made superfans served as avatars, embodying the spirit of competition and camaraderie inherent to the West Coast.
Jumbotron
At the core of both games was the jumbotron, designed as the brain that powered the experience and orchestrated the seamless transition of players between the virtual worlds of Splashville and Dribble Town. Designed to look like a technological innovation of the near future, it was composed of multiple screens, displaying vital game information such as the timer and player scores, along with branded marquees that added an extra layer of dynamic visual engagement.
A standout feature of the jumbotron was the inclusion of NBA star Paul George, who was digitally integrated into the experience. PG's presence on the jumbotron added a layer of celebrity interaction that was both motivating and immersive. To achieve this, a comprehensive array of lines was recorded, creating the illusion of real-time, responsive feedback that reacted intelligently to the players' performance. This intelligent scoring logic allowed PG to offer encouragement, celebrate high scores, and interact with participants in a manner that felt personal and dynamically tailored to the unfolding gameplay.
PG's virtual appearances involved crafting a wide range of permutations for his dialogue, ensuring that his responses felt spontaneous and directly linked to the players' actions and achievements. Integrating real-time feedback from a celebrated NBA figure not only enhanced the realism of the experience but also fostered a deeper sense of connection and engagement among participants, elevating the overall impact of the event.
Avatars
From initial sketches to the final on-screen representation, the creation of avatars was a meticulous process that leveraged AI and advanced motion capture technology. Depth tracking cameras and gesture recognition allowed for real-time avatar control, mirroring players' movements in the game. This seamless integration of tech and creativity offered a personalized and immersive gaming experience, with a selection screen enabling players to choose their digital alter egos through swipe gestures.
The avatar selection, inspired by classic video game character selection screens, served as the gateway to the worlds of Splashville and Dribble Town, featuring a cloud-inspired motif that set the tone for the journey ahead. Players were presented with an array of avatars, including full-scale representations of NBA mascots from the East and West coasts, alongside bespoke characters designed exclusively for the event. This diverse lineup ensured that every participant could find an avatar that resonated with their personal style or regional allegiance.
Utilizing intuitive swipe gestures, participants could seamlessly cycle through the available options, making the selection process both engaging and user-friendly. Once a character was selected, players were prompted to strike a pose—a fun and interactive step that not only personalized the experience but also served as the foundation for the creation of a unique social asset. This moment of personalization deepened the connection between the players and their avatars, setting the stage for their transition into the digital arena where their selected characters came to life against the vivid backdrops of their respective games.
Interactive Mural
The All-Star graffiti wall, a collaboration with artist Dave Arcade, was the centerpiece that symbolically connected the East and West coasts. Utilizing [Tool's] proprietary interactive mural platform, attendees were invited to contribute to a virtual mural using digital paint tools. This activation not only showcased the potential of interactive art in engaging audiences but also emphasized the communal spirit of the event. Daily timelapses captured the mural's evolution, highlighting the collective creativity of the All-Star Weekend attendees.
The Social Takeaway
The social takeaway was crafted to ensure that every participant left with a personalized digital memento. Our approach resulted in three distinct takeaways: one for their performance in Dribble Town, another for Splashville, and a third being a timelapse capturing the collective creativity at the interactive mural wall.
For Dribble Town and Splashville, participants received highlight reels that commenced with the player transitioning into their chosen avatar and striking a dynamic pose that set the stage for the slick sequences that followed. Drawing inspiration from the attitude-laden music videos of the 90’s, the reels were a blend of pre-rendered animations and live footage, all seamlessly stitched together to showcase the player's best moments on the court.
The pre-rendered animations were brought to life using advanced motion tracking technology, which animated the highly detailed, textured, and rigged characters. This process allowed for the creation of a diverse array of shots that were then customized for each mascot and superfan avatar, resulting in a complex patchwork of clips that could be stitched together on the fly. These reels were rendered in the cloud in real-time, ensuring that each participant received a unique and polished product.
Upon the completion of their game, participants were sent their personalized highlight reel via text, allowing them to instantly share their achievements and memories from the event with friends, family, and on social media, extending the reach and impact of the NBA Crossover experience far beyond the physical confines of the venue.
The Tech
What seems to be a seemingly straightforward execution, is actually packed with many different components to make it work.
Hardest challenges:
– Real time 3D people tracking
– Real time 3D ball tracking
– Real time calibrated video capture
– RGB feed
– Real time video stitching
– Double all of the above combined in one seamless scene.
– Avatar Feed
The base setup exists out of 1 frontend and 1 backend and two games. The whole infrastructure was running on 4 computers, each packed with an Nvidia 4090 GPUThe frontend had a user facing portion for registration and an admin section for viewing and administering sessions. The backend was responsible for all communication between the games, session management and database handling. Each game setup was running on two computers. For scene rendering one of the computers served as the main, while the other was used as the sub with a nodejs socket protocol in between for sending out game events like score.
