Imagine seamlessly switching between an intense Call of Duty match and mirroring a pivotal Warzone battle royale moment on your phone. A new Activision patent outlines technology to synchronize gaming and video viewing, potentially transforming how we consume entertainment.
The patent filing from Activision predates Microsoft’s acquisition, but could now benefit Xbox specifically as an exclusive feature. Microsoft has invested heavily in cloud gaming technology, and this type of real-time interactivity aligns with that vision.
The patent describes techniques for monitoring live streams to identify key events as they occur. Tools like machine learning models, trained to recognize visual and audio cues, would pick out pivotal moments. These could range from big plays in esports to signature moves in streamed shows.
When an event is detected, the system can instantly generate and prompt contextual mini-games or gameplay buffs that complement the action on-screen. For example, you could be playing Call of Duty on Xbox and second-screening a tense Warzone stream on Twitch on your mobile.
The technology would monitor the stream and detect when an intense final circle firefight breaks out. It could then instantly prompt you to play a quick mini-game on your Xbox, synchronized to mirror the real-time Twitch combat simultaneously. Accepting the prompt overlays the mini-game on your display, allowing you to experience the gunfight in tandem with the streamers on the TV .
In another scenario, your favorite hero landing a massive ultimate ability in an Overwatch League match could unlock that same ultimate for you temporarily in Overwatch 2. Or a big tournament victory in Hearthstone Battlegrounds could grant you bonus cards. Similarly, Forza Horizon could sync weather conditions and showcase events with the real world location where you are racing.
The patent also suggests in-game rewards or buffs tied to real events, but given out in real-time. Your favorite Call of Duty pro scoring a quad kill could give your loadout a temporary accuracy boost. Or an Overwatch hero achieving a goal could prompt related power-ups. A clutch play from your preferred pro in a Fortnite tournament might give you bonus XP and cosmetic rewards in the regular Fortnite game.
By acquiring Activision Blizzard IP, including this patent, Microsoft is positioned to potentially integrate this technology into Xbox Game Pass subscriptions. It fits into Microsoft’s broader vision for interactive Xbox gaming and streaming experiences. Microsoft could develop Xbox-exclusive implementations leveraging Activision franchises like Call of Duty and Overwatch.
One major technical challenge with the technology is quickly recognizing video context and responding dynamically with tailored gaming elements, but even partial success synchronizing big on-screen moments with optional gameplay could make for more engaging viewer experiences.
Machine learning and AI would monitor the audio and video feeds of broadcasts to identify key events as they happen in real-time, with the algorithms being trained on many hours of gameplay footage to recognize the visual and audio cues associated with exciting moments like rapid eliminations or narrow escapes.
If Activision together with its new owner, Microsoft, can overcome some complex hurdles, which we have to admit seem fairly large at this particular point in time, this vision for converging viewing and gaming could significantly impact entertainment. It’s also worth noting that with the rapid advances in machine learning currently taking place, robust real-time analyzing could be feasible soon. Microsoft also has a 49% stake in OpenAI, the company behind Chat-GPT and a number of other cutting-edge AI products.
Remember too, that the technology could also be licensed to other companies. To take a sports game example, which would seem to mesh best with the underlying second-screen engagement theme of the patent, you could be playing a standard FIFA match on your console, while keeping an eye on the local football match on another device. The technology could detect when a tense penalty shootout begins. It would then instantly prompt you to play a FIFA penalty kick mini-game on your console, timed perfectly to mirror the penalties that are taking place simultaneously in real-life. Accepting the prompt would overlay a mini-game, allowing you to experience the penalties in tandem with actual players, for greater immersion.
Ultimately, this vision combines the stories of TV and film with gaming’s active participation. It appears that Activision is keenly aware that we increasingly watch streams while gaming simultaneously, and we’re certain that Microsoft is too, and this technology aims to complement that lifestyle, keeping us immersed across both screens.
