Towards Automatic Cinematography and Annotation for 360° Video

Omnidirectional (360°) video is a novel media format, rapidly becoming adopted in media production and consumption as part of today’s ongoing virtual reality revolution. Due to its novelty, there is a lack of tools for producing highly engaging 360° video for consumption on a multitude of platforms (VR headsets, smartphones or conventional TV sets). In this work, we describe our preliminary work on tools for automating several tasks in the production of 360° video, which are tedious and time consuming when done manually. We propose tools for automated cinematography (generate a lean-back experience without user interaction for conventional TV sets) and automated annotation of 360° video for simplifying linking to other resources like text or 2D images/videos. Both tools employ deep learning based methods for extracting the information about the objects in the scene. We will discuss the current state of these tools and ways how to improve the tools in the future.