Bibtex

@Article{Saura-Herreros2021,

  author={Saura-Herreros, Miguel and Lopez, Angeles and Ribelles, Jose},

  title={Spherical panorama compositing through depth estimation},

  journal={The Visual Computer},

  year={2021},

  month={Sep},

  day={01},

  volume={37},

  number={9},

  pages={2809-2821},

  doi={10.1007/s00371-021-02239-7},

  url={https://doi.org/10.1007/s00371-021-02239-7}

}

Spherical Panorama Compositing

through Depth Estimation

Miguel Saura-Herreros     Angeles Lopez     Jose Ribelles

Universitat Jaume I

Abstract

Final publication is available at Springer via this link

A spherical panorama is a photography that captures a view of 360 degrees horizontally and 180 degrees vertically. Such image stores a projection of the 3D world on a sphere whose center is the observer position. These images are used as resources for immersing the user in a given environment, which is known as Virtual Reality Photography. Despite the proliferation of these type of visual resources, composition of new spherical panoramas has been scarcely addressed. Image editing software (e.g., Photoshop) is able to manage spherical panoramas and allows users to select a viewing direction and manually isolate contents in layers in order to, for instance, insert an object partially occluded. However, this procedure is time consuming, usually requires a skilled user and must be repeated for each view direction. The use of depths has been proposed as an effective way to make image composition easy. Depth availability allows users to work in the 2.5D space of the image, and tasks like the insertion of partially occluded objects are automatically managed. Fortunately, depth estimation from a single image has been widely studied. Our goal is to obtain the relative depths from a spherical panorama by taking advantage of these solutions for single images. We validate the obtained depth maps by creating new compositions with outdoor panoramas. The user can interactively adjust the relative depth ranges to obtain the objects of interest, while occlusions are managed automatically.

Full paper 4MB

In this paper we propose to work in the 2.5D space of the scene to facilitate composition of new spherical panoramas. For adding depths to spherical panoramas, we extend an existing method that was designed to estimate relative depths from a single perspective image through user interaction. We analyze the difficulties to interactively provide such depth information for spherical panoramas, through three different types of presentation. Then, we propose a set of basic tools to interactively manage the relative depths of the panoramas in order to obtain a composition in a very simple way. We conclude that the relative depths obtained by the extended depth estimation method are enough for the purpose of compositing new photorealistic panoramas through a few elementary editing tools.

Paper

Citation

Miguel Saura-Herreros, Angeles Lopez and Jose Ribelles. " Spherical panorama compositing through depth estimation ". Vis Comput 37(9), 2809-2821, 2021, https://doi.org/10.1007/s00371-021-02239-7