Stitching photographic images – eliminating errors in stitching.

Background

Stitching a series of image tiles captured in a mosaic into a coherent image is a frequent requirement in photogrammetry, mapping, etc. Current software packages appear to use a common methodology of bottom up matching patterns, by simplifying the image on each tile, merging these images, then filling in the details from the original tiles.  However and increasingly, the tile images are captured across a tightly controlled imaging network, so the positional relationship between the edges, or overlaps, of the tiles are available.  The standard method of stitching largely ignores both these interfaces and the internal structure of each tile.  As a result, confusing tiles result in misaligned and distorted mosaics (fig.1) which are not tested and corrected against the detail in the original tiles. Researchers currently have considerable difficulty in finding a methodology that will reliably stitch such data sets.

In many fields (Google Maps is the most obvious application), high resolution images are readily obtained for small areas of the object of interest.  For use, these tiles must be stitched to create a wider field image at the same resolution and without loss or distortion of the detail captured.  This current project is part of the creation of an online catalogue of JMW Turner’s C19th prints. The engravers of the time achieved a resolution of line and image that could exceed the resolution of the paper substrate, and has been rarely matched since.  One aim of the project is to provide digital images of each print that match the resolution of the original, which can vary from ca 1200 to ca 3000 dpi.  The prints vary in size from 5×5 to 700×800 cm.

Description

The researchers want a reliable method that does not require many iterations of settings to optimise (but rarely do) the stitching of each image.  There are two basic approaches depending on the type of source data being used.  The creation of panoramas from a fixed camera position requires considerable adjustment, i.e. distortion, to align tiles in the perspective geometry.  This approach has informed and underpinned many available stitching programs. Examples are PTGui  fig. 2 (https://ptgui.com/), Agisoft fig.3 (https://www.agisoft.com/).

However, the scientific community usually starts with a set of images taken from a mesh of camera positions, each with a more or less orthogonal geometry, which might be a microscope slide (https://pages.nist.gov/MIST/ and https://abria.github.io/TeraStitcher/) or the Milky Way (http://montage.ipac.caltech.edu/).  It is proposed that this additional location data is incorporated into the stitching algorithm in order to reduce the tendency of image matching to create false matches.

Because there is usually a considerable overlap between adjacent tiles, there are few places in the overall image that do not contain the true local image.

The preparation of the mosaic needs to have a correction stage where the constructed interfaces can be compared with, and corrected by, the known true image.

Deliverables

A software package to create mosaics from well characterised sets of tiles, with minimal misalignments and distortions.  The required input parameters and data structure should be explicit and readily provided by the person preparing the set of tiles.

A number to training sets can be provided, captured in various ways.

Image Results References:

Fig 1. A cascade of misalignments of horizontal lines, seen on the left and on the far right. This image was prepared with Microsoft Image Composite Editor from a set of 8 scans of a Turner print, each 1.9 GB, R652_i_cvh_cvh3092_PM_stitch.tif. ICE is one of the more reliable stitching programs.

Fig. 2  A more obvious misalignment, similarly the result of MCI, R207_i_cvh3337__stitch.psb

Fig. 3  A typical misplacement of tiles by PTGui, R699_PTGui_frame_stitch.jpg.

Fig.4   A mosaic prepared from a set of tiles using Agisoft. The edges of the rectangular set of tiles were not recognised by the software. R650_etc_cvh0252_AgisoftExportOrthoMosaic

-Image test results from a suite of possible tools: url: 2022 https://visualisationmatters.com/2022/11/03/msc-project-proposal/
-Horie, C. V. (2018), “Turner’s Prints: Reassessment using Digital Tools, leading to an Online Catalogue Raisonne”, Turner Society Newsletter 130 (Autumn 2018), 14-19. URL: http://www.turnersociety.com/magazine/
-Wei LYU and Zhong ZHOU and Lang CHEN and Yi ZHOU “A survey on image and video stitching” Virtual Reality & Intelligent Hardware, Vol 1, number 1, pages 55-83, 2019, issn 2096-5796, doi https://doi.org/10.3724/SP.J.2096-5796.2018.0008, url https://www.sciencedirect.com/science/article/pii/S2096579619300063
-Chen, YS., Chuang, YY. (2016). Natural Image Stitching with the Global Similarity Prior. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds) Computer Vision – ECCV 2016. ECCV 2016. Lecture Notes in Computer Science, vol 9909. Springer, Cham. https://doi.org/10.1007/978-3-319-46454-1_12