GPGPU Image Processing

Summary

Since the 1990’s graphics cards have developed quickly from a primitive drawing device to a major computing resource in a PC. High end Graphics Processor Units (GPUs) have already far more transistors than a typical CPU. Also, they devote the majority of these transistors to computations whereas a large percentage of a CPU is occupied by caches.

We have started research on implementations of partial differential equation (PDE) solvers in graphics hardware in 2000. At this time the GPUs were very restricted in the precision of number formats and the programmability. Their main advantage was the much higher memory bandwidth as opposed to a PC. Many image processing applications pose exactly these requirements and allow these restrictions. They involve large image data which needs to be transferred quickly and do not need ultimate precision for exact computations, but rather a faithful reconstruction of the image evolution known from the continuous PDE model. In case of the non-linear diffusion these are the decreasing diffusivity in areas of large gradients and the smoothing in image regions which are expected to be apart from edges (Fig. 1). Whereas for the level-set evolution these are the fast front propagation in homogeneous regions and the deceleration of the front at segments’ borders (Fig. 2).

With the advent of floating point units in DirectX9 GPUs not only the number format has changed but also the balance between the bandwidth of the video memory and the computational power of the GPUs. Previously, the bandwidth basically sufficed to provide all processing elements of the GPU with individual data. But floating point computations on GPUs nowadays imply - similar to micro-processors - a bandwidth shortage. As a consequence operations with higher computational intensity should be executed, i.e. several operations should be performed on each read data item. The greatly increased programmability also supports the implement ion of more complex algorithms and allows the incorporation of more advanced numerical methods. The task of image registration requires to find a deformation between two images which minimizes a certain energy, e.g. intensity differences. We implemented a cascaded gradient flow PDE for the minimization of the energy (Fig. 3). The algorithm operates on a multi-scale which is represented by a multi-grid hierarchy with several scales per grid. Efficient multi-grid solvers and an adaptive time-step control accelerate the solution. Without the high level programming languages this complexity could be hardly realized on GPUs.

Figures

Fig. 1: Smoothing of a noisy image by nonlinear diffusion computed in DirectX7 graphics hardware.

Fig. 2: Parallel segmentation of tree branches computed in DirectX7 graphics hardware.

Fig. 3: Elimination of a possible acquisition artifact computed in DirectX9 graphics hardware.

Publications

Robert Strzodka

Hardware efficient PDE solvers in quantized image processing

Ph. D. thesis, University of Düsseldorf, Germany. , 2004

link | bib

@phdthesis{DBLP:phd/de/Strzodka2004,
  author = {Strzodka, Robert},
  title = {Hardware efficient {PDE} solvers in quantized image processing},
  school = {University of D{\"{u}}sseldorf, Germany},
  year = {2004},
  url = {http://www.ub.uni-duisburg.de/ETD-db/theses/available/duett-02242005-000216/}
  urn = {urn:nbn:de:hbz:464-duett-02242005-0002162},
  timestamp = {Sat, 17 Jul 2021 01:00:00 +0200},
}

Robert Strzodka, Marc Droske and Martin Rumpf

Image Registration by a Regularized Gradient Flow. A Streaming Implementation in DX9 Graphics Hardware

Computing, 73(4), 373–389, 2004

link | bib

@article{DBLP:journals/computing/StrzodkaDR04,
  author = {Strzodka, Robert and Droske, Marc and Rumpf, Martin},
  title = {Image Registration by a Regularized Gradient Flow. {A} Streaming Implementation
                    in {DX9} Graphics Hardware},
  journal = {Computing},
  volume = {73},
  number = {4},
  pages = {373--389},
  year = {2004},
  url = {https://doi.org/10.1007/s00607-004-0087-x}
  doi = {10.1007/S00607-004-0087-X},
  timestamp = {Sun, 06 Aug 2023 01:00:00 +0200},
}

Robert Strzodka

Virtual 16 Bit Precise Operations an RGBA8 Textures

7th International Fall Workshop on Vision, Modeling, and Visualization, VMV 2002, Erlangen, Germany, November 20-22, 2002, 171–178, Aka GmbH, 2002

link | bib

@inproceedings{DBLP:conf/vmv/Strzodka02,
  author = {Strzodka, Robert},
  editor = {Greiner, G{\"{u}}nther},
  title = {Virtual 16 Bit Precise Operations an {RGBA8} Textures},
  booktitle = {7th International Fall Workshop on Vision, Modeling, and Visualization,
                    {VMV} 2002, Erlangen, Germany, November 20-22, 2002},
  pages = {171--178},
  publisher = {Aka GmbH},
  year = {2002},
  timestamp = {Thu, 16 Jul 2020 14:54:48 +0200},
}

Martin Rumpf and Robert Strzodka

Level set segmentation in graphics hardware

Proceedings of the 2001 International Conference on Image Processing, ICIP 2001, Thessaloniki, Greece, October 7-10, 2001, 1103–1106, IEEE, 2001

link | bib

@inproceedings{DBLP:conf/icip/RumpfS01,
  author = {Rumpf, Martin and Strzodka, Robert},
  title = {Level set segmentation in graphics hardware},
  booktitle = {Proceedings of the 2001 International Conference on Image Processing,
                    {ICIP} 2001, Thessaloniki, Greece, October 7-10, 2001},
  pages = {1103--1106},
  publisher = {{IEEE}},
  year = {2001},
  url = {https://doi.org/10.1109/ICIP.2001.958320}
  doi = {10.1109/ICIP.2001.958320},
  timestamp = {Wed, 16 Oct 2019 14:14:52 +0200},
}

Martin Rumpf and Robert Strzodka

Using Graphics Cards for Quantized FEM Computations

Proceedings of the IASTED International Conference on Visualization, Imaging and Image Processing (VIIP 2001), Marbella, Spain, September 3-5, 2001, 193–202, ACTA Press, 2001

link | bib

@inproceedings{DBLP:conf/viip/RumpfS01,
  author = {Rumpf, Martin and Strzodka, Robert},
  editor = {Hamza, M. H.},
  title = {Using Graphics Cards for Quantized {FEM} Computations},
  booktitle = {Proceedings of the {IASTED} International Conference on Visualization,
                    Imaging and Image Processing {(VIIP} 2001), Marbella, Spain, September
                    3-5, 2001},
  pages = {193--202},
  publisher = {{ACTA} Press},
  year = {2001},
  timestamp = {Mon, 17 Jan 2005 15:16:16 +0100},
}

Martin Rumpf and Robert Strzodka

Nonlinear Diffusion in Graphics Hardware

3rd Joint Eurographics - IEEE TCVG Symposium on Visualization, VisSym 2001, Ascona, Switzerland, May 28-30, 2001, 75–84, Eurographics Association, 2001

link | bib

@inproceedings{DBLP:conf/vissym/RumpfS01,
  author = {Rumpf, Martin and Strzodka, Robert},
  editor = {Ebert, David S. and Favre, Jean M. and Peikert, Ronald},
  title = {Nonlinear Diffusion in Graphics Hardware},
  booktitle = {3rd Joint Eurographics - {IEEE} {TCVG} Symposium on Visualization,
                    VisSym 2001, Ascona, Switzerland, May 28-30, 2001},
  pages = {75--84},
  publisher = {Eurographics Association},
  year = {2001},
  url = {https://doi.org/10.1007/978-3-7091-6215-6\_9}
  doi = {10.1007/978-3-7091-6215-6\_9},
  timestamp = {Wed, 01 Jul 2020 11:58:34 +0200},
}

Code

Contact

Robert Strzodka