Seam Carving Algorithm

Note#97

notes
Published

2023-06-11

The Seam Carving algorithm, also known as content-aware resizing or retargeting, is an innovative image resizing technique that strives to maintain the perceptually important parts of an image, focusing on preserving the core visual content when resizing.

Historical Context

The concept of seam carving was introduced in 2007 by researchers Shai Avidan and Ariel Shamir. The technique offered a novel approach to image resizing, focusing on the preservation of content, which was a noticeable improvement over traditional methods like scaling and cropping. Over the years, the algorithm has been refined and is commonly used in applications requiring sophisticated image resizing.

The Algorithm

Greyscale Conversion

The algorithm starts by converting the original color image into greyscale. The greyscale value v of a pixel is calculated using the formula v = round(.299*r + .587*g + .114*b), where r, g, and b represent the red, green, and blue values of the pixel respectively.

Energy Map Calculation

An ‘energy map’ of the greyscale image is computed. This map uses an edge detection function to represent the “energy” or significance of each pixel.

Cumulative Energy Map Computation

The algorithm computes a cumulative energy map after creating the energy map. The cumulative energy map is a matrix of the same dimensions as the image where the value at each pixel is not just the energy of that pixel, but the total energy of the lowest-energy path from the top of the image to that pixel.

To calculate this, the algorithm starts with the top row of the image, where the cumulative energy is the same as the original energy from the energy map. For each subsequent row, the algorithm finds the cumulative energy for each pixel by adding its own energy value (from the energy map) to the minimum cumulative energy of the pixels directly above it in the previous row. The pixels considered are directly above, or one pixel to the right or to the left of the current pixel in the row above. This process is repeated for each pixel and for each row, down to the bottom of the image.

Minimum-Energy Seam Identification

The process of identifying the minimum-energy seam begins once the cumulative energy map is complete. This seam is a connected path of pixels from the top to the bottom of the image that has the lowest total energy.

To identify this seam, the algorithm starts at the bottom row of the cumulative energy map and locates the pixel with the smallest cumulative energy. This pixel is the start of the minimum-energy seam. The algorithm then traces the seam back up to the top of the image. At each step, the algorithm moves to the pixel in the row above that has the smallest cumulative energy and is adjacent to the current pixel. The adjacent pixels considered are directly above, or one pixel to the right or to the left of the current pixel.

This process creates a path from the bottom of the image to the top, representing the seam of least cumulative energy. This seam is then used in the final step of the algorithm: seam removal.

Seam Removal

The identified seam is then removed from the original color image, effectively reducing the image’s width by one pixel. This process is repeated until the image is resized to the desired size.

Advantages and Disadvantages

Advantage

The Seam Carving algorithm excels in preserving the visual content of an image during resizing. It selectively removes less important portions of the image, ensuring that the core visual content remains intact, a significant improvement over traditional resizing methods.

Disadvantage

The algorithm might struggle with images that have a lot of high-energy content, or when important details are evenly distributed across the image. In such cases, important content might be removed, leading to a loss in image quality.

References