ACTIVITY 13 Correcting Geometric Distortions

In this activity, we generated a SciLab code that can be used to correct distortions, specifically geometric distortions, in an image. These distortions are usually due to the manner the sensor in a capturing device detects light. Any straight line in the object space is perceived by a sensor as a curved line. The two common observable distortions are the barrel distortion and pincushion distortion. Barrel distortion happens when the edges of the image seem to be bent outwards. The image is bulging but smaller than the undistorted image. On the other hand, pincushion distortion occurs when the edges of the image seem to be bent inwards. The boundaries are expanded, so the image is bigger than the undistorted image. However, even with the distortions, the center of the image usually remains undistorted. See Figure 1 for the illustrations of the aforementioned distortions.

Figure 1. Left: undistorted; center: with barrel distortion; right: with pincushion distortion.
(Source: http://toothwalker.org/optics/distortion.html)

In generating the code, a checkerboard image with barrel distortion is used as the sample image as shown in Figures 2 and 3. The correction of the image was done on the two image properties, pixel coordinates and graylevel values. The pixel coordinates were corrected by finding the transformation function for mapping the pixel coordinates in the distorted image to the undistorted image. Moreover, the graylevel values were corrected using bilinear interpolation. A detailed discussion of the correction process can be found below.

Figure 2. Original Image
(Source: http://immersaview.com/images/sol7smile.png)

Figure 3. The cropped portion of the image that was restored to its undistorted form.

Before correcting the image, it is important to have the correct pixel coordinates. In the sample image (checkerboard), these pixel coordinates correspond to the corners of the rectangles. For images with nonperiodic objects/structures, gridlines need to be created in the image, as well as in the reference (undistorted) image, such as those shown in Figure 4. The pixel coordinates will be the intersections of the gridlines.

The pixel coordinates in the sample image were determined after isolating the corner points of each of the white rectangles in the checkerboard. Several image processing techniques were applied on the binarized image to isolate all the corner points. First, the image was reduced to only the structure of the checkerboard being shown by perfoming edge detection using the imcorrcoef() function in Scilab. Then, the image was dilated with a square to make the lines thicker. After which, the image is eroded with a cross so that only the intersections/corners would remain. A series of erosion of the image with different sizes of rectangles (depending on the size of the corners left with the previous erosion with a cross) was then performed. The goal was that only single corner points must remain. The code below ensured that only one point in every corner after every erosion was left.

[image,n] = bwlabel(image);
for i=1:n,
f = find(image1==i);
reg_size = size(f,'*');
if reg_size > 1,
image1(f) = 0;
end
end
image = (image>0)*1;

However, the generated code for determining the pixel coordinates in the distorted images does not work for most images. It would only work for perfect checkerboard images. Another way to determine the pixel coordinates would be to locate the desired points in the image. This is the easiest method but requires so much time and must be accurate in locating the points. Figure 5 shows the isolated single corner points using the generated code. It must be noted that the x coordinate is the column number while the y coordinate is the row number. The said convention was carefully followed throughout the code to avoid errors.

Figure 5. Corners of the white/black rectangles determined after image segmentation.

The next pixel coordinates determined were of the undistorted or the ideal checkerboard. The data needed were the following: height and width of the undistorted (central) rectangle, lowest x and y coordinates. The undistorted pixel coordinates were determined by first creating a matrix of the same size as the image. Then starting from the lowest x and y coordinates, the other coordinates were determined by generating the set of points with a column interval equal to the width and a row interval equal to the height. Figure 6 shows the image of the ideal points, which was created by assigning a value of 1 in the determined ideal pixel coordinates of the zero matrix. The ideal checkerboard was then created from these points as shown in Figure 7.

Figure 6. Corners of the ideal (undistorted) rectangles with the same height and width as the central rectangles (not distorted) of the image .

Figure 7. Ideal checkerboard when there are no distortions at the edges.

After determining the pixel coordinates in the distorted image and the ideal checkerboard, we proceeded to determining the transformation function which was used in mapping the pixel coordinates in the distorted image to the ideal checkerboard (see Figure 8). Equations 1 to 6 are the important equations describing the mapping process. As described in equation 6, coefficients C1 to C8 need to be determined first by using uquation 5. However, the first four equations require the grouping of the corner points into 4 corresponding to one rectangle. Once the pixel coordinates have been grouped into 4, the determination of the coefficients is much easier as illustrated in the code below after the equations. With the coefficients already known, the corresponding undistorted pixel coordinates were determined using equation 6. Note that it must done for all points/pixel coordinates in the distorted image and not only the corner points. The set of coefficients to be used depends on what corner points surround the distorted pixel coordinates to be mapped.

Figure 8.

Equation 1. r and s are the transformation functions for x and y coordinates, respectively.

Equation 2. Expression of the r and s transformation functions.

Equation 3. Matrix equations mapping undistorted to distorted pixel coordinates.

Equation 4. Compact form of equation 3.

Equation 5. Solution for determining the coefficients C1 to c8.

Equation 6. Reverse of equation 3: mapping of distorted to undistorted pixel coordinates.

for i = 1:196,
xC = cornerDx(i,:);
yC = cornerDy(i,:);
xT = cornerUNDx(i,:);
yT = cornerUNDy(i,:);
T = [xT(1) yT(1) xT(1)*yT(1) 1; xT(2) yT(2) xT(2)*yT(2) 1; xT(3) yT(3) xT(3)*yT(3) 1; xT(4) yT(4) xT(4)*yT(4) 1];
T = inv(T);
cx(i,1:4) = (T * xC')';
cy(i,1:4) = (T * yC')';
end;

The next step was to determine the graylevel values of the mapped undistorted pixel coordinates. If the obtained pixel coordinates were integers, the value at that location in the distorted image was its value. However, if the obtained pixel coordinates were non-integers, bilinear interpolation was implemented (see equation 7). Just like in finding the transformation functions, the coefficients a, b, c and d need to be determined first. Similar equations as equations 3 to 5 were used in determining the values of this constants. For the 4 points needed, its four neighboring points were used. Below is the code used to map the distorted points to the ideal points and calculate its graylevel value.

Equation 7. Bilinear interpolation

imagenew = zeros(s(1),s(2));

for i = 1:196,
for y = min(cornerUNDy(i,:)):max(cornerUNDy(i,:)),
for x = min(cornerUNDx(i,:)):max(cornerUNDx(i,:)),
xnew = sum(CoeffX(i,:) .* [x y x*y 1]);
ynew = sum(CoeffY(i,:) .* [x y x*y 1]);
if xnew/floor(xnew) > 1 | ynew/floor(ynew) > 1,
xnew1 = floor(xnew);
ynew1 = floor(ynew);
neighy = [xnew1 xnew1+1];
neighx = [ynew1 ynew1+1];
dy = xnew - xnew1;
dx = ynew - ynew1;
vnew = dx*dy*image(neighy(1),neighx(1)) + (1-dx)*dy*image(neighy(1),neighx(2)) + (1-dx)*(1-dy)*image(neighy(2),neighx(2)) + dx*(1-dy)*image(neighy(2),neighx(1));
imagenew(y,x) = vnew;
else;
vnew = image(xnew,ynew);
imagenew(y,x) = vnew;
end;
end;
end;

end;

Figure 8. Restored image with no distortions at the edges anymore.

Figure 8 shows the corrected image, in which there are no more distortions. It can be observed that the boundaries/edges of the rectangles are somewhat blurry or not defined. But if you zoom the original distorted image in Figure 3, you can see that the edges are not defined also. So, it just makes sense why the corrected image looks like that.

I would myself a grade of 10 for this activity since I was able to correct the distorted image.