In previous activities, the segmentation of the ROI from the background was done by binarization of the grayscale image. However, if the the grayscale image is similar to that shown below, it would be hard choose a threshold value which can properly separate the ROI from the background. Thus, new techniques of segmentation are introduced in this activity: parametric and nonparametric. There is no need to convert the truecolor image into a grayscale image. The histogram of the RGB values will be used to separate the ROI from the background.
Before proceeding with the segmentation process, the variation in the brightness level of the 3D objects were dealt with first. Brightness information can be disregarded if the RGB values are normalized to the intensity values at each point. That is, the RGB values at a point are divided by the sum of the of the RGB values at that point. This color space, which contains only chromaticity/color information, is called the normalized chromaticity coordinates or NCC. Figure 1 shows the NCC where the x-axis is r and the y-axis is g. b need not be shown here because its value can already be derived from r and g.
Before proceeding with the segmentation process, the variation in the brightness level of the 3D objects were dealt with first. Brightness information can be disregarded if the RGB values are normalized to the intensity values at each point. That is, the RGB values at a point are divided by the sum of the of the RGB values at that point. This color space, which contains only chromaticity/color information, is called the normalized chromaticity coordinates or NCC. Figure 1 shows the NCC where the x-axis is r and the y-axis is g. b need not be shown here because its value can already be derived from r and g.
Parametric segmentation is done by assuming that the probability distribution function (PDF) of the image is a Gaussian distribution independent along r and g. The Gaussian PDF equations, form shown in the manual, for r and g are established by getting the mean, standard deviation and variance of the rg values of the ROI, which must be a monochromatic patch. The probability values along r and g are then multiplied and the product is already the correlation between the color information of the points in the image and the ROI. The process of segmentation is done by determining the probability that the rg values at points in the image belong to the color distribution of the ROI. This means the portion in the image that has the same color as the ROI, or high probability, will appear as bright spots in the resulting segmented image.
In nonparametric segmentation, no form of PDF is assumed and the 2D histogram of the binned rg values is used. The histogram is a N x N matrix where N is the number of bins. A sample code of creating a 2D histogram is shown in the manual. The process of segmentation is done by histogram backprojection. That is the pixel locations are assigned with new values depending on the r and g values. The new value is the value in the 2D histogram of the (r*N, g*N) location. Bright spots correspond to the portions of the image with the same color information as the ROI.
In nonparametric segmentation, no form of PDF is assumed and the 2D histogram of the binned rg values is used. The histogram is a N x N matrix where N is the number of bins. A sample code of creating a 2D histogram is shown in the manual. The process of segmentation is done by histogram backprojection. That is the pixel locations are assigned with new values depending on the r and g values. The new value is the value in the 2D histogram of the (r*N, g*N) location. Bright spots correspond to the portions of the image with the same color information as the ROI.
Figure 1. Resulting images after nonparametric and parametric segmentation of the patches of the sample images (third and last column). The patches are the small images just above the sample images. The second column consists of the 2D histogram of the patches.
Figure 1 shows several examples of nonparametric and parametric segmentation (third and last columns, respectively). The first four images illustrate how the segmentation process can also be used to highlight only the portions in a monochromatic object that have the same shade as the patch. The variation in the color of a 3D object is also evident in these results. The fifth image is a clear proof that the techniques used can properly segment the ROI from the background without having to convert the image into grayscale. The succeeding images segment the fruits in the image having the same color as the patch. The colors may not be exactly the same. As long as the fruit has r or/and g values, which are more or less the same as that in the patch, that fruit will be highlighted in the segmented image. The correlation in the in the r and g values of the image and the patch is illustrated in the variation of brightness of the highlighted spots. The nearer the color of the fruit to the color of the patch, the brighter the highlighted spots will be.
The created 2D histograms were checked by comparing the bright portions in the histogram with the NCC plot above. It can be seen that the bright portions are located at the same position as the location in the NCC plot of the color of the patch.
Comparing the results of parametric and nonparametric segmentation, it can be seen that the former is a better technique of highlighting portions that have color more or less the same as the patch. The trace of the fruits in the segmented image is more solid when the former technique is used as compared to when the latter technique is used. This is probably because the Gaussian distribution generates higher probability than when using the 2D histogram of the patch. However, it must be noted that the number of bins used for the parametric segmentation above is 256. This means the colors/shades of colors are highly separated, and so fewer portions in the image will be detected having the same color as the patch. This explains why in the results above the parametric method has darker and fewer highlighted spots than the parametric method. Figure 2 shows the different results when different number of bins is used. It can be seen that more fruits are highlighted when the bin is just 10. The trace is even more solid than the trace using parametric method. This is because with smaller bins, more colors of the same shade are grouped. From the results below, it can be seen that using 100 bins gives us the best result for highlighting only the fruits that have exactly the same color as the patch.
The created 2D histograms were checked by comparing the bright portions in the histogram with the NCC plot above. It can be seen that the bright portions are located at the same position as the location in the NCC plot of the color of the patch.
Comparing the results of parametric and nonparametric segmentation, it can be seen that the former is a better technique of highlighting portions that have color more or less the same as the patch. The trace of the fruits in the segmented image is more solid when the former technique is used as compared to when the latter technique is used. This is probably because the Gaussian distribution generates higher probability than when using the 2D histogram of the patch. However, it must be noted that the number of bins used for the parametric segmentation above is 256. This means the colors/shades of colors are highly separated, and so fewer portions in the image will be detected having the same color as the patch. This explains why in the results above the parametric method has darker and fewer highlighted spots than the parametric method. Figure 2 shows the different results when different number of bins is used. It can be seen that more fruits are highlighted when the bin is just 10. The trace is even more solid than the trace using parametric method. This is because with smaller bins, more colors of the same shade are grouped. From the results below, it can be seen that using 100 bins gives us the best result for highlighting only the fruits that have exactly the same color as the patch.
Figure 2. Parametric segmentation of the banana and grape patches using different bins (10, 100, 256).I would like to thank Thirdy, Master and all other who have helped me understand what has to be done for this activity. I would give myself a grade of 10 for a job well done (according to me).
References:
2. Mango: http://carinderia.net/blog/wp-content/uploads/2008/12/mango13.jpg
3. Green Apple: http://2.bp.blogspot.com/_wxeBei5m--0/SeaZnk8DBfI/AAAAAAAAARE/uci2eKrBTtU/s400/apple_green_fruit_240421_l.jpg
4. Red Apple: http://www.ableweb.org/news/winter2009/images/fruitApple1c4.jpg
5. Fruits: http://files.myopera.com/buksiy/albums/739313/Fruits.jpg
6. Apple Tree: http://www.kevinecotter.com/appletree.jpg
For going the extra mile in exploring the effect of bin size, you deserve an 11.
ReplyDelete