Apple quality grading for harvestability monitoring using computer vision and deep learning algorithms

Abstract

The agricultural industry comprises an activity that has a marked influence on economic growth and people’s quality of life. Given the need to meet the food demand due to population growth, systems capable of optimizing crop yield are currently required. This work contributes with a practical tool that aids farmers in recognizing fruit quality, which is aimed at improving the apple quantification process and harvestability monitoring based on object identification employing computer vision techniques and deep learning algorithms. The development of the system presents i) detection of the apple classes for product counting and ii) quality classification for inspection and validation of the fruit by category. For grading of apple types, the SSD-MobileNet detection network model was used. A fast convolutional neural network FCN-ResNet 18 was used to segment quality instances at the pixel level. The proposed system was trained, validated, and tested on several experimental laboratory and field scenarios using two image databases generated in controlled and real agricultural environments. Results show that it is possible to detect and classify the quality status of apples during harvest, obtaining an accuracy ranging between 86.7% and 92.6% for detection and 94.7 ± 2.5% for segmentation, exceeding the results presented in related works in both cases.