Deep learning from noisy multi-label food images

Abstract

The performance of deep learning methods depends not only on the model’s design but also on the data’s quantity, variety, and quality. Collecting abundant data from public repositories is feasible, but their review and annotation are laborious. As an alternative, unsupervised databases have been developed, where the automatic assignment of labels may generate noise due to possible deviations in the collected data. This paper proposes a robust deep-learning model for noisy labels to classify food images at the ingredient level by extending the single-label AFM method. The proposed ML-AFM uses Attentive Grouping and MixUp to mitigate label noise and capture complex feature-label relationships in the training data. Additionally, the activation and loss function is adapted to be suitable for multi-label classification problems. The experimental evaluation is performed on the public Food-101N dataset, with extended annotations at the ingredient level. The results show that ML-AFM performs better than the reference model, achieving an F1 of 86.99%, an AUPRC of 92.85%, and a Jaccard index of 77.19%. The improved performance demonstrates the proposed model robustness to the given problem, which supports its usefulness in practical food recognition applications.