Fair out-of-Distribution detection for addressing skin tone representation in dermatology.

Abstract

Addressing representation issues in dermatological settings is crucial due to variations in how skin conditions manifest across skin tones, thereby providing competitive quality of care across different segments of the population. Although bias and fairness assessment in skin lesion classification has been a nactive research area, there issubstantially less exploration of the implications of skin tone representations and Out-of-Distribution (OOD) detectors’ performance. Current OOD methods detect samples from different hardware devices, clinical settings, or unknown disease samples. However, the absence of robustness analysis across skin tones questions whether these methods are fair detectors. As most skin datasets are reported to suffer from bias in skin tone distribution, this could lead to higher false positive rates in a particular skin tone. This research presents a framework to evaluate OOD detectors across different skin tones and scenarios. We review and compare state-of-the-art OOD detectors across two categories of skin tones, FST I-IV (lighter tones) and FST V-VI (brown and darker tones), over samples collected from dermatoscopic and clinical protocols. We conducted a Gray-Level Co-Occurrence Matrix (GLCM) texture analysis on ”Fitzpatrick17k dataset” samples from two main skin tone categories FST I-IV and FST V-VI, and compared statistical parameters across skin tone categories and nine skin conditions. This analysis indicates that FST V-VI textures are more heterogeneous and varied, while FST I-IV textures are more uniform and consistent. Our OOD detection experiments yield that in poorly performing OOD models, the representation gap measured between skin types is wider (from ? 10% to 30%) up for samples from darker skin tones. Compared to better performing models, skin type performance only differs for? 2%. Furthermore, this work shows that understanding OOD methods’ performance beyond average metrics is critical to developing more fair approaches. We used the AIF360 tool to assess fairness in our OOD detectors and evaluated their performance with group fairness metrics. Our observations show that models with similar overall performance can have significant differences in representation gaps, with group fairness metrics correlating negatively with the representation gap. This indicates that increasing the representation of FST V-VI leads to improved group fairness resulting in fairer OOD detectors.