As healthcare increasingly relies on data-driven insights, predictive analytics models are becoming critical tools. Predictive algorithms and models are trained on diverse patient data, such as patient-specific demographics, medical history, and disease details, to make diagnostic and prognostic predictions related to a particular disease or specific outcome.
While AI can perform various clinical tasks such as robotic surgery, medical image analysis, etc., arguably with greater efficiency than humans, recent studies have raised concerns about biases within these algorithms due to biased training data, potentially leading to discrimination and harm. If not addressed, biases within AI-based predictions can systematize and perpetuate discrimination faced by underrepresented groups by embedding disparities within decision-making algorithms.
Identifying sources of bias in AI and machine learning models and defining strategies to mitigate potential disparities are critical to ensuring equitable and fair predictions from medical AI applications.
Sources of Bias in Medical AI Algorithms
As mentioned above, medical AI algorithms reflect and amplify existing disparities in the training data. If not detected or addressed, bias in conceptualizing, developing, and applying AI-powered medical models can perpetuate existing health inequities and lead to poorer health outcomes for underrepresented, underserved, and under-resourced groups.
Identification and mitigation of AI bias starts with understanding the possible sources of bias. Here are the common sources of bias in medical AI.
Human Bias in AI Design
AI models are designed by humans and, hence, inevitably inherit societal biases. These biases are often embedded throughout the development process. The selection of problems to address, methods to solve them, and data to collect reflects human priorities and perceptions, which may not always align with real needs or priorities. This selection influences the areas that get focus, funding, and development resources.
Data Generalizability Issues
The efficiency of an AI model depends on the data it is trained on. Access to large and diverse medical datasets is crucial for developing smarter AI models. Since AI systems don’t have inherent knowledge or understanding, their efficiency is limited by the training data. As a result, AI can have blind spots in its analysis and decision-making capabilities when faced with new data or scenarios. Several historically disadvantaged and underserved people populations — ranging from race, ethnicity, gender, and age to socioeconomic status — may be underrepresented in the datasets used to train healthcare AI systems. This underrepresentation can affect the model’s ability to generalize to these underrepresented groups.
Inadequate representation of certain racial, ethnic, or socioeconomic groups in patient health records can perpetuate health disparities and lead to biased decision-making, as the AI models trained on such data may not identify specific health problems prevalent in these communities.
Algorithmic Bias
Human-driven biases and data generalizability problems translate into algorithmic bias. Detecting whether the bias originates from algorithms, training data, or a combination of both is challenging because they align with and reinforce existing social biases. For example, deep-seated institutional biases emanating from ethnicity, race, and socioeconomic status already impact health outcomes. Therefore, if an AI algorithm generates inaccurate or unfair results, it is difficult to pinpoint if the bias results from algorithmic inefficiency, biased data, or both. Despite their significant potential benefits, algorithms present the greatest potential risks because of the black box self-learning model, where the decision-making process is not transparent and easily interpretable.
Strategies to Mitigate Bias in AI-based Models
Here are strategies to mitigate bias in medical AI models:
Data Diversification and Representation
While developing AI-enabled medical systems, it is important to ensure the quality and representation of the datasets. Developers should combine various datasets consisting of key variables such as race, ethnicity, and social determinants of health and include them in prediction algorithms to minimize bias to enhance generalizability and representation of AI models.
The training data must be diverse and representative of all patient demographic groups and conditions to enable the model to make accurate predictions for a diverse group of patients. Model developers must engage with medical experts and stakeholders representing different patient populations to set data requirements and ensure the dataset doesn’t over or underrepresent any interest group.
Using Reweighting and Resampling to Create a Balanced Sample
Reweighting and resampling techniques are crucial strategies in machine learning for correcting class imbalance in datasets. In reweighting, underrepresented groups receive higher weight to ensure proportionate representation in model training, counteracting the dominance of majority groups. Resampling involves either oversampling minority classes by duplicating existing samples or using techniques like SMOTE (Synthetic Minority Over-sampling Technique) to create synthetic data or undersampling majority class instances by removing some samples to balance the dataset. These strategies ensure that all classes are adequately represented in AI algorithms, leading to more accurate, fair, and unbiased models.
Medical Data Annotation to Minimize Training Data Bias
In supervised learning, human data labeling of the specific features or classes relevant to tasks is often required to guide the model in understanding the input data and predicting outcomes. Human labeling provides nuanced and contextual insights that automated processes can’t fully replicate. However, it is essential to collaborate with a high-quality data solution company that uses a diverse set of expert opinions in the data annotation process. By incorporating diverse perspectives and employing algorithmic methods to synthesize multiple opinions, annotation platforms mitigate individual bias, ensuring the final dataset reflects a fair and accurate representation.
Regular Audit and Review of Data and Model Performance
Regular evaluation and updating of training data and machine learning models are crucial for preventing bias development over time. This is particularly important in dynamic fields like healthcare, where patient demographics and data sources evolve rapidly, rendering previously unbiased models biased if the training data doesn’t accurately represent the current patient population. Regular auditing and monitoring, compliant with HIPAA regulations, help identify and remove emerging biases while protecting senstive information, ensuring that models continue to function effectively and fairly.
Final Words
With the growing evidence of bias in data-driven prediction models, identifying and mitigating the sources of bias is critical to ensuring equitable healthcare outcomes, driving stakeholder buy-in, and meeting regulatory standards. By employing fairness toolkits such as AI Fairness 360 Toolkit (IBM) and Fairlearn (Microsoft), adopting bias mitigation strategies, and collaborating with experts and stakeholders, it is possible to enhance the representation of all patient classes, improve performance and reliability of medical AI models, and achieve their full potential for delivering superior healthcare for all.
Reviewed by
