Principles of responsible AI: Leaders’ FAQs answered
Principles of responsible AI: Leaders’ FAQs answered
Responsible AI refers to the design, development, deployment, and use of AI systems in a manner that is ethical, safe, transparent, regulatory-compliant, and beneficial to society at large.
It acknowledges that AI has the power to bring about significant improvements across numerous sectors and industries, as well as wider society, but that it also gives rise to significant risks and potential negative outcomes.
Responsible AI aims to embed ethical principles into AI systems and workflows to mitigate these risks and negative outcomes, while maximizing the benefits of AI.
Businesses and other organizations have published various principles-based frameworks for responsible AI, from tech giants such as Microsoft and Google to international bodies such as the OECD and the World Economic Forum.
While individual frameworks differ, some common themes or requirements for responsible AI can be identified.
These include:
Reliability and safety
Fairness and inclusivity
Accountability
Transparency
Security and resilience, and
Sustainability
Ensuring that AI systems adhere to these principles is essential for upholding ethical standards, protecting individuals and society, and promoting long-term acceptance and beneficial use of AI technologies.
Why are reliability and safety essential in AI systems?
Reliability and safety are cornerstones of RAI as they ensure that AI systems can be trusted to perform their intended tasks consistently and without causing unintentional harm.
Reliability means that users can be confident that an AI system is producing dependable and consistent outputs. Consistent, predictable AI behavior reassures users and other stakeholders, such as policymakers and the general public, that the technology is both dependable and trustworthy.
Safety is equally important. AI systems should be developed with safety in mind to prevent foreseeable harm, whether to individuals, organizations, or broader society. Properly engineered safeguards can help prevent accidents, malicious use, and harmful outcomes, while serving to mitigate risks if/when failures occur.
While reliability and safety are distinct, they are also interconnected factors in terms of RAI. A reliable system is less likely to behave unpredictably or fail at critical moments, which are factors that can directly impact user safety.
Organizations can adopt several practices throughout an AI system’s lifecycle to help ensure reliability and safety.
When developing AI tools, it is vital to define clear success criteria and performance metrics, both from a business and technical perspective. These objectives should guide issues such as design choices, data management strategies, model development and testing, and performance monitoring.
For example, a medical company developing an AI diagnostic tool might define success metrics that include a minimum accuracy target for diagnosing a disease or illness, as well as acceptable false positive and false negative rates. All model development and testing efforts would then aim to meet or surpass these targets.
Similarly, banks typically set out key performance indicators (KPIs) for AI fraud detection systems.
For instance, a bank might specify (say) a 1% false-positive rate for fraud alerts, ensuring a balance between effective prevention of fraud and customers being unnecessarily inconvenienced by false alerts.
Potential AI failures or undesirable behaviors that might arise can be identified by conducting thorough risk assessments.
These consider both the likelihood and potential impact of various risks, and prioritize resources to address them proactively.
Consider, for instance, an autonomous vehicle (self-driving car). Auto manufacturers perform a “Hazard Analysis and Risk Assessment (HARA)” to test the functional safety of vehicles and identify safety-critical events that could arise from autonomous functions, for example, malfunctioning sensors in bad weather. This assessment helps prioritize measures to address any issues and reduce the associated risk to acceptable levels, such as (in this example) putting in place measures to handle sensor failures or unexpected weather conditions.
As another example, a bank using an AI-driven credit risk model might conduct a thorough risk assessment of its model to identify potential failure modes. For instance, the bank might evaluate how the model reacts to changing economic or political conditions, such as a sudden recession or trade war, and incorporate a contingency plan for each identified risk, such as real-time or more frequent monitoring of macroeconomic and political risk indicators.
AI systems need to be tested under a variety of conditions, including edge cases and stress scenarios, to ensure that they perform as expected. This might include employing techniques such as:
Adversarial testing (deliberately probing a model with inputs designed to expose weaknesses or potential vulnerabilities)
Model-in-the-loop testing (integrating the trained model itself into the test or development workflow rather than testing it in isolation)
Human-in-the-loop (HITL) testing (integrating human input and expertise into the training of AI systems)
For example, before a fraud detection system goes live, a bank might test it with a range of adversarial scenarios using synthetic fraudulent transactions and large-scale coordinated fraud patterns. The bank might also stress-test the system by mimicking periods of high transaction volumes, such as holiday seasons, to ensure the AI can maintain speed and reliability in times of heavy load.
Data used for model training and validation should be accurate, relevant, and representative to reduce the likelihood of biased, discriminatory, or unpredictable outputs. Such data needs to be continuously monitored and updated to keep the model’s knowledge current as real-world conditions change.
Consider, for instance, a customer service chatbot.
To minimize mislabeling and eliminate duplication, businesses using such chatbots need strict processes for cleaning and labeling customer conversation data (from phone calls, e-mails, messaging services, historical chatbot sessions, and so on). They also need to conduct periodic data quality checks to remove outdated or irrelevant data.
Post-deployment, model outputs and user feedback should be tracked and monitored to detect errors, performance drifts, or safety concerns. Processes should be in place to roll back or update models if harmful or unsafe behaviors are detected.
Take the example of a social media content filter system. After launching a new filter for abusive or harmful content, the platform collects user feedback and real-time performance metrics. When the filter flags large volumes of benign posts (false positives), it triggers an update or rollback to reduce the volume of such flags.
Ruairi is a content marketer for Intuition Publishing, focusing on the creation of written pieces around corporate learning and finance. If you'd like to speak to an eLearning solutions specialist about an upcoming project, Ruairi can put you in touch with the appropriate Intuition representative. Email him at rodonnellan@intuition.com.
