I approached ORI because we’re making changes in the use of digital technology, which will affect many of our employees. We want to do this well by continuing to value the in-depth knowledge of our employees while we supplement it with data and advanced analytics. ORI helped us by systematically assessing the technical, organizational and value impacts associated with the introduction of new technology. One element of the service that I found most valuable was the effort made to restate the findings in the business language of our company. This makes the findings actionable and inspires employees and change leaders alike.
Abraham van Poortvliet, former VP of Data Analytics and Decision Science, Technical Safety BC
The Client: Technical Safety BC
Technical Safety BC oversees the safety of residents in the province of British Columbia (BC, Canada). Its mandate is to ensure that a broad set of technical systems (e.g. boilers, electrical systems, elevators, etc.) found in BC’S homes, schools, and workplaces are properly installed, inspected, and maintained for safe operation. Technical Safety BC continuously collects data as part of its safety inspection activities, and has decided to improve the efficiency and effectiveness of their operations with data science and machine learning.
The Context: Machine Learning incorporated into a decision-support system
Technical Safety BC has an existing decision-support system that its frontline workers—safety officers—have been using to help prioritize their daily safety inspection tasks. To address shortcomings in the existing system, and to strategically assist in the allocation of Technical Safety BC’S human resources, a new decision-support system is being designed.
This new program predicts the likelihood of a technical hazard, and thus predicts where safety officers are likely to be needed most. The new machine learning system is designed to learn from Technical Safety BC’s constantly evolving data set to improve its predictions over time.
The Challenge
The senior management and development teams at Technical Safety BC recognized that their decision to use predictive algorithms in their operations raises new ethical risks inherent to machine learning systems. Technical Safety BC wanted to ensure these risks were addressed during the development stage of the project, so that they could plan for a successful system rollout, protect the interest of their employees and ultimately increase public safety.
Identifying and characterizing the ethical risks, and the strategies to manage them, requires a comprehensive analysis to be performed by a specialized team of experts. However, Technical Safety BC (like most organizations today) does not have an internal team of experts specializing in predictive algorithm ethics with which to accomplish such an analysis.
I believe that good corporate governance requires those of us using AI in service to our clients to do an ethics review: not just because future regulatory environments will require it, but because it encompasses every aspect of the use of the tools – from deployment for employees, to impact on clients and stakeholders, to enterprise risk management. ORI (previously Generation R) did an outstanding assessment for us which advanced our thinking around AI by years — an outstanding return on investment.
Catherine Roome, former CEO of Technical Safety BC
The Solution
Technical Safety BC hired ORI (previously Generation R) to perform a comprehensive ethics assessment to identify, characterize, and analyze the various ethical challenges/risks uniquely posed by the new decision-support system. Technical Safety BC also asked for concrete, practical and actionable recommendations tailored to their organization, to help mitigate the challenges/risks in the development and rollout of the system.
To address Technical Safety BC’s needs, we applied the process outlined in the FAIE toolkit. Taking into account key stakeholder input, the organizational dynamics, and the technical elements of the decision-support system, we identified the primary ethical risks (i.e. barriers to success) that Technical Safety BC would likely encounter in the development and rollout of the new decision-support system.
With the ethical risks identified, we then delivered a prioritized list of recommendations in the form of practical, concrete and actionable design tasks, policy considerations and business options. This provided Technical Safety BC’s senior management and development teams with clear actionable tasks that would help to mitigate foreseeable ethical challenges/risks inherent in the decision-support system.
The Outcome
ORI has enabled the executive team of Technical Safety BC by:
- Identifying the ethical risks directly relevant to their technology, their employees, and the business more broadly
- Providing 31 points of consideration that guide the executive team and the implicated leaders in making critical business, policy, and design decisions
- Presenting five key prioritized recommendations that Technical Safety BC teams can implement immediately