A crosswalk of master controls aligning ISO42001, ISO27001, ISO27701, the NIST Risk Management Framework, EU AI Act, and SOC2, for a real-world AI governance control framework.
This is a tour de force, @James Kavanagh. I'm looking forward to your further work on this. Some questions:
1. Does one or more of these standards cover ethical treatment of data workers who do enrichment (labeling or annotation)? (Perhaps under Governance & Leadership?)
2. Do the 5 controls under Safe and Responsible AI cover proactive attention to identifying and mitigating biases?
3. Where do consent, credit, and compensation (3Cs) to creators and environmental resource efficiency fit in?
4. You mentioned "Not building general-purpose foundational models (e.g., this is not for OpenAI, Anthropic - they have some additional requirements under the EU AI Act that are not generally applicable)." Everything in the map (and more) still does apply to the foundational model companies, right?
5. Do you know of any person or organization who is, or will be, tracking which companies have certified their compliance with the standards you include here? (e.g. Anthropic getting ISO 42001 certification recently)
1: Yes, that would fall under GL-1 Leadership I think, maybe some others in RM too or Third-part supply chain. That said, I don't think any of the frameworks are very specific on that point. I haven't mapped these, but there is some guidance on this in World Economic Forum's "Guidelines for AI Procurement", and the Partnership on AI's "Responsible Sourcing of Data Enrichment Services.". If I get a chance, I'll try to map them out.
2. Yes, there's one master control all about Fairness & Bias.
3. So I debated (internally :) ) whether to include environmental as a separate control, or deal with it within Risk Management (Impact Assessments). I'm going to take another look at it
4. Correct
5. IAF CertSearch (https://www.iafcertsearch.org/) is getting better at this but there is a time lag and be aware that they only track accredited certifications (which is reasonable)
Great work @James Kavanagh. I've undertaken a similar exercise, compiling the recommended practices or controls from some of the frameworks you mentioned (plus some others) and grouping them under 7 domains: Strategy, Governance, Procurement, People, Compliance, Data and AI development. I found it helpful to tag each control by lifecycle phase and potential functional owner also. It was largely manual work, conducted by reading through the source documents and making some judgement calls much as you described....but worth it, to come up with some universal controls that respond to several standards.
Amazing work and thanks for sharing. I wonder if we extract entities and then use graph technology to create the relationships, this might become an interactive chart.
This is one of the most insightful and practical works I have read for navigating the multitude of frameworks, policies, and standards. I can only imagine the grueling and painstaking process you must have gone through to distill and simplify this into 12 domains.
I look forward to exploring your body of knowledge and perhaps considering its adoption where it makes sense for our organization.
Great article about your Master Control Set (MCS) & stuff!
You also may think of integrating it into / with ISO-31'000 to enable Corporate Integrated Risk Management capabilities depending on the context requirements.
Just amazing James … as usual with all your work - it is beyond comprehensive, logical & usable! I can't imagine the time that's gone into this - thanks for sharing, look forward to seeing where this heads.
This is a tour de force, @James Kavanagh. I'm looking forward to your further work on this. Some questions:
1. Does one or more of these standards cover ethical treatment of data workers who do enrichment (labeling or annotation)? (Perhaps under Governance & Leadership?)
2. Do the 5 controls under Safe and Responsible AI cover proactive attention to identifying and mitigating biases?
3. Where do consent, credit, and compensation (3Cs) to creators and environmental resource efficiency fit in?
4. You mentioned "Not building general-purpose foundational models (e.g., this is not for OpenAI, Anthropic - they have some additional requirements under the EU AI Act that are not generally applicable)." Everything in the map (and more) still does apply to the foundational model companies, right?
5. Do you know of any person or organization who is, or will be, tracking which companies have certified their compliance with the standards you include here? (e.g. Anthropic getting ISO 42001 certification recently)
Thanks!
Thanks Karen - great questions.
1: Yes, that would fall under GL-1 Leadership I think, maybe some others in RM too or Third-part supply chain. That said, I don't think any of the frameworks are very specific on that point. I haven't mapped these, but there is some guidance on this in World Economic Forum's "Guidelines for AI Procurement", and the Partnership on AI's "Responsible Sourcing of Data Enrichment Services.". If I get a chance, I'll try to map them out.
2. Yes, there's one master control all about Fairness & Bias.
3. So I debated (internally :) ) whether to include environmental as a separate control, or deal with it within Risk Management (Impact Assessments). I'm going to take another look at it
4. Correct
5. IAF CertSearch (https://www.iafcertsearch.org/) is getting better at this but there is a time lag and be aware that they only track accredited certifications (which is reasonable)
Thank you for the detailed reply, James. I’m looking forward to your further writing on the work you’re doing on this!
Great work @James Kavanagh. I've undertaken a similar exercise, compiling the recommended practices or controls from some of the frameworks you mentioned (plus some others) and grouping them under 7 domains: Strategy, Governance, Procurement, People, Compliance, Data and AI development. I found it helpful to tag each control by lifecycle phase and potential functional owner also. It was largely manual work, conducted by reading through the source documents and making some judgement calls much as you described....but worth it, to come up with some universal controls that respond to several standards.
Amazing work and thanks for sharing. I wonder if we extract entities and then use graph technology to create the relationships, this might become an interactive chart.
Coming soon :). I'll provide the data and code for an interactive tool to build these yourself from Excel
This is one of the most insightful and practical works I have read for navigating the multitude of frameworks, policies, and standards. I can only imagine the grueling and painstaking process you must have gone through to distill and simplify this into 12 domains.
I look forward to exploring your body of knowledge and perhaps considering its adoption where it makes sense for our organization.
Thank you for your effort.
Thank you for reading and providing that feedback. I really appreciate you taking the time to do so.
Wow this is amazing 🤩
Great article about your Master Control Set (MCS) & stuff!
You also may think of integrating it into / with ISO-31'000 to enable Corporate Integrated Risk Management capabilities depending on the context requirements.
Thanks @James Kavanagh. Curious to hear your thoughts on SCF and its metaframework. https://securecontrolsframework.com/scf-download/
Thank you so very much. How helpful!!!
Just amazing James … as usual with all your work - it is beyond comprehensive, logical & usable! I can't imagine the time that's gone into this - thanks for sharing, look forward to seeing where this heads.
Thanks James, looking forward to following along