Title: How do we write and promulgate domain-specific guidelines for analyses supporting risk management? Issues and Arguments
This webinar/online workshop will confront fundamental questions behind the development of domain specific risk management guidelines. As a backdrop, we define “domain-specific” as specific to a field of application such as terrorism risk management (RM), health RM, or financial RM. We also propose that the development of the guidelines be a joint effort of the SRA’s Applied Risk Management Specialty Group and the Foundational Issues in Risk Analysis Specialty Group, in collaboration with all the other SRA Specialty Groups. We look forward to a broad ranging discussion on what domain-specific guidelines could/should include in the following broad issue areas: data, technical assessment, uncertainty, process, engagement with the risk management decision process and stakeholders, guideline format and distribution, guideline application, additions/omissions. Our Final Big Question: What is the best way to get these guidelines written? To give us a concrete example to work on, we will distribute and consider draft guidelines for the domain of terrorism risk management. Participants are encouraged to come to the webinar “armed” with their own arguments and ideas. In fact, after intro material we will start by asking “So … does anybody have any arguments or better ideas?” Hope to see you there!
Issues with data
- Data sources?
- Data quality (including subject matter experts and their elicitations)?
- Implications of data shortfalls?
Issues with technical assessment
- Should a guideline list the specific analyses that are considered acceptable in the domain, or would that be too restrictive?
- Should a guideline list the specific shortfalls/challenges of the analyses of that field of application, then ask that each of those shortfalls/challenges be explicitly addressed?
- Should a guideline list standards for how each of those shortfalls/challenges should be addressed?
- Issues of completeness and imagination, including addressing scenarios “not on the list”?
- Issues of resilience?
Issues with uncertainty
- Clarity on what we don’t know and/or can’t know, and the implications of that?
- Handling of uncertainty?
- Communication of the uncertainty and its implications?
Issues with process
- Setting the scope and presenting the implications of that scope?
- Selecting the assumptions and presenting their implications?
- Third Party Review?
- Addressing scenarios “not on the list”?
Engagement with the Risk Management Decision Process and Stakeholders
- Transparency / Full Disclosure?
- Effective communication of results and caveats to the decision makers and stakeholders?
- Effective communication, back and forth, between analysts and decision makers, and between analysts and stakeholders, re values, concerns, uncertainties, assumptions, data shortfalls and analysis shortfalls, and the implications of all of those?
- Issues of political effectiveness? … of trust?
Guideline format and distribution
- Can the guidelines be summarized in a checklist?
- How should guidelines be promulgated? Besides posting them on the SRA website, should we circulate the guidelines to all journals in the domain and request:
1) that they post them on their “Guidance for Authors” websites, and
2) send them to all referees, asking them to use them in their reviews of submitted articles?
But that only covers journal articles.
- Any hope for “enforcing” guidelines? Or do we not want to even try that?
- But then, if we don’t want to “enforce,” how best to get the guidelines complied with, short of enforcement?
- Can we figure out a way to set up a “Culture of Analysis Quality” in each domain?
- What have we left out?
- Do you have better ideas?