AI-SDC / ACRO

Tools for the Automatic Checking of Research Outputs. These are the tools for researchers to use as drop-in replacements for commands that produce outputs in Stata Python and R
MIT License
15 stars 2 forks source link

Adding SDC guidance for researchers and checkers #71

Open rpreen opened 1 year ago

rpreen commented 1 year ago

Assuming the availability of SDC guidance such as in this handbook or as is being produced in WP1 (e.g., statistic type, description, requirements, disclosure issues, mitigation, possibly TRE specific rules, etc.) this information needs to be made available to both researchers and TRE checkers. So the question is how best to make this available for ease of access, maintenance, and including ACRO examples.

Should this information be made available on an independent website or included directly within ACRO documentation? ACRO documentation could simply link to the relevant part of an external website? How coupled should the guidance be with ACRO implementation?

Should the checker GUI be able to directly search/display guidance or just provide links?

Currently we attempt to include the whole ACRO command within the JSON output, but we should also include a simplified form of either (a) the function name (e.g., "function": "ols") or (b) a type (e.g., "type": "regression") so that the GUI can then use this to decide what SDC guidance to reference for more information?

rpreen commented 1 year ago

As of https://github.com/AI-SDC/ACRO/pull/89 JSON output now includes a field named "type" which is:

Each also contains a "properties" field, which is a dictionary that contains a "method" field (as well as output-specific entries) for each. The "method" may be:

jim-smith commented 1 year ago

@rpreen just occurred t me that for researchers we could have a function describe_risk(family:str) or describe_risk(output:str) that they could call to get more nicely formatted and detailed feedback and help than just splurging it out via INFO logging. (although we should still do that as well)

jim-smith commented 11 months ago

Think that we have everything we nee to do this in place technically. However, am placing on hold for now while we get the excel spreadsheet of risks from WP1 into a more structured document.