So far the 6 sections follow roughly the data scientist activities: data, bias/modeling, evaluation, usage in production. However that is not that clear, as:
section 2 and 3 cover together modeling, training, bias, evaluation, without a clear split
section 4 covers responsibilities and reproducibility
section 6 is transversal
We propose here to improve the sections scope to offer a clearer structure. The update sections could be:
A first set of sections dedicated to the data scientists activities:
Data preparation and protection
Modelisation, training, evaluation
Deployment, monitoring/MLOps
A second set dedicated to organisation aspects, processes and externalities, more global to the org doing the assessment:
Organisation, governance, responsibilities - internal to the org
Organisation, governance, responsibilities - related to external parties
Negative externalities
These new sections would include the following evaluation elements:
Data preparation and protection: 7 elements
1.1 and 1.2
1.5 and 1.6
2.1
3.1 to 3.3
Modelisation, training, evaluation: 8 elements
1.7 and 1.8
2.2 to 2.4
3.4 and 3.6
Deployment, monitoring/MLOps: 5 elements
3.5
4.3
5.1 to 5.4
Organisation, governance, responsibilities - internal to the org: 6 elements
1.3 and 1.4
3.7
4.1 and 4.2
4.4 (specific to internal parties)
Organisation, governance, responsibilities - related to external parties: 5 elements
As indicated in the related PR #168, I am putting this on hold, as there isn't a real ask from the community, nor a clear benefit from the new organisation targeted.
So far the 6 sections follow roughly the data scientist activities: data, bias/modeling, evaluation, usage in production. However that is not that clear, as:
We propose here to improve the sections scope to offer a clearer structure. The update sections could be:
These new sections would include the following evaluation elements:
Data preparation and protection: 7 elements
Modelisation, training, evaluation: 8 elements
Deployment, monitoring/MLOps: 5 elements
Organisation, governance, responsibilities - internal to the org: 6 elements
Organisation, governance, responsibilities - related to external parties: 5 elements
Negative externalities: 3 elements