Changes to auto score criteria for instructor training applications

[maneesha]

From @ErinBecker

Currently the instructor training auto scoring system includes +1 point for the following domain expertise areas:

        scoring_domains = [
            "Humanities",
            "Library and information science",
            "Economics/business",
            "Social sciences",
            "Chemistry",
        ]

This should be expanded to also include Economics/business and Space sciences.

Additionally, we should have +1 point for if the applicant answers "other" and the free response contains (regexes) /astronomy/i or /eukref/i

[maneesha]

Revisiting my original notes with @ErinBecker Erin had also wanted to remove:

• Library Carpentry
• Social Sciences Data Carpentry lessons
• Geospatial Data Carpentry Lessons

Erin, just to be clear, you would want to change the list in my original comment above to allow +1 point for the following.

Specific lesson programs or lessons are not automatically awarded points. This will be reflected in the scoring rubric. Should we take off " "Library and information science" and ""Social sciences" since we are taking off the corresponding lessons/lesson program?

        scoring_domains = [
            "Humanities",
            "Economics/business",
            "Chemistry",
           "Space sciences"
        ]

[ErinBecker]

Yes, sorry I missed this. Please take off library and information science and social sciences. We are no longer actively prioritizing applicants from those areas.

[maneesha]

I can see times we'd want to change this without having to put through code changes. A better option (which will take more work to implement) may be to add in an interface that lets admin users select which domains are to be awarded extra points.

[maneesha]

We discussed this at the instructor training team meeting today. @karenword will be bringing this to @ErinBecker .

[karenword]

@ErinBecker and I discussed this and I think this issue can be closed or designated to wait for later. In the long run, it would be ideal to create an interface that allows us to select domains for points. In the short run, we have other more urgent priorities for developer time and do not want this task to compete with any of them.

For the record, however: the only category that should currently be receiving extra points is EukRef. We plan to a) update the rubric to reflect existing categories that have not yet been removed from autoscoring (like social sciences) and b) add points for EukRef manually for now.

[maneesha]

Removing "Getting started" label because this needs discussion

[maneesha]

Based on Karen's note from a year ago, I am closing this issue for now. We can consider opening a new issue that looks at how the auto-scoring criteria is managed in general, and what needs to remain in the codebase and what could be managed through a UI.

The auto-score is only one stage of the application review. Applications are always downloaded and then additional manual scoring is applied. A more feasible solution may be to apply any of these new/changing criteria to the downloaded data set instead of doing it in AMY.