[Proposal]: Evaluating AI in Teacher Performance Reviews: Benefits, Biases, and Best Practices

esilberberg commented 3 months ago

1. Lesson Topic

In this tutorial, you'll try your hand at building your own AI tools that analyze sentiment in student feedback with the Google Gemini LLM and Python. Through experience using the technology, you'll better understand the benefits, biases, and best practices of this approach to teacher performance evaluations.

2. Lesson Language

English

3. Draft materials

https://github.com/esilberberg/AI-Teacher-Reviews-Lesson

4. Requirements for existing materials

[X] do your lesson materials conform to our Code of Conduct?
[ ] are your materials already on GitHub and do they use The Carpentries lesson template or The Carpentries Workbench? (you can visit the Workbench tutorial to learn more about how to use our lesson infrastructure).

5. New repository creation

[ ] I need a new repository created

6. Transfer existing repository

[ ] my lesson meets both criteria in part 4 and I would like to transfer the repository to The Carpentries Incubator

7. Collaborators

No response

8. Info/Questions

The lesson content exists in a GitHub repo but it does not follow lesson template. I am currently in the UCLA Lessons for Open Science projec. I will eventually learn the template as part of the project, but I am happy to start sooner. This lesson is based on some scholarly communication work at my library, where I used Gemini to isolated journal names in faculty publication citations. The current lesson just looks at how to incorporate Gemini in Python functions. My scholarly communication work went a step further and then sent the isolated journal names to the Sherpa Romeo API to determine the Open Access policies of the journals. Librarians at other institutions have asked for a tutorial on how to do this. Do you feel that this extension would be more interesting?

tobyhodges commented 3 months ago

Thank you for submitting this lesson to The Carpentries Incubator, @esilberberg!

We can accept the lesson into the Incubator as soon as it is set up to use The Carpentries template. Would you like me to create a template lesson repository for you to transfer the existing content into? Or will you adjust the current repository to use the template? Please let me know your preference and I will be happy to help.

My scholarly communication work went a step further and then sent the isolated journal names to the Sherpa Romeo API to determine the Open Access policies of the journals. Librarians at other institutions have asked for a tutorial on how to do this. Do you feel that this extension would be more interesting?

I am unfamiliar with the tools and processes you mention, so this is a judgement best made by others. This would be a good question to post to the relevant channels on our Slack workspace (join via this link then look for the #libraries channel) or TopicBox mailing list.

esilberberg commented 3 months ago

Thats great thank you. If you create the template will it live in a Carpentry's repo? Can you share documentation on the markdown?

esilberberg commented 2 months ago

an accept the lesson into the Incubator as soon as it is set up to use The Carpentries template. Would you like me to create a template lesson repository for you to transfer the existing content into?

If you create the template will it live in a Carpentry's repo or as one of my own repos?

esilberberg commented 2 months ago

I will teach this lesson twice this semester. Here is the current repo: https://github.com/esilberberg/AI-Teacher-Reviews-Lesson

tobyhodges commented 2 months ago

@esilberberg thanks for your patience.

If you create the template will it live in a Carpentry's repo or as one of my own repos?

If I create it, it would exist as a repository here in the carpentries-incubator organisation, but you would have full administrative ownership of the project. If you want to create you own, under your namespace, you can do so by generating a new repository from https://github.com/carpentries/workbench-template-md.

esilberberg commented 2 months ago

Ok, could you please create it in the carpentries-incubator organisation?

tobyhodges commented 2 months ago

@esilberberg I have created the repository for you at https://github.com/carpentries-incubator/ai-teacher-feedback-lesson.

You should have received an invitation to join the maintainer team for this lesson. Please accept that invitation so that I can give you the necessary access to invite additional collaborators in the future.

The README of the repository includes a brief list of recommended next steps. You might find the Workbench documentation a helpful resource while working with The Carpentries lesson infrastructure. The Carpentries Curriculum Development Handbook and our Collaborative Lesson Development Training curriculum provide an overview of the approach we recommend to lesson design and development. But please feel free to post back here or ping me if you get stuck or if you have any further questions. I am always happy to support lesson development!

To keep in touch with The Carpentries curriculum community you may also find it useful to join the incubator-developers list on The Carpentries TopicBox and/or the lesson-dev channel on The Carpentries Slack.

esilberberg commented 2 months ago

Thank you!

carpentries-incubator / proposals