Package ensemble annotator code

rogthefrog commented 2 months ago

The list of Ryan's annotators is in the wg meeting notes:

https://docs.google.com/document/u/2/d/1q1UPz9I_Hzft6-sS24P2TVKq6Ov7ghp-LRyVLS9rfFo/edit

Background:

Before this batch, engineering wrote annotators as plugins in modelgauge or in the core code.

Now they're building them for us to integrate.

Their annotators need to go from modeltune to modelgauge. The approach is to build a package artifact containing the annotator files, and install them in MG via poetry from a private repo on GH.

Potential snag: they have been using modelgauge as a submodule inside modeltune, pinned to an older commit. So the steps are:

bring modelgauge up to date inside modeltune and do basic tests
package up the annotators for poetry to install into modelgauge
test poetry install and then run the modelgauge command that lists annotators (poetry run modelgauge list), and see if they show up
let Barbara know so she can run tests w/ the new annotators.

rogthefrog commented 2 months ago

Part 1 https://github.com/mlcommons/modeltune/pull/122

rogthefrog commented 2 months ago

Part 2 https://github.com/mlcommons/modeltune/pull/123

rogthefrog commented 2 months ago

Note: modeltune is currently broken for local development after moving the modelgauge dependency to the dev group. I will fix that.

rogthefrog commented 2 months ago

Part 3 https://github.com/mlcommons/modeltune/pull/124

mlcommons / modelgauge

Package ensemble annotator code #531