Have model cards in `HFPusher`

[merveenoyan]

HFPusher component recently introduced in this blog post and is currently in this library has model cards feature (so it's in the project repository as of now, and not in tfx-addons' HFPusher) built for Hugging Face Hub. Reason why we had to go with HF model cards was that it follows a different structure, has metadata part as yaml on top to enable easy discovery and a free text markdown section below that. So it would be good to contribute that to the HFPusher in tfx-addons. Later on, we can introduce many other things if demanded, see this issue for discussion. Also the Hugging Face Hub client library has many functions for parsing and programmatically adding and editing model cards, so it's convenient for people too! WDYT?

Also pinging @sayakpaul and @deep-diver.

[sayakpaul]

@rcrowe-google this would definitely be a nice feature to have.

[rcrowe-google]

@hanneshapke The difference between the Google Model Cards and the Hugging Face Model Cards might be something to explore in the MCT team.

[hanneshapke]

Wow, what a timing @rcrowe-google. We just discussed this topic in the MCT meeting this morning as part of the framework agnostic MCT.

@sayakpaul @deep-diver @merveenoyan If you can, please join in on the effort! Any input and contribution from the HF side is extremely valuable!

[merveenoyan]

@hanneshapke can you give me pointers to get started? Or should I just open a PR to integrate the model cards?

[hanneshapke]

@merveenoyan Thank you for offering your help! We are currently collecting the different model/attributes the MCT should support. @deutranium is working on a comparison. If you can, join our next MCT call. We wanted to discuss a project roadmap and decide on the scope.

[merveenoyan]

@hanneshapke sure it would be nice! I previously worked on model cards for Keras models, see here (currently working on cards for different libraries) so for the model cards in HFPusher I thought of adding similar things (model history, hyperparameters, architecture and more) I'd love to meet 🙂

[hanneshapke]

Hi @merveenoyan, From your experience with the HuggingFace model cards, do you see anything missing in the original MCT proto (https://github.com/tensorflow/model-card-toolkit/blob/main/model_card_toolkit/proto/model_card.proto)?

We would love to add it in the future versions. The plan is to support the HF cards too.

[merveenoyan]

Hello @hanneshapke,

Reason why I particularly would pick HF over MCT would be choosing markdown over protobuf interface. We have a python client side library that allows us to programmatically create and edit model cards and also retrieve them when the model pushed to Hugging Face Hub. These model cards consist of a yaml metadata part and markdown part, where former is used for discoverability and filtering of models on HF Hub and latter is purely informative. You can check out an example one here, where the hyperparameters and model architecture plot are added automatically and rest is filled by the users. This is essentially what I'm working on to add to HfPusher component as well. We can also add model history too, as the markdown format provides a lot of flexibility.