search

nateraw / modelcards

📝 Utility to create, edit, and publish model cards on the Hugging Face Hub. [**Now lives in huggingface_hub**]
MIT License
15 stars 4 forks source link

from_template silently drops non-matching kwargs #60

Open BenjaminBossan opened 2 years ago

BenjaminBossan commented 2 years ago

As a user, it is surprising to me that I can add any argument to ModelCard.from_template only to have them silently being ignored if they don't match the template.

To reproduce, use the same code as in the README and add an arbitrary argument, e.g.

card = ModelCard.from_template(card_data=..., model_id=..., model_description=..., foo='123')
card.save(...)

The foo='123' part is silently dropped.

ping @adrinjalali @merveenoyan

osanseviero commented 2 years ago

What would be the expected behaviour for you?

adrinjalali commented 2 years ago

I'd expect an exception with a message containing the keys which are not present in the template.

BenjaminBossan commented 2 years ago

Yes, it's easy too easy to miss currently. And even if you know that only supported keys should be used, you could still make a typo and there would be no indication of anything being wrong. At the very least, there should be a warning.

That being said, I looked through the jinja2 docs and code and there seems to be no support from jinja2 in validating the keys.

nateraw commented 2 years ago

I believe there are ways to locate the existing keys in a template then we can compare those against the provided template kwargs. Definitely something to look into

BenjaminBossan commented 2 years ago

I believe there are ways to locate the existing keys in a template then we can compare those against the provided template kwargs. Definitely something to look into

That would be fantastic, let me know if you need help. Ideally, the validation could be provided in a separate method, so that 3rd party libraries can make use of it.