search

materialsproject / matbench

Matbench: Benchmarks for materials science property prediction
https://matbench.materialsproject.org
MIT License
119 stars 48 forks source link

Doesn't seem like per-task plots are being regenerated correctly? #259

Open ardunn opened 1 year ago

ardunn commented 1 year ago

https://matbench.materialsproject.org/Leaderboards%20Per-Task/matbench_v0.1_matbench_mp_is_metal/

For example, matformer shows up on the table but not in the figure.

ardunn commented 1 year ago

image but image

hrushikesh-s commented 1 year ago

When I clone the repo and try rebuilding the website on a local server by running scripts/rebuild_docs.py and having mkdocs serve running in another terminal, I'm able to get the correct figure (see below).

Is there a way to rebuild the official website without making any changes to the repo?

image
ardunn commented 1 year ago

When I clone the repo and try rebuilding the website on a local server by running scripts/rebuild_docs.py and having mkdocs serve running in another terminal, I'm able to get the correct figure (see below).

Is there a way to rebuild the official website without making any changes to the repo?

image

Well we can rebuild it locally (mkdocs build) and then push it, and then cancel the rebuild_docs GH action workflow.

But really I'm interested in why it's not regenerating automatically, otherwise we'll have to manually fix this stuff all the time

hrushikesh-s commented 1 year ago

I'm trying to setup the rebuild_docs GH action workflow in my forked repo and was looking to change the matbench's website to a temporary github hosted website.

I was expecting that this modification can be made by editing the site_url tag in mkdocs.yml, but there isn't any such tag there.

@ardunn , any idea how can I setup the GH action workflow in my forked repo without messing around with the actual website?

ardunn commented 1 year ago

I'm trying to setup the rebuild_docs GH action workflow in my forked repo and was looking to change the matbench's website to a temporary github hosted website.

I was expecting that this modification can be made by editing the site_url tag in mkdocs.yml, but there isn't any such tag there.

@ardunn , any idea how can I setup the GH action workflow in my forked repo without messing around with the actual website?

@hrushikesh-s sorry just saw this now. I think you can just enable GH actions on your own fork and have them run?

otherwise, I am pretty sure you can enable them to run for a specific branch on matbench as well. Having a staging branch might be a thing worth doing. Maybe try doing it on your fork and if that doesn't work we can look into enabling it for a different matbench branch?