search

FluxML / fluxml.github.io

Flux Website
https://fluxml.ai
MIT License
20 stars 45 forks source link

[Discussion]: Update and periodically test posts and model-zoo tutorials #141

Open Saransh-cpp opened 1 year ago

Saransh-cpp commented 1 year ago

This is a placeholder issue for updating the existing tutorials and adding workflows to periodically test them.

My proposal's text -

Flux's website has plenty of good tutorials, but some of them have been outdated for some time. These tutorials go stale with every new Julia and Flux release making them unreliable for newcomers. Additionally, as discussed with Dhairya Gandhi, the FluxBot can be used to automatically test these examples, but it is currently not integrated with the FluxML ecosystem. The bot would ideally run all the model-zoo examples on every PR created on Flux.jl. This section would aim to -

  • Update the tutorials on Flux’s website.
  • Add tests to (or run) these examples to ensure they do not go stale.
  • Integrate FluxBot with the existing FluxML ecosystem and get it running.

I tried triggering the FluxBot (https://github.com/FluxML/Flux.jl/pull/2016#issuecomment-1207408307), but it did not respond. I am guessing that the bot is not deployed at the moment. I did find a buildkite-related file and a PR (https://github.com/DhairyaLGandhi/FluxBot.jl/pull/2) that probably aims to shift it to GitLab's CI, but I am not sure how and where it is deployed currently. I am also not sure if there is a way to run it locally for some personal test repositories.

For the tutorials, should I migrate them from Flux.params to Optimisers.jl, or would it be too soon? I will audit all the tutorials and update them as needed!

For testing, I think it would be straightforward to periodically test the model-zoo examples using Julia's doctests, as model-zoo is a Julia package. For the website, I think the -

doctest(source, modules; kwargs...)

definition of doctest should work.

Also, #136 aims to migrate this website to Franklin.jl. Is this still in progress or has this been abandoned? I can take this up if required (cc: @logankilpatrick @darsnack)!

I will start by adding these periodic tests to both the repositories!

cc: @DhairyaLGandhi @mcabbott @ToucheSir

ToucheSir commented 1 year ago

Will let those more qualified to speak on the site stuff to fill in there, but I'm interested about your thoughts on how we might automate testing of the model-zoo! One past/current blocker is that it's not clear how/what to test for given the limited compute resources available on CI. I was hoping other DL libraries had tried this with their model zoos, but last I checked none of them did any kind of structured correctness testing (let alone automated).

Saransh-cpp commented 1 year ago

Ah, I had not considered the limited resources available on CI. This does make it tricky. I wonder, how was FluxBot aiming to achieve this?

ToucheSir commented 1 year ago

No idea, I'm afraid.

darsnack commented 1 year ago

I started revamping the site, but I didn't finish before getting pulled away. I'll push what I have as a draft today.

I don't think the model zoo or any code belongs on the website. Anything that is instructional should ideally go into the docs as tutorials. Blog posts are fine, but those should be more news-oriented than instructional. We can still test the tutorials in the docs. It seems like this discussion will fit in well with the planned discussion on documentation systems during this weeks ML community call.