radis / radis-lab

Online environment for advanced spectrum processing and comparison with experimental data
https://radis.github.io/radis-lab
5 stars 7 forks source link

Improve launch time : separate environment (+ databases) from content (notebooks) #13

Open erwanp opened 3 years ago

erwanp commented 3 years ago

We would need two repo : a new "radis-lab-env" repo with Anaconda & the databases (unzipped and cached), which generates a first Docker, and this one "radis-lab" with the notebooks that builds on the first Docker, so we can modify them without having to rebuild everything

See how : https://discourse.jupyter.org/t/how-to-reduce-mybinder-org-repository-startup-time/4956

This would also allow to use the "radis-lab-env" repo for other Forks, e.g. a student class with only the required notebooks.


Implementation :

we can have GitHub Actions build the Dockers. See link above, and also : https://github.com/jupyterhub/repo2docker-action Note : if so then we'll have to remove the Binder Build Action #16

anandxkumar commented 2 years ago

Possible solution: What we can do is we can create a docker image that will have the radis environment (May not be required if we only want the databases) and all the databases files, we can export this image to docker-hub. Next is we will have the original repo without a database and we will have a docker-compose file which will first build the environment and then pull the docker image that we exported to docker-hub and use it to extract its contents. This way every time there is a commit, even though the docker image will be built again, it only requires to pull the database image that already exists which can save a lot of time. Same has to be processed using repo2docker and dockerhub.

erwanp commented 2 years ago

@arunavabasu-03 I'd like if we could include this as part of your project --> it would allow to quickly deploy Notebooks to run examples and illustrate cases; would help all other GSOC students, and it's high impact . Do you think you'd have time for it ? Would you be intested ?

erwanp commented 2 years ago

This looks like a very useful ressource : https://mybinder.readthedocs.io/en/latest/howto/external_binder_setup.html