write a jupyter overlay on inspire/scoap(?)

[MJedr]

When we are bugninjing or testing on qa/prod we're using pod shell which is very inconvenient because of few issues:

• you can't save your scripts which would be very useful for you and other people,
• pasting in the shell is terrible, especially when you paste a long script/text or an ES query. It would be extremely useful to add an overlay with jupyter lab on the top of the inspirehep app. It should have a shared storage with some dotfiles (maybe it would be nice to have a private and shared storage).

[MJedr]

In the last iteration, we tried to implement OAuth authorization, we added a new config for authorization but the variables from sealed secrets seem to be incorrect.

[MJedr]

Currently, the authentication is working. Steps to have inspire on jupyter:

• generate requirements.txt from poetry.lock on inspirehep repo (maybe add a pre-commit hook to do it whenever there's a change in poetry.lock, the conversion is fast enough)
• on merge to master build the image for jupyter and push it to registry
• use the image in jupyter (it would be good to have any automation for changing tags when new image is available e

[MJedr]

Updates:

• current image that we use registry.cern.ch/cern-sis/inspirehep/jupyter:test-requirements-3 is almost working, we need to update poetry dependencies to have freezgun as a main dependency
• to create an app you need to execute this code snippet:

  from inspirehep.factory import create_app as inspire_create_app
  app = inspire_create_app()

  maybe we can improve it (ideally have it in a cell when notebook opens idk if it's possible)

• we need to inject hep-cfg to the jupyter cause now the app uses local setup and doesn't work

[MJedr]

Updates 2/12: The app is finally working inside of the notebook. The next steps are:

• [ ] build CI on hep to release requirements.txt, build jupyterhub image & push the image to harbor.
• [ ] investigate how to start app context implicitly or at least don't use app_context context manager for each command.
• [ ] check the app on prod (will require creating hep-cfg probably) and adding some variables in kustomization.
• [ ] build an overlay for next.

[benjamin-bergia]

Could you maybe either:

• create overlays inside the jupyter app for inspire, scoap, etc
• create new apps like inspire-jupyter

The current hep app on argocd is way to large and we will have to split it up down the road. If the differences between inspire and scoap3 overlays are not to large I would recommend going with something like:

jupyter/
  base/
  overlays/
    inspire/
    scoap3/
  environments/
    inspire-prod/
    inspire-qa/
    scoap3-prod/
    scoap3-qa/

[MJedr]

@benjamin-bergia in the end we won't add jupyter on scoap, so should we restructure it anyway?

[benjamin-bergia]

yeah it's better.