Incorporate git notebooks into docker build more tightly.

[planetf1]

Currently when we deploy the notebook environment via docker-compose or k8s

• the egeria code comes from the last docker build
• the notebooks are downloaded on first-use via git.

One problem I've seen is that as the code (in master) is evolving rapidly along with the notebooks, sometimes the 'package' will break. This has varied from minutes to days (if the docker build doesn't work) but is typically < 1 day.

Recently the build of the egeria docker image has been made a lot quicker (#821) and ideally will be built after merge (#1630) which narrows the window, as long as the user keeps updating the image - a suggestion that was implemented via docs/defaults in #1617

the current blog proposal #1492 covers this scenario

However one way of avoiding this is to 'bake in' the notebooks as part of build process -this could be done in a few ways

• by strong version identification - so that we pull from git by sha hash , or a specific maven snapshot revision
• by including the notebooks within the image (we might need to build a custom notebook image layered on the base one for this).

This is certainly doable (and interesting timing given other changes in this area I'm working through)

Any comments @cmgrote @mandy-chessell

My git feel is to go ahead with getting the blog post off, the notebooks working as best they can right now, and only then change - what we have is pretty good & ready for more people to use, and then reconsider along with other feedback.

[planetf1]

To add another xref - this came about from discussion in #1634

[cmgrote]

My git feel is to go ahead with getting the blog post off, the notebooks working as best they can right now, and only then change - what we have is pretty good & ready for more people to use, and then reconsider along with other feedback.

I have the same "git" feeling 😉

[planetf1]

The current approach of downloading the notebooks from git master works fairly ok with docker images from master, but as we are about to ship 1.1 we get a problem - the 1.1 containers will likely rapidly get out of sync/break against the master notebooks.

The notebooks need to be more closely tied to the build - be it via embedding, pulling an artifact, or specifying a SHA.

I will create a PR for 1.1 to do a 'quick' fix, for example to just pull from git on the correct branch, perhaps hardcoded, and will then consider a better long term fix moving forward.

[planetf1]

One observation -- if the user's environment doesn't have network access after the initial pull of the images, startup will fail due to inability to get the notebooks. Could be more resiliant to account for this, but reduces flexibility