search

Kotlin / kotlin-jupyter

Kotlin kernel for Jupyter/IPython
Apache License 2.0
1.1k stars 105 forks source link

Preloading the Jupyter Notebook with external libraries #317

Open cnchann opened 2 years ago

cnchann commented 2 years ago

Is there a way to preload external libraries, such as %use lets-plot, before the start of every notebook/lab session similar to the startup feature in iPython, or are there any ways I could help in contributing to this feature?

altavir commented 2 years ago

Could you explain why do you need it? The dependencies are cached in the local maven repository, so they are effectively pre-loaded after the first run of the notebook.

cnchann commented 2 years ago

I am introducing a custom library using the %use command but it takes a long time to load. I wish to speed up the load time, are there any ways you would suggest going about it?

ileasile commented 2 years ago

There is no such way for now

altavir commented 2 years ago

@cnchann could you check why it takes a long time to load? As I said previously, if you have a large JAR, then you will have to wait for the first time until it is downloaded to the maven cache. The subsequent launches should be fast. If you have large resources (like JS bundle), then it could really take a lot of time to pre-load them in the notebook.

cnchann commented 2 years ago

@altavir i think it is because of a large JAR file, is there a way to preload it on maven cache, so it would be fast even on the first run? another question i would like to ask is if the maven cache is shared amongst all the notebooks in the jupyterHub or each notebook will have its own cache? thanks

altavir commented 2 years ago

@cnchann Maven cache is shared and usually located in ~/.m2 directory. Unlike npm or pip/conda environments, it stores different versions of the same artifact so you can use different versions in different notebooks. The simplest way to load your jars into the cache is to run a notebook with a single use directive.

I am not sure which dependency resolver is bundled in kotlin-scripting right now, but if it is updated to maven, then you can create a kts script include @DependsOn loads there and run it by kotlin command-line compiler.

Otherwise, I believe it is possible to install the maven command-line tool, create appropriate pom and do something along these lines: https://stackoverflow.com/questions/35944964/pre-download-all-dependencies. I am not sure that there is a simple solution using kotlin kernel own facilities.