search

Kotlin / kotlin-jupyter

Kotlin kernel for Jupyter/IPython
Apache License 2.0
1.09k stars 106 forks source link

Kernel fails after new version of a resolved dependency is published to maven local #381

Open koperagen opened 1 year ago

koperagen commented 1 year ago

I use local build of the dataframe in a notebook to test some things. 

@file:Repository("*mavenLocal")
@file:DependsOn("org.jetbrains.kotlinx:dataframe:0.9.0-dev")

After publishing a new version, i get this error when running cells:

Exception ``` ZipFile invalid LOC header (bad signature) java.util.zip.ZipException: ZipFile invalid LOC header (bad signature) at java.base/java.util.zip.ZipFile$ZipFileInputStream.initDataOffset(ZipFile.java:1009) at java.base/java.util.zip.ZipFile$ZipFileInputStream.read(ZipFile.java:1019) at java.base/java.util.zip.ZipFile$ZipFileInflaterInputStream.fill(ZipFile.java:469) at java.base/java.util.zip.InflaterInputStream.read(InflaterInputStream.java:159) at java.base/java.io.FilterInputStream.read(FilterInputStream.java:133) at java.base/java.io.FilterInputStream.read(FilterInputStream.java:107) at kotlin.io.ByteStreamsKt.copyTo(IOStreams.kt:106) at kotlin.io.ByteStreamsKt.copyTo$default(IOStreams.kt:103) at kotlin.io.ByteStreamsKt.readBytes(IOStreams.kt:136) at kotlin.io.TextStreamsKt.readBytes(ReadWrite.kt:150) at org.jetbrains.kotlinx.jupyter.libraries.LibrariesScanner.scanForLibraries(LibrariesScanner.kt:59) at org.jetbrains.kotlinx.jupyter.libraries.LibrariesScanner.addLibrariesFromClassLoader(LibrariesScanner.kt:50) at org.jetbrains.kotlinx.jupyter.libraries.LibrariesScanner.addLibrariesFromClassLoader$default(LibrariesScanner.kt:49) at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl$execute$1$3.invoke(CellExecutorImpl.kt:98) at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl$execute$1$3.invoke(CellExecutorImpl.kt:97) at org.jetbrains.kotlinx.jupyter.config.LoggingKt.catchAll(logging.kt:41) at org.jetbrains.kotlinx.jupyter.config.LoggingKt.catchAll$default(logging.kt:40) at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl.execute(CellExecutorImpl.kt:97) at org.jetbrains.kotlinx.jupyter.repl.CellExecutor$DefaultImpls.execute$default(CellExecutor.kt:14) at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl$ExecutionContext.execute(CellExecutorImpl.kt:203) at org.jetbrains.kotlinx.dataframe.jupyter.Integration.onLoaded$execute(Integration.kt:115) at org.jetbrains.kotlinx.dataframe.jupyter.Integration.onLoaded$execute-19(Integration.kt:124) at org.jetbrains.kotlinx.dataframe.jupyter.Integration.access$onLoaded$execute-19(Integration.kt:51) at org.jetbrains.kotlinx.dataframe.jupyter.Integration$onLoaded$4.invoke(Integration.kt:128) at org.jetbrains.kotlinx.dataframe.jupyter.Integration$onLoaded$4.invoke(Integration.kt:127) at org.jetbrains.kotlinx.dataframe.jupyter.Integration$onLoaded$$inlined$updateVariable$1.execute(JupyterIntegration.kt:178) at org.jetbrains.kotlinx.jupyter.codegen.FieldsProcessorImpl$process$2.invoke(FieldsProcessorImpl.kt:43) at org.jetbrains.kotlinx.jupyter.codegen.FieldsProcessorImpl$process$2.invoke(FieldsProcessorImpl.kt:42) at org.jetbrains.kotlinx.jupyter.exceptions.ReplLibraryExceptionKt.rethrowAsLibraryException(ReplLibraryException.kt:27) at org.jetbrains.kotlinx.jupyter.codegen.FieldsProcessorImpl.process(FieldsProcessorImpl.kt:42) at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl$execute$1$1.invoke(CellExecutorImpl.kt:86) at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl$execute$1$1.invoke(CellExecutorImpl.kt:85) at org.jetbrains.kotlinx.jupyter.config.LoggingKt.catchAll(logging.kt:41) at org.jetbrains.kotlinx.jupyter.config.LoggingKt.catchAll$default(logging.kt:40) at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl.execute(CellExecutorImpl.kt:85) at org.jetbrains.kotlinx.jupyter.repl.CellExecutor$DefaultImpls.execute$default(CellExecutor.kt:14) at org.jetbrains.kotlinx.jupyter.ReplForJupyterImpl$evalEx$1.invoke(repl.kt:438) at org.jetbrains.kotlinx.jupyter.ReplForJupyterImpl$evalEx$1.invoke(repl.kt:427) at org.jetbrains.kotlinx.jupyter.ReplForJupyterImpl.withEvalContext(repl.kt:391) at org.jetbrains.kotlinx.jupyter.ReplForJupyterImpl.evalEx(repl.kt:427) at org.jetbrains.kotlinx.jupyter.ReplForJupyterImpl.eval(repl.kt:475) at org.jetbrains.kotlinx.jupyter.messaging.ProtocolKt$shellMessagesHandler$res$1.invoke(protocol.kt:301) at org.jetbrains.kotlinx.jupyter.messaging.ProtocolKt$shellMessagesHandler$res$1.invoke(protocol.kt:300) at org.jetbrains.kotlinx.jupyter.JupyterConnection$runExecution$execThread$1.invoke(connection.kt:180) at org.jetbrains.kotlinx.jupyter.JupyterConnection$runExecution$execThread$1.invoke(connection.kt:178) at kotlin.concurrent.ThreadsKt$thread$thread$1.run(Thread.kt:30) ZipFile invalid LOC header (bad signature) java.util.zip.ZipException: ZipFile invalid LOC header (bad signature) at java.base/java.util.zip.ZipFile$ZipFileInputStream.initDataOffset(ZipFile.java:1009) at java.base/java.util.zip.ZipFile$ZipFileInputStream.read(ZipFile.java:1019) at java.base/java.util.zip.ZipFile$ZipFileInflaterInputStream.fill(ZipFile.java:469) at java.base/java.util.zip.InflaterInputStream.read(InflaterInputStream.java:159) at java.base/java.io.FilterInputStream.read(FilterInputStream.java:133) at java.base/java.io.FilterInputStream.read(FilterInputStream.java:107) at kotlin.io.ByteStreamsKt.copyTo(IOStreams.kt:106) at kotlin.io.ByteStreamsKt.copyTo$default(IOStreams.kt:103) at kotlin.io.ByteStreamsKt.readBytes(IOStreams.kt:136) at kotlin.io.TextStreamsKt.readBytes(ReadWrite.kt:150) at org.jetbrains.kotlinx.jupyter.libraries.LibrariesScanner.scanForLibraries(LibrariesScanner.kt:59) at org.jetbrains.kotlinx.jupyter.libraries.LibrariesScanner.addLibrariesFromClassLoader(LibrariesScanner.kt:50) at org.jetbrains.kotlinx.jupyter.libraries.LibrariesScanner.addLibrariesFromClassLoader$default(LibrariesScanner.kt:49) at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl$execute$1$3.invoke(CellExecutorImpl.kt:98) at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl$execute$1$3.invoke(CellExecutorImpl.kt:97) at org.jetbrains.kotlinx.jupyter.config.LoggingKt.catchAll(logging.kt:41) at org.jetbrains.kotlinx.jupyter.config.LoggingKt.catchAll$default(logging.kt:40) at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl.execute(CellExecutorImpl.kt:97) at org.jetbrains.kotlinx.jupyter.repl.CellExecutor$DefaultImpls.execute$default(CellExecutor.kt:14) at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl$ExecutionContext.execute(CellExecutorImpl.kt:203) at org.jetbrains.kotlinx.dataframe.jupyter.Integration$onLoaded$$inlined$updateVariable$1.execute(JupyterIntegration.kt:182) at org.jetbrains.kotlinx.jupyter.codegen.FieldsProcessorImpl$process$2.invoke(FieldsProcessorImpl.kt:43) at org.jetbrains.kotlinx.jupyter.codegen.FieldsProcessorImpl$process$2.invoke(FieldsProcessorImpl.kt:42) at org.jetbrains.kotlinx.jupyter.exceptions.ReplLibraryExceptionKt.rethrowAsLibraryException(ReplLibraryException.kt:27) at org.jetbrains.kotlinx.jupyter.codegen.FieldsProcessorImpl.process(FieldsProcessorImpl.kt:42) at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl$execute$1$1.invoke(CellExecutorImpl.kt:86) at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl$execute$1$1.invoke(CellExecutorImpl.kt:85) at org.jetbrains.kotlinx.jupyter.config.LoggingKt.catchAll(logging.kt:41) at org.jetbrains.kotlinx.jupyter.config.LoggingKt.catchAll$default(logging.kt:40) at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl.execute(CellExecutorImpl.kt:85) at org.jetbrains.kotlinx.jupyter.repl.CellExecutor$DefaultImpls.execute$default(CellExecutor.kt:14) at org.jetbrains.kotlinx.jupyter.ReplForJupyterImpl$evalEx$1.invoke(repl.kt:438) at org.jetbrains.kotlinx.jupyter.ReplForJupyterImpl$evalEx$1.invoke(repl.kt:427) at org.jetbrains.kotlinx.jupyter.ReplForJupyterImpl.withEvalContext(repl.kt:391) at org.jetbrains.kotlinx.jupyter.ReplForJupyterImpl.evalEx(repl.kt:427) at org.jetbrains.kotlinx.jupyter.ReplForJupyterImpl.eval(repl.kt:475) at org.jetbrains.kotlinx.jupyter.messaging.ProtocolKt$shellMessagesHandler$res$1.invoke(protocol.kt:301) at org.jetbrains.kotlinx.jupyter.messaging.ProtocolKt$shellMessagesHandler$res$1.invoke(protocol.kt:300) at org.jetbrains.kotlinx.jupyter.JupyterConnection$runExecution$execThread$1.invoke(connection.kt:180) at org.jetbrains.kotlinx.jupyter.JupyterConnection$runExecution$execThread$1.invoke(connection.kt:178) at kotlin.concurrent.ThreadsKt$thread$thread$1.run(Thread.kt:30) ZipFile invalid LOC header (bad signature) java.util.zip.ZipException: ZipFile invalid LOC header (bad signature) at java.base/java.util.zip.ZipFile$ZipFileInputStream.initDataOffset(ZipFile.java:1009) at java.base/java.util.zip.ZipFile$ZipFileInputStream.read(ZipFile.java:1019) at java.base/java.util.zip.ZipFile$ZipFileInflaterInputStream.fill(ZipFile.java:469) at java.base/java.util.zip.InflaterInputStream.read(InflaterInputStream.java:159) at java.base/java.io.FilterInputStream.read(FilterInputStream.java:133) at java.base/java.io.FilterInputStream.read(FilterInputStream.java:107) at kotlin.io.ByteStreamsKt.copyTo(IOStreams.kt:106) at kotlin.io.ByteStreamsKt.copyTo$default(IOStreams.kt:103) at kotlin.io.ByteStreamsKt.readBytes(IOStreams.kt:136) at kotlin.io.TextStreamsKt.readBytes(ReadWrite.kt:150) at org.jetbrains.kotlinx.jupyter.libraries.LibrariesScanner.scanForLibraries(LibrariesScanner.kt:59) at org.jetbrains.kotlinx.jupyter.libraries.LibrariesScanner.addLibrariesFromClassLoader(LibrariesScanner.kt:50) at org.jetbrains.kotlinx.jupyter.libraries.LibrariesScanner.addLibrariesFromClassLoader$default(LibrariesScanner.kt:49) at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl$execute$1$3.invoke(CellExecutorImpl.kt:98) at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl$execute$1$3.invoke(CellExecutorImpl.kt:97) at org.jetbrains.kotlinx.jupyter.config.LoggingKt.catchAll(logging.kt:41) at org.jetbrains.kotlinx.jupyter.config.LoggingKt.catchAll$default(logging.kt:40) at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl.execute(CellExecutorImpl.kt:97) at org.jetbrains.kotlinx.jupyter.repl.CellExecutor$DefaultImpls.execute$default(CellExecutor.kt:14) at org.jetbrains.kotlinx.jupyter.ReplForJupyterImpl$evalEx$1.invoke(repl.kt:438) at org.jetbrains.kotlinx.jupyter.ReplForJupyterImpl$evalEx$1.invoke(repl.kt:427) at org.jetbrains.kotlinx.jupyter.ReplForJupyterImpl.withEvalContext(repl.kt:391) at org.jetbrains.kotlinx.jupyter.ReplForJupyterImpl.evalEx(repl.kt:427) at org.jetbrains.kotlinx.jupyter.ReplForJupyterImpl.eval(repl.kt:475) at org.jetbrains.kotlinx.jupyter.messaging.ProtocolKt$shellMessagesHandler$res$1.invoke(protocol.kt:301) at org.jetbrains.kotlinx.jupyter.messaging.ProtocolKt$shellMessagesHandler$res$1.invoke(protocol.kt:300) at org.jetbrains.kotlinx.jupyter.JupyterConnection$runExecution$execThread$1.invoke(connection.kt:180) at org.jetbrains.kotlinx.jupyter.JupyterConnection$runExecution$execThread$1.invoke(connection.kt:178) at kotlin.concurrent.ThreadsKt$thread$thread$1.run(Thread.kt:30) ```

I cannot re-run cell with dependencies, only restart of the kernel helps

ileasile commented 1 year ago

Hot jar reloading isn't supported by Java Machine, but we can improve message render here to give a hint about possible reason: https://github.com/Kotlin/kotlin-jupyter/blob/88f14af1306530126cfeb2da49e24374aba408b3/jupyter-lib/shared-compiler/src/main/kotlin/org/jetbrains/kotlinx/jupyter/exceptions/ReplException.kt#L11

Related links: https://youtrack.jetbrains.com/issue/IDEA-238769/Lots-of-ZipFile-invalid-LOC-header-bad-signature-in-the-logs#focus=Comments-27-4216894.0-0 https://randomizd.blogspot.com/2019/10/jar-file-handles-clean-up-after-your.html https://bugs.java.com/bugdatabase/view_bug.do?bug_id=4425695

koperagen commented 1 year ago

How can i write a unit test for it? I cannot trigger rendering of this exception in my local installation of kernel :( Running this test with a debugger also doesn't seem to reach that render function

   @Test
    fun `custom message for ZipException`() {
        exec("""
            import java.util.zip.ZipException
            throw Throwable(ZipException())
        """".trimIndent())
    }