search

enso-org / enso

Enso Analytics is a self-service data prep and analysis platform designed for data teams.
https://ensoanalytics.com
Apache License 2.0
7.38k stars 323 forks source link

Deleting a cache file causes run-time compiler error to be thrown #8421

Closed radeusgd closed 10 months ago

radeusgd commented 11 months ago

The repro that seems to work for me is:

  1. first do a clean buildEngineDistribution.
  2. then, remove rm .\built-distribution\enso-engine-0.0.0-dev-windows-amd64\enso-0.0.0-dev\lib\Standard\Database\0.0.0-dev\.enso\cache\ir\0.0.0-dev\Standard\Database\Connection\SQLite_Format.ir (the path may differ slightly on other platforms)
  3. and run enso run .\test\Table_Tests\src\IO\Formats_Spec.enso

For me it fails with

    - [FAILED] should fail gracefully when provided with an unsupported format [50ms]
        Reason: An unexpected panic was thrown: (Compile_Error.Error 'The name `Nothing` could not be found')
        at <enso> case_branch(C:\NBO\enso\built-distribution\enso-engine-0.0.0-dev-windows-amd64\enso-0.0.0-dev\lib\Standard\Database\0.0.0-dev\src\Connection\SQLite_Format.enso:33:18-24)
        at <enso> SQLite_Format.type.for_file_write(C:\NBO\enso\built-distribution\enso-engine-0.0.0-dev-windows-amd64\enso-0.0.0-dev\lib\Standard\Database\0.0.0-dev\src\Connection\SQLite_Format.enso:30-33)
        at <enso> <anonymous>(C:\NBO\enso\built-distribution\enso-engine-0.0.0-dev-windows-amd64\enso-0.0.0-dev\lib\Standard\Base\0.0.0-dev\src\System\File_Format.enso:69:24-44)
        at <enso> reader<arg-2>(C:\NBO\enso\built-distribution\enso-engine-0.0.0-dev-windows-amd64\enso-0.0.0-dev\lib\Standard\Base\0.0.0-dev\src\System\File_Format.enso:39:22-44)
        at <enso> reader(C:\NBO\enso\built-distribution\enso-eng
radeusgd commented 11 months ago

I was also encountering this randomly when just rebuilding the project with buildStdLibAll, possibly also buildEngineDistribution without a full-clean-build, after modifying some files (normal library development cycle).

However, the 'remove cache file' scenario seems to be the simplest repro that I could find and is less random than 'modify some file but I'm not sure which one'.

radeusgd commented 11 months ago

I'm getting this from time to time, e.g.

Various File Format support on Table:  [7/8, 1229ms]
    - [FAILED] should fail gracefully when provided with an unsupported format [48ms]
        Reason: An unexpected panic was thrown: (Compile_Error.Error 'The name `Nothing` could not be found')
        at <enso> Image_File_Format.type.for_file_read(X:\NBO\enso\built-distribution\enso-engine-0.0.0-dev-windows-amd64\enso-0.0.0-dev\lib\Standard\Image\0.0.0-dev\src\Image_File_Format.enso:21:78-84)
        at <enso> Image_File_Format.type.for_file_write(X:\NBO\enso\built-distribution\enso-engine-0.0.0-dev-windows-amd64\enso-0.0.0-dev\lib\Standard\Image\0.0.0-dev\src\Image_File_Format.enso:26:27-62)
        at <enso> <anonymous>(X:\NBO\enso\built-distribution\enso-engine-0.0.0-dev-windows-amd64\enso-0.0.0-dev\lib\Standard\Base\0.0.0-dev\src\System\File_Format.enso:68:24-44)
        at <enso> reader<arg-2>(X:\NBO\enso\built-distribution\enso-engine-0.0.0-dev-windows-amd64\enso-0.0.0-dev\lib\Standard\Base\0.0.0-dev\src\System\File_Format.enso:38:22-44)
JaroslavTulach commented 10 months ago

I tried to reproduce the problem, but I am hitting various issues. I replaced \ with /, but overall I am getting completely different errors.

radeusgd commented 10 months ago

I tried to reproduce the problem, but I am hitting various issues. I replaced \ with /, but overall I am getting completely different errors.

Could you please share what error are you getting?

JaroslavTulach commented 10 months ago 
enso$ rm -rf *; git checkout -f .
enso$ git log | head -n5
commit a1207e029d7780a54a6d09ca2665f7370493b4f8
Author: Radosław Waśko <radoslaw.wasko@enso.org>
Date:   Thu Jan 4 04:57:05 2024 +0100

    Unify `File_Format_Metadata` with `File_For_Read` (#8628)
enso$ sbt --java-home /graalvm buildEngineDistribution
$ rm -rf ~/.local/share/enso/cache/ir/
enso$ rm built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/lib/Standard/Database/0.0.0-dev/.enso/cache/ir/0.0.0-dev/Standard/Database/Connection/SQLite_Format.ir
enso$ ./built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/bin/enso --run ./test/Table_Tests/src/IO/Formats_Spec.enso 
enso/test/Table_Tests/src/IO/Formats_Spec.enso:11:1: error: The `project` keyword was used in an import statement, but the module does not belong to a project.
   11 | import project.Util
      | ^~~~~~~~~~~~~~~~~~~
Aborting due to 1 errors and 0 warnings.

the error persists even if I add --in-project ./test/Table_Tests/

JaroslavTulach commented 10 months ago

With following command I am able to reproduce the error:

enso$ ./built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/bin/enso --in-project test/Table_Tests/ --run test/Table_Tests/src/IO/Formats_Spec.enso
 - [FAILED] should fail gracefully when provided with an unsupported format [49ms]
        Reason: An unexpected panic was thrown: (Compile_Error.Error 'The name `Nothing` could not be found')
        at <enso> case_branch(/home/devel/NetBeansProjects/enso/enso/built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/lib/Standard/Database/0.0.0-dev/src/Connection/SQLite_Format.enso:36:18-24)
        at <enso> SQLite_Format.type.for_file_write(/home/devel/NetBeansProjects/enso/enso/built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/lib/Standard/Database/0.0.0-dev/src/Connection/SQLite_Format.enso:33-36)
        at <enso> <anonymous>(/home/devel/NetBeansProjects/enso/enso/built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/lib/Standard/Base/0.0.0-dev/src/System/File_Format.enso:69:24-44)
        at <enso> reader<arg-2>(/home/devel/NetBeansProjects/enso/enso/built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/lib/Standard/Base/0.0.0-dev/src/System/File_Format.enso:39:22-44)
        at <enso> reader(/home/devel/NetBeansProjects/enso/enso/built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/lib/Standard/Base/0.0.0-dev/src/System/File_Format.enso:38-41)

Prior to that error I also see following in the log, CCing @hubertp:

java.nio.file.NoSuchFileException: enso/built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/lib/Standard/Database/0.0.0-dev/.enso/cache/ir/0.0.0-dev/Standard/Database/Connection/SQLite_Format.ir
        at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
        at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
        at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
        at java.base/sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:224)
        at org.graalvm.truffle/com.oracle.truffle.polyglot.FileSystems$NIOFileSystem.newByteChannel(FileSystems.java:872)
        at org.graalvm.truffle/com.oracle.truffle.api.TruffleFile.newByteChannel(TruffleFile.java:803)
        at org.graalvm.truffle/com.oracle.truffle.api.TruffleFile.readAllBytes(TruffleFile.java:880)
        at org.enso.runtime/org.enso.interpreter.caches.Cache.loadCacheFrom(Cache.java:254)
        at org.enso.runtime/org.enso.interpreter.caches.Cache.lambda$load$1(Cache.java:208)
        at java.base/java.util.Optional.flatMap(Optional.java:289)
        at org.enso.runtime/org.enso.interpreter.caches.Cache.load(Cache.java:191)
        at org.enso.runtime/org.enso.interpreter.runtime.TruffleCompilerContext.loadCache(TruffleCompilerContext.java:179)
        at org.enso.runtime/org.enso.interpreter.runtime.SerializationManager.deserialize(SerializationManager.scala:412)
        at org.enso.runtime/org.enso.interpreter.runtime.TruffleCompilerContext.deserializeModule(TruffleCompilerContext.java:255)
        at org.enso.runtime/org.enso.compiler.Compiler.parseModule(Compiler.scala:543)
        at org.enso.runtime/org.enso.compiler.Compiler.$anonfun$runCompilerPipeline$1(Compiler.scala:244)
        at org.enso.runtime/org.enso.compiler.Compiler.$anonfun$runCompilerPipeline$1$adapted(Compiler.scala:242)
        at org.enso.runtime/scala.collection.immutable.List.foreach(List.scala:333)
        at org.enso.runtime/org.enso.compiler.Compiler.runCompilerPipeline(Compiler.scala:242)
        at org.enso.runtime/org.enso.compiler.Compiler.go$1(Compiler.scala:226)
        at org.enso.runtime/org.enso.compiler.Compiler.runInternal(Compiler.scala:233)
        at org.enso.runtime/org.enso.compiler.Compiler.run(Compiler.scala:128)
        at org.enso.runtime/org.enso.interpreter.runtime.Module.compile(Module.java:391)
        at org.enso.runtime/org.enso.interpreter.runtime.Module.compileScope(Module.java:318)
        at org.enso.runtime/org.enso.interpreter.runtime.Module$InvokeMember.doInvoke(Module.java:667)
        at org.enso.runtime/org.enso.interpreter.runtime.ModuleGen$InteropLibraryExports$Cached.invokeMember(ModuleGen.java:99)
        at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotValueDispatch$InteropValue$SharedInvokeNode.doDefault(PolyglotValueDispatch.java:4713)
        at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotValueDispatchFactory$InteropValueFactory$SharedInvokeNodeGen$Inlined.executeShared(PolyglotValueDispatchFactory.java:10235)
        at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotValueDispatch$InteropValue$InvokeNode.doDefault(PolyglotValueDispatch.java:4753)
        at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotValueDispatchFactory$InteropValueFactory$InvokeNodeGen.executeImpl(PolyglotValueDispatchFactory.java:10463)
        at org.graalvm.truffle/com.oracle.truffle.polyglot.HostToGuestRootNode.execute(HostToGuestRootNode.java:124)
        at org.graalvm.truffle.runtime/com.oracle.truffle.runtime.OptimizedCallTarget.executeRootNode(OptimizedCallTarget.java:746)
        at org.graalvm.truffle.runtime/com.oracle.truffle.runtime.OptimizedCallTarget.profiledPERoot(OptimizedCallTarget.java:669)
        at org.graalvm.truffle.runtime/com.oracle.truffle.runtime.OptimizedCallTarget.callBoundary(OptimizedCallTarget.java:602)
        at org.graalvm.truffle.runtime/com.oracle.truffle.runtime.OptimizedCallTarget.doInvoke(OptimizedCallTarget.java:586)
        at org.graalvm.truffle.runtime/com.oracle.truffle.runtime.OptimizedRuntimeSupport.callProfiled(OptimizedRuntimeSupport.java:266)
        at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotValueDispatch$InteropValue.invoke(PolyglotValueDispatch.java:2634)
        at org.graalvm.polyglot/org.graalvm.polyglot.Value.invokeMember(Value.java:974)
        at org.enso.base.file_format.FileFormatSPI.getTypeObject(FileFormatSPI.java:21)
        at org.enso.base.file_format.FileFormatSPI.lambda$get_types$0(FileFormatSPI.java:15)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
        at java.base/java.util.ServiceLoader$ProviderSpliterator.tryAdvance(ServiceLoader.java:1499)
        at java.base/java.util.Spliterator.forEachRemaining(Spliterator.java:332)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:575)
        at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260)
        at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:616)
        at org.enso.base.file_format.FileFormatSPI.get_types(FileFormatSPI.java:15)
enso-bot[bot] commented 10 months ago

Jaroslav Tulach reports a new STANDUP for yesterday (2024-01-04):

Progress: - Bugfix #8421 by https://github.com/enso-org/enso/pull/8669

Next Day: TCK, interop & co.

enso-bot[bot] commented 10 months ago

Jaroslav Tulach reports a new STANDUP for yesterday (2024-01-05):

Progress: - merged https://github.com/enso-org/enso/pull/8669

Next Day: TCK, interop & co.