search

enso-org / enso

Hybrid visual and textual functional programming.
https://enso.org
Apache License 2.0
7.34k stars 323 forks source link

Transient issues in backend tests #7775

Closed Akirathan closed 1 week ago

Akirathan commented 1 year ago

As of today, September 2023, we experience frequent Engine and Stdlib transient job failures. This issue serves as a list of all these issues so that we can keep track of them.

4e6 commented 1 year ago

Issues with the project rename tests (RefactoringTests suite) should be fixed by #7794

4e6 commented 1 year ago

Added to the list: InliningBuiltinsTest (NoSuchMethodError) - https://github.com/enso-org/enso/actions/runs/6147863209/job/16680272086?pr=7794#step:10:4583

GitHub
Fix flaky project rename test · enso-org/enso@03a2d33
Hybrid visual and textual functional programming. Contribute to enso-org/enso development by creating an account on GitHub.
Akirathan commented 1 year ago

"Enso test failures in Excel" transient failures - After discussion with @JaroslavTulach, we think that the issue might be cause by enso_project.data returning a different directory than expected. Probably caused by a GC issue in https://github.com/enso-org/enso/blob/develop/engine/runtime/src/main/java/org/enso/interpreter/node/expression/builtin/meta/EnsoProjectNode.java#L96C12-L96C12. Diagnostics added in https://github.com/enso-org/enso/pull/7793/commits/7f9c05d4d64575919cc3054447c5c77a7c798579

GregoryTravis commented 1 year ago

Added to the list: java.io.IOException: Could not create lock for \\.\pipe\sbt-load-802998070017202719_lock, error 1336

GitHub
Improve HTTP tests · enso-org/enso@fc8ad0e
Hybrid visual and textual functional programming. Contribute to enso-org/enso development by creating an account on GitHub.
Akirathan commented 1 year ago

Added to the list: java.io.IOException: Could not create lock for \\.\pipe\sbt-load-802998070017202719_lock, error 1336

* https://github.com/enso-org/enso/actions/runs/6238498583/job/16934446773?pr=7847

GitHubImprove HTTP tests · enso-org/enso@fc8ad0eHybrid visual and textual functional programming. Contribute to enso-org/enso development by creating an account on GitHub.

This is actually the same error as under sbt: ServerAlreadyBootingException category - https://github.com/enso-org/enso/actions/runs/6238498583/job/16934446773?pr=7847#step:10:676. Moving it up in the list.

GitHub
Improve HTTP tests · enso-org/enso@fc8ad0e
Hybrid visual and textual functional programming. Contribute to enso-org/enso development by creating an account on GitHub.
radeusgd commented 11 months ago

I added to the list an error I saw on my CI build:

 INFO ide_ci::program::command: sbtℹ️ /runner/_work/enso/enso/distribution/lib/Standard/Base/0.0.0-dev/src/Data.enso: error: The module Standard.Base.Data.Enso_Cloud does not exist.
 INFO ide_ci::program::command: sbtℹ️ [error] Test org.enso.interpreter.test.MetaIsATest.checkNumberIsNotInstanceOfNumber failed: org.graalvm.polyglot.PolyglotException: Compilation aborted due to errors., took 0.066 sec
 INFO ide_ci::program::command: sbtℹ️ [error]     at org.graalvm.polyglot.Value.invokeMember(Value.java:974)
 INFO ide_ci::program::command: sbtℹ️ [error]     at org.enso.interpreter.test.ValuesGenerator.v(ValuesGenerator.java:71)
 INFO ide_ci::program::command: sbtℹ️ [error]     at org.enso.interpreter.test.ValuesGenerator.v(ValuesGenerator.java:60)
 INFO ide_ci::program::command: sbtℹ️ [error]     at org.enso.interpreter.test.ValuesGenerator.typeNumber(ValuesGenerator.java:157)
 INFO ide_ci::program::command: sbtℹ️ [error]     at org.enso.interpreter.test.MetaIsATest.checkNumberIsNotInstanceOfNumber(MetaIsATest.java:150)
 INFO ide_ci::program::command: sbtℹ️ [error]     at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 INFO ide_ci::program::command: sbtℹ️ [error]     at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
 INFO ide_ci::program::command: sbtℹ️ [error]     at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 INFO ide_ci::program::command: sbtℹ️ [error]     at java.lang.reflect.Method.invoke(Method.java:568)
 INFO ide_ci::program::command: sbtℹ️ [error]     ...

So apparently it looks like an import Standard.Base.Data.Enso_Cloud that comes from a different PR must have somehow leaked to my branch.

radeusgd commented 9 months ago

I added one more to the list - the startup time benchmark seems to be failing from time to time with

 INFO ide_ci::program::command: cmdℹ️ Found 2 cases to execute (ETA 50s)
 INFO ide_ci::program::command: cmdℹ️ Benchmarking 'Startup.empty_startup' with configuration: [warmup={2 iterations, 5 seconds each}, measurement={3 iterations, 5 seconds each}]
 INFO ide_ci::program::command: cmdℹ️ Execution finished with an error: Cannot find enso-engine-* in C:\runner\_work\enso\enso\built-distribution
ERROR ide_ci::program::command: cmdℹ️ Failed to decode a line from output: invalid utf-8 sequence of 1 bytes from index 5
 WARN ide_ci::program::command: cmdℹ️ Raw buffer: [67, 58, 92, 114, 32, 133, 13]. Decoded with placeholders: C:\r �
 INFO ide_ci::program::command: cmdℹ️         at <enso> Panic.throw(Internal)
 INFO ide_ci::program::command: cmdℹ️         at <enso> find_prefix<arg-2>(src\Startup\Startup.enso:55:13-27)
 INFO ide_ci::program::command: cmdℹ️         at <enso> find_prefix(src\Startup\Startup.enso:51-55)
 INFO ide_ci::program::command: cmdℹ️         at <enso> Startup.enso_bin(src\Startup\Startup.enso:60:19-63)
 INFO ide_ci::program::command: cmdℹ️         at <enso> suspended-block<arg-1>(src\Startup\Startup.enso:37:20-27)
 INFO ide_ci::program::command: cmdℹ️         at <enso> null(Internal)
 INFO ide_ci::program::command: cmdℹ️         at <enso> Data.enso_bin(Internal)
 INFO ide_ci::program::command: cmdℹ️         at <enso> Data.startup(src\Startup\Startup.enso:18:15-27)
 INFO ide_ci::program::command: cmdℹ️         at <enso> Data.bench_empty(src\Startup\Startup.enso:12:9-58)
 INFO ide_ci::program::command: cmdℹ️         at <enso> <anonymous><arg-2>(src\Startup\Startup.enso:40:47-62)
 INFO ide_ci::program::command: cmdℹ️         at <enso> <anonymous>(C:\runner\_work\enso\enso\built-distribution\enso-engine-2023.2.1-dev-windows-amd64\enso-2023.2.1-dev\lib\Standard\Test\2023.2.1-dev\src\Bench.enso:74:54-62)
 INFO ide_ci::program::command: cmdℹ️         at <enso> <anonymous><arg-4>(C:\runner\_work\enso\enso\built-distribution\enso-engine-2023.2.1-dev-windows-amd64\enso-2023.2.1-dev\lib\Standard\Test\2023.2.1-dev\src\Bench.enso:172:58-65)
 INFO ide_ci::program::command: cmdℹ️         at <enso> Runtime.no_inline(Internal)
 INFO ide_ci::program::command: cmdℹ️         at <enso> Bench.type.single_call(C:\runner\_work\enso\enso\built-distribution\enso-engine-2023.2.1-dev-windows-amd64\enso-2023.2.1-dev\lib\Standard\Test\2023.2.1-dev\src\Bench.enso:214:13-33)
 INFO ide_ci::program::command: cmdℹ️         at <enso> case_branch(C:\runner\_work\enso\enso\built-distribution\enso-engine-2023.2.1-dev-windows-amd64\enso-2023.2.1-dev\lib\Standard\Test\2023.2.1-dev\src\Bench.enso:196:31-51)
 INFO ide_ci::program::command: cmdℹ️         at <enso> Bench.type.measure(C:\runner\_work\enso\enso\built-distribution\enso-engine-2023.2.1-dev-windows-amd64\enso-2023.2.1-dev\lib\Standard\Test\2023.2.1-dev\src\Bench.enso:194-207)
 INFO ide_ci::program::command: cmdℹ️         at <enso> <anonymous>(C:\runner\_work\enso\enso\built-distribution\enso-engine-2023.2.1-dev-windows-amd64\enso-2023.2.1-dev\lib\Standard\Test\2023.2.1-dev\src\Bench.enso:172:13-66)
 INFO ide_ci::program::command: cmdℹ️         at <enso> <anonymous>(C:\runner\_work\enso\enso\built-distribution\enso-engine-2023.2.1-dev-windows-amd64\enso-2023.2.1-dev\lib\Standard\Test\2023.2.1-dev\src\Bench.enso:135:60-70)
 INFO ide_ci::program::command: cmdℹ️         at <enso> f(C:\runner\_work\enso\enso\built-distribution\enso-engine-2023.2.1-dev-windows-amd64\enso-2023.2.1-dev\lib\Standard\Base\2023.2.1-dev\src\Data\Vector.enso:258:26-50)
 INFO ide_ci::program::command: cmdℹ️         at <enso> go<arg-2>(C:\runner\_work\enso\enso\built-distribution\enso-engine-2023.2.1-dev-windows-amd64\enso-2023.2.1-dev\lib\Standard\Base\2023.2.1-dev\src\Data\Range.enso:312:31-50)
 INFO ide_ci::program::command: cmdℹ️         at <enso> go(C:\runner\_work\enso\enso\built-distribution\enso-engine-2023.2.1-dev-windows-amd64\enso-2023.2.1-dev\lib\Standard\Base\2023.2.1-dev\src\Data\Range.enso:311-313)
 INFO ide_ci::program::command: cmdℹ️         at <enso> Range.fold<arg-2>(C:\runner\_work\enso\enso\built-distribution\enso-engine-2023.2.1-dev-windows-amd64\enso-2023.2.1-dev\lib\Standard\Base\2023.2.1-dev\src\Data\Range.enso:314:13-30)
 INFO ide_ci::program::command: cmdℹ️         at <enso> Range.fold(C:\runner\_work\enso\enso\built-distribution\enso-engine-2023.2.1-dev-windows-amd64\enso-2023.2.1-dev\lib\Standard\Base\2023.2.1-dev\src\Data\Range.enso:308-314)
 INFO ide_ci::program::command: cmdℹ️         at <enso> Vector.fold(C:\runner\_work\enso\enso\built-distribution\enso-engine-2023.2.1-dev-windows-amd64\enso-2023.2.1-dev\lib\Standard\Base\2023.2.1-dev\src\Data\Vector.enso:259:9-41)
 INFO ide_ci::program::command: cmdℹ️         at <enso> case_branch(C:\runner\_work\enso\enso\built-distribution\enso-engine-2023.2.1-dev-windows-amd64\enso-2023.2.1-dev\lib\Standard\Test\2023.2.1-dev\src\Bench.enso:135:34-71)
 INFO ide_ci::program::command: cmdℹ️         at <enso> Bench.fold(C:\runner\_work\enso\enso\built-distribution\enso-engine-2023.2.1-dev-windows-amd64\enso-2023.2.1-dev\lib\Standard\Test\2023.2.1-dev\src\Bench.enso:133-136)
 INFO ide_ci::program::command: cmdℹ️         at <enso> <anonymous>(C:\runner\_work\enso\enso\built-distribution\enso-engine-2023.2.1-dev-windows-amd64\enso-2023.2.1-dev\lib\Standard\Test\2023.2.1-dev\src\Bench.enso:134:56-66)
 INFO ide_ci::program::command: cmdℹ️         at <enso> f(C:\runner\_work\enso\enso\built-distribution\enso-engine-2023.2.1-dev-windows-amd64\enso-2023.2.1-dev\lib\Standard\Base\2023.2.1-dev\src\Data\Vector.enso:258:26-50)
 INFO ide_ci::program::command: cmdℹ️         at <enso> go<arg-2>(C:\runner\_work\enso\enso\built-distribution\enso-engine-2023.2.1-dev-windows-amd64\enso-2023.2.1-dev\lib\Standard\Base\2023.2.1-dev\src\Data\Range.enso:312:31-50)
 INFO ide_ci::program::command: cmdℹ️         at <enso> go(C:\runner\_work\enso\enso\built-distribution\enso-engine-2023.2.1-dev-windows-amd64\enso-2023.2.1-dev\lib\Standard\Base\2023.2.1-dev\src\Data\Range.enso:311-313)
 INFO ide_ci::program::command: cmdℹ️         at <enso> Range.fold<arg-2>(C:\runner\_work\enso\enso\built-distribution\enso-engine-2023.2.1-dev-windows-amd64\enso-2023.2.1-dev\lib\Standard\Base\2023.2.1-dev\src\Data\Range.enso:314:13-30)
 INFO ide_ci::program::command: cmdℹ️         at <enso> Range.fold(C:\runner\_work\enso\enso\built-distribution\enso-engine-2023.2.1-dev-windows-amd64\enso-2023.2.1-dev\lib\Standard\Base\2023.2.1-dev\src\Data\Range.enso:308-314)
 INFO ide_ci::program::command: cmdℹ️         at <enso> Vector.fold(C:\runner\_work\enso\enso\built-distribution\enso-engine-2023.2.1-dev-windows-amd64\enso-2023.2.1-dev\lib\Standard\Base\2023.2.1-dev\src\Data\Vector.enso:259:9-41)
 INFO ide_ci::program::command: cmdℹ️         at <enso> case_branch(C:\runner\_work\enso\enso\built-distribution\enso-engine-2023.2.1-dev-windows-amd64\enso-2023.2.1-dev\lib\Standard\Test\2023.2.1-dev\src\Bench.enso:134:29-67)
 INFO ide_ci::program::command: cmdℹ️         at <enso> Bench.fold(C:\runner\_work\enso\enso\built-distribution\enso-engine-2023.2.1-dev-windows-amd64\enso-2023.2.1-dev\lib\Standard\Test\2023.2.1-dev\src\Bench.enso:133-136)
 INFO ide_ci::program::command: cmdℹ️         at <enso> Bench.run_main(C:\runner\_work\enso\enso\built-distribution\enso-engine-2023.2.1-dev-windows-amd64\enso-2023.2.1-dev\lib\Standard\Test\2023.2.1-dev\src\Bench.enso:168-172)
 INFO ide_ci::program::command: cmdℹ️         at <enso> <anonymous>(src\Main.enso:118:9-22)
 INFO ide_ci::program::command: cmdℹ️         at <enso> <anonymous>(C:\runner\_work\enso\enso\built-distribution\enso-engine-2023.2.1-dev-windows-amd64\enso-2023.2.1-dev\lib\Standard\Base\2023.2.1-dev\src\Data\Vector.enso:723:13-26)
 INFO ide_ci::program::command: cmdℹ️         at <enso> go<arg-2>(C:\runner\_work\enso\enso\built-distribution\enso-engine-2023.2.1-dev-windows-amd64\enso-2023.2.1-dev\lib\Standard\Base\2023.2.1-dev\src\Data\Range.enso:243:21-36)
 INFO ide_ci::program::command: cmdℹ️         at <enso> go(C:\runner\_work\enso\enso\built-distribution\enso-engine-2023.2.1-dev-windows-amd64\enso-2023.2.1-dev\lib\Standard\Base\2023.2.1-dev\src\Data\Range.enso:242-244)
mwu-tow commented 8 months ago

Another case of RuntimeServerTest - The language did not complete all polyglot threads: https://github.com/enso-org/enso/actions/runs/7639249532/job/20811863728?pr=8844#step:10:3283 Log archive

GitHub
Align collapsed node to the leftmost selected one · enso-org/enso@2b9d798
Hybrid visual and textual functional programming. Contribute to enso-org/enso development by creating an account on GitHub.
mwu-tow commented 8 months ago

A new one: Transient Engine Test Failures: RuntimeTextEditsTest and RuntimeSuggestionUpdatesTest

farmaazon commented 7 months ago

I think I found one not yet listed above: should be able to use external libraries if they are needed by the visualization *** FAILED *** https://github.com/enso-org/enso/actions/runs/7786527964/job/21231559557?pr=8972#step:8:2204

GitHub
Fix vector widget in non-argument literal · enso-org/enso@d85ce71
Hybrid visual and textual functional programming. Contribute to enso-org/enso development by creating an account on GitHub.
mwu-tow commented 7 months ago

Reported next one: Transient Engine Test Failure: S3.get_object "Read timed out"

mwu-tow commented 7 months ago

Another one: HttpDownloaderTest.downloadBigFileWithProgress

mwu-tow commented 6 months ago

A new one:

mwu-tow commented 6 months ago
JaroslavTulach commented 1 week ago

We fixed something. Let's close this ticket and collect new issues.