JS Errors obscure all previous JS console logs

Is there an existing issue for this?

[X] I have searched the existing issues and didn't find mine.

Steps to reproduce

Test file

appId: test
jsEngine: graaljs
---
- runScript: script-error.js

test script

console.log(`${Date.now()} hello`);
throw new Error('This is an error');

Run test with maestro test test-error.yaml

Actual results

Error message is displayed console. Console.log is not seen.

Expected results

Error message is displayed console. Console.log output should be visible somewhere

About app

no app

About environment

java -version
openjdk version "21.0.4" 2024-07-16 LTS
OpenJDK Runtime Environment Temurin-21.0.4+7 (build 21.0.4+7-LTS)
OpenJDK 64-Bit Server VM Temurin-21.0.4+7 (build 21.0.4+7-LTS, mixed mode)

OSX 14.7 Sonoma arm64

Logs

``` Error: This is an error at :program(script-error.js:2:42-70) at org.graalvm.polyglot.Context.eval(Context.java:399) at maestro.js.GraalJsEngine.evaluateScript(GraalJsEngine.kt:70) at maestro.js.GraalJsEngine.evaluateScript(GraalJsEngine.kt:22) at maestro.orchestra.Orchestra.runScriptCommand(Orchestra.kt:419) at maestro.orchestra.Orchestra.executeCommand(Orchestra.kt:279) at maestro.orchestra.Orchestra.executeCommands(Orchestra.kt:191) at maestro.orchestra.Orchestra.runFlow(Orchestra.kt:127) at maestro.cli.runner.MaestroCommandRunner.runCommands(MaestroCommandRunner.kt:200) at maestro.cli.runner.TestRunner$runSingle$result$1.invoke(TestRunner.kt:61) at maestro.cli.runner.TestRunner$runSingle$result$1.invoke(TestRunner.kt:53) at maestro.cli.runner.TestRunner.runCatching(TestRunner.kt:155) at maestro.cli.runner.TestRunner.runSingle(TestRunner.kt:53) at maestro.cli.command.TestCommand$handleSessions$1$1$results$1$1$1.invoke(TestCommand.kt:306) at maestro.cli.command.TestCommand$handleSessions$1$1$results$1$1$1.invoke(TestCommand.kt:258) at maestro.cli.session.MaestroSessionManager.newSession(MaestroSessionManager.kt:102) at maestro.cli.session.MaestroSessionManager.newSession$default(MaestroSessionManager.kt:54) at maestro.cli.command.TestCommand$handleSessions$1$1$results$1$1.invokeSuspend(TestCommand.kt:258) at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:104) at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:111) at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:99) at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:585) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:802) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:706) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:693) ==== Debug output (logs & screenshots) ==== ``` maestro.log ``` [INFO ] m.cli.runner.MaestroCommandRunner - Apply configuration RUNNING [INFO ] m.cli.runner.MaestroCommandRunner - Apply configuration metadata CommandMetadata(numberOfRuns=null, evaluatedCommand=MaestroCommand(tapOnElement=null, tapOnPoint=null, tapOnPointV2Command=null, scrollCommand=null, swipeCommand=null, backPressCommand=null, assertCommand=null, assertConditionCommand=null, assertNoDefectsWithAICommand=null, assertWithAICommand=null, inputTextCommand=null, inputRandomTextCommand=null, launchAppCommand=null, applyConfigurationCommand=ApplyConfigurationCommand(config=MaestroConfig(appId=test, name=null, tags=[], ext={jsEngine=graaljs}, onFlowStart=null, onFlowComplete=null), label=null), openLinkCommand=null, pressKeyCommand=null, eraseTextCommand=null, hideKeyboardCommand=null, takeScreenshotCommand=null, stopAppCommand=null, killAppCommand=null, clearStateCommand=null, clearKeychainCommand=null, runFlowCommand=null, setLocationCommand=null, repeatCommand=null, copyTextCommand=null, pasteTextCommand=null, defineVariablesCommand=null, runScriptCommand=null, waitForAnimationToEndCommand=null, evalScriptCommand=null, scrollUntilVisible=null, travelCommand=null, startRecordingCommand=null, stopRecordingCommand=null, addMediaCommand=null, setAirplaneModeCommand=null, toggleAirplaneModeCommand=null), logMessages=[], insight=Insight(message=, level=NONE)) [INFO ] m.cli.runner.MaestroCommandRunner - Apply configuration COMPLETED [INFO ] m.cli.runner.MaestroCommandRunner - Run script-error.js RUNNING [INFO ] m.cli.runner.MaestroCommandRunner - Run script-error.js metadata CommandMetadata(numberOfRuns=null, evaluatedCommand=MaestroCommand(tapOnElement=null, tapOnPoint=null, tapOnPointV2Command=null, scrollCommand=null, swipeCommand=null, backPressCommand=null, assertCommand=null, assertConditionCommand=null, assertNoDefectsWithAICommand=null, assertWithAICommand=null, inputTextCommand=null, inputRandomTextCommand=null, launchAppCommand=null, applyConfigurationCommand=null, openLinkCommand=null, pressKeyCommand=null, eraseTextCommand=null, hideKeyboardCommand=null, takeScreenshotCommand=null, stopAppCommand=null, killAppCommand=null, clearStateCommand=null, clearKeychainCommand=null, runFlowCommand=null, setLocationCommand=null, repeatCommand=null, copyTextCommand=null, pasteTextCommand=null, defineVariablesCommand=null, runScriptCommand=RunScriptCommand(script=console.log(`${Date.now()} hello`); throw new Error('This is an error'); , env={}, sourceDescription=script-error.js, condition=null, label=null), waitForAnimationToEndCommand=null, evalScriptCommand=null, scrollUntilVisible=null, travelCommand=null, startRecordingCommand=null, stopRecordingCommand=null, addMediaCommand=null, setAirplaneModeCommand=null, toggleAirplaneModeCommand=null), logMessages=[], insight=Insight(message=, level=NONE)) [INFO ] m.cli.runner.MaestroCommandRunner - Run script-error.js metadata CommandMetadata(numberOfRuns=null, evaluatedCommand=MaestroCommand(tapOnElement=null, tapOnPoint=null, tapOnPointV2Command=null, scrollCommand=null, swipeCommand=null, backPressCommand=null, assertCommand=null, assertConditionCommand=null, assertNoDefectsWithAICommand=null, assertWithAICommand=null, inputTextCommand=null, inputRandomTextCommand=null, launchAppCommand=null, applyConfigurationCommand=null, openLinkCommand=null, pressKeyCommand=null, eraseTextCommand=null, hideKeyboardCommand=null, takeScreenshotCommand=null, stopAppCommand=null, killAppCommand=null, clearStateCommand=null, clearKeychainCommand=null, runFlowCommand=null, setLocationCommand=null, repeatCommand=null, copyTextCommand=null, pasteTextCommand=null, defineVariablesCommand=null, runScriptCommand=RunScriptCommand(script=console.log(`${Date.now()} hello`); throw new Error('This is an error'); , env={}, sourceDescription=script-error.js, condition=null, label=null), waitForAnimationToEndCommand=null, evalScriptCommand=null, scrollUntilVisible=null, travelCommand=null, startRecordingCommand=null, stopRecordingCommand=null, addMediaCommand=null, setAirplaneModeCommand=null, toggleAirplaneModeCommand=null), logMessages=[1728397000272 hello], insight=Insight(message=, level=NONE)) [INFO ] maestro.Maestro - Taking screenshot [INFO ] maestro.utils.ScreenshotUtils - Taking screenshot to output sink [ERROR] maestro.cli.runner.TestRunner - Failed to run flow org.graalvm.polyglot.PolyglotException: Error: This is an error at .:program(script-error.js:2) at org.graalvm.polyglot.Context.eval(Context.java:399) at maestro.js.GraalJsEngine.evaluateScript(GraalJsEngine.kt:70) at maestro.js.GraalJsEngine.evaluateScript(GraalJsEngine.kt:22) at maestro.orchestra.Orchestra.runScriptCommand(Orchestra.kt:419) at maestro.orchestra.Orchestra.executeCommand(Orchestra.kt:279) at maestro.orchestra.Orchestra.executeCommands(Orchestra.kt:191) ```

Maestro version

1.38.1

How did you install Maestro?

install script (https://get.maestro.mobile.dev)

Anything else?

Once an error is thrown in JS then there is no way to get any other console.logs produced by JS. This is a pain when debugging.

The console output is deleted and replaced by the error.
None of the output files have the JS console.log() outputs in them

The workaround is to comment out code that is causing the error, debug and then put the code back in again when fixed, which is painful.

I've also tried maestro test test-error.yaml > out.txt maestro test test-error.yaml > out.txt 2>&1 none of these options gets me the error and the console.logs at the same time.

Suggested fix:

add the JS output to the maestro.log
don't replace the console output with errors from JS, just stop the test and append any errors

mobile-dev-inc / maestro