Print properly formatted error messages [FEATURE REQUEST]

[ashfaq-shaik]

In its current form. JSON API / DAML Runtime reports its error/stack trace as a continuous string, that is not readable. Example in the end.

For anyone who is troubleshooting, this imposes various challenges in terms of readability and accuracy of problem context, because if a parenthesis is missed while reading, the definition/hierarchy changes.

From a personal note. I am dyslexic, and it's a nightmare for me to understand the error and trace. With a pen and paper, I have to spend a good 10-15 min to read through, and in time-sensitive situations where there is an error in production environment, this challenge quadruples.

Is there a way to print/report the stack trace in a readable format? Similar to how Java/Javascript prints it.

2021-10-29 12:31:43.727 [http-json-ledger-api-akka.actor.default-dispatcher-69080] ERROR com.daml.http.CommandService - 'exercise failure  
io.grpc.StatusRuntimeException: INVALID_ARGUMENT: Command interpretation error in LF-DAMLe: Interpretation error: Error: User abort: fromSome: None. Details: Last location: [unknown source], partial transaction: root node NodeId(10): NodeExercises(ContractId(0070bf80a0fd35d0449ad539728249e9c97a7b2f70e3349f89475dc8aa8e5df972),cec09aafe949078c104466bcd52aa2774cc1ea4265f0fbdd81d7e3b3b6a45da5:Main.xyz.abc:abc,SomeChoice,None,false,TreeSet(ENTITYA),ValueRecord(Some(cec09aafe949078c104466bcd52aa2774cc1ea4265f0fbdd81d7e3b3b6a45da5:Main.xyz.abc:SomeChoice),ImmArray((Some(Something),ValueContractId(ContractId(0085768140e6ceb55102fbab768c074111a8244471a9bf9dd5addfe0a870db8e41))),(Some(SomeStatus),ValueText(AVAILABLE)))),TreeSet(ENTITYA, ENTITYB),TreeSet(ENTITYA),TreeSet(),ImmArray(NodeId(11),NodeId(12),NodeId(13),NodeId(14),NodeId(15),NodeId(16)),Some(ValueVariant(Some(40f452260bef3f29dede136108fc08a88d5a5250310281067087da6f0baddff7:DA.Types:Either),Right,ValueContractId(ContractId(0047dfab76ec5df03a9fcf50d16c632facdd991b1ad4971b203bfdd1ebbc2871a6)))),Some(KeyWithMaintainers(ValueRecord(Some(40f452260bef3f29dede136108fc08a88d5a5250310281067087da6f0baddff7:DA.Types:Tuple2),ImmArray((Some(_1),ValueParty(ENTITYA)),(Some(_2),ValueParty(ENTITYB)))),TreeSet(ENTITYA))),false,V11), node NodeId(11): NodeFetch(ContractId(0085768140e6ceb55102fbab768c074111a8244471a9bf9dd5addfe0a870db8e41),cec09aafe949078c104466bcd52aa2774cc1ea4265f0fbdd81d7e3b3b6a45da5:Main.xyz.abc:abc,None,TreeSet(ENTITYA),TreeSet(ENTITYA),TreeSet(ENTITYA),Some(KeyWithMaintainers(ValueRecord(Some(40f452260bef3f29dede136108fc08a88d5a5250310281067087da6f0baddff7:DA.Types:Tuple5),ImmArray((Some(_1),ValueParty(ENTITYA)),(Some(_2),ValueParty(ENTITYB)),(Some(_3),ValueText(ENTITYB)),(Some(_4),ValueText(SOMECODE)),(Some(_5),ValueText(SPLIT)))),TreeSet(ENTITYA))),false,V11), node NodeId(12): NodeLookupByKey(cec09aafe949078c104466bcd52aa2774cc1ea4265f0fbdd81d7e3b3b6a45da5:Main.xyz.abc:abc,None,KeyWithMaintainers(ValueRecord(Some(40f452260bef3f29dede136108fc08a88d5a5250310281067087da6f0baddff7:DA.Types:Tuple5),ImmArray((Some(_1),ValueParty(ENTITYA)),(Some(_2),ValueParty(ENTITYB)),(Some(_3),ValueText(ENTITYB)),(Some(_4),ValueText(SOMECODE)),(Some(_5),ValueText(AVAILABLE)))),TreeSet(ENTITYA)),Some(ContractId(0037ee3fd5a2ee9bea2cf2202a870c4cc0a5194dadf289f01a1494b977e735eb72)),V11), node NodeId(13): NodeFetch(ContractId(0037ee3fd5a2ee9bea2cf2202a870c4cc0a5194dadf289f01a1494b977e735eb72),cec09aafe949078c104466bcd52aa2774cc1ea4265f0fbdd81d7e3b3b6a45da5:Main.xyz.abc:abc,None,TreeSet(ENTITYA),TreeSet(ENTITYA),TreeSet(ENTITYA, ENTITYB),Some(KeyWithMaintainers(ValueRecord(Some(40f452260bef3f29dede136108fc08a88d5a5250310281067087da6f0baddff7:DA.Types:Tuple5),ImmArray((Some(_1),ValueParty(ENTITYA)),(Some(_2),ValueParty(ENTITYB)),(Some(_3),ValueText(ENTITYB)),(Some(_4),ValueText(SOMECODE)),(Some(_5),ValueText(AVAILABLE)))),TreeSet(ENTITYA))),false,V11), node NodeId(14): NodeExercises(ContractId(0037ee3fd5a2ee9bea2cf2202a870c4cc0a5194dadf289f01a1494b977e735eb72),cec09aafe949078c104466bcd52aa2774cc1ea4265f0fbdd81d7e3b3b6a45da5:Main.xyz.abc:abc,Archive,None,true,TreeSet(ENTITYA),ValueRecord(Some(d14e08374fc7197d6a0de468c968ae8ba3aadbf9315476fd39071831f5923662:DA.Internal.Template:Archive),ImmArray()),TreeSet(ENTITYA, ENTITYB),TreeSet(ENTITYA),TreeSet(),ImmArray(),Some(ValueUnit),Some(KeyWithMaintainers(ValueRecord(Some(40f452260bef3f29dede136108fc08a88d5a5250310281067087da6f0baddff7:DA.Types:Tuple5),ImmArray((Some(_1),ValueParty(ENTITYA)),(Some(_2),ValueParty(ENTITYB)),(Some(_3),ValueText(ENTITYB)),(Some(_4),ValueText(SOMECODE)),(Some(_5),ValueText(AVAILABLE)))),TreeSet(ENTITYA))),false,V11), node NodeId(15): NodeCreate(ContractId(0047dfab76ec5df03a9fcf50d16c632facdd991b1ad4971b203bfdd1ebbc2871a6),cec09aafe949078c104466bcd52aa2774cc1ea4265f0fbdd81d7e3b3b6a45da5:Main.xyz.abc:abc,ValueRecord(Some(cec09aafe949078c104466bcd52aa2774cc1ea4265f0fbdd81d7e3b3b6a45da5:Main.xyz.abc:abc),ImmArray((Some(operator),V...
    at io.grpc.Status.asRuntimeException(Status.java:534)
    at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:533)
    at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:553)
    at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:68)
    at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:739)
    at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:718)
    at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
    at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

[stefanobaghino-da]

Thanks for raising this. This looks like an error in the logs though. Ideally you should look at errors in the client. Since 1.18.x these errors will be reported in a cleaner way and flow through the JSON API to the client (almost) untouched.

@S11001001 is there already something done as part of the recent work in error reporting that improves the readability of errors coming from the ledger (I'd guess probably not for this instance in particular, as the work your recently did was around reporting errors from the ledger verbatim). Is there something coming along with self-service error codes that could possibly help here or would something like this need be addressed separately?

/cc @cocreature to have your thoughts on the engine error reporting, not sure if there's some change needed there as well.

[ashfaq-shaik]

This is a log excerpt from JSON API. I interface with the ledger using JSON API. Could you clarify what you mean by client?

In my context I have "Application" ----Connecting----> "JSON API" ------Talking to------> "VMBC LEDGER"

[cocreature]

/cc @cocreature to have your thoughts on the engine error reporting, not sure if there's some change needed there as well.

The engine reports errors as structured scala types which I believe are mapped to structural errors by the ledger API server and then that error turns into a JSON API exception which is what triggers the stacktrace.

[S11001001]

Is there something coming along with self-service error codes that could possibly help here or would something like this need be addressed separately?

9834 reflects that JSON API will emit a JSON representation of the details structure that it receives from ledger API on receiving StatusRuntimeException, as part of the self-service error codes changes.

That will help to the degree that this structure gives...structure to an error such as Command interpretation error in LF-DAMLe: Interpretation error: Error.... To the degree that it does not, it will not make a difference, and I don't think JSON API should get in the business of "parsing" and reformatting these errors.

[stefanobaghino-da]

/cc @cocreature to have your thoughts on the engine error reporting, not sure if there's some change needed there as well.

The engine reports errors as structured scala types which I believe are mapped to structural errors by the ledger API server and then that error turns into a JSON API exception which is what triggers the stacktrace.

@cocreature At which point is the error turned into a string instead of exposing the full structure of the partial transaction? If the JSON API receives a string, I agree with @S11001001 that it doesn't make sense to parse it.

[cocreature]

I think it’s in https://github.com/digital-asset/daml/blob/08236012af6348057ec27c43815e6809766c784b/ledger/participant-integration-api/src/main/scala/platform/apiserver/services/ApiSubmissionService.scala#L312. ErrorCause is the engine error type and then the ledger API server calls explain to turn that into a string.

[stefanobaghino-da]

@gerolf-da Your thoughts on the possibility of letting the Daml Engine error structure through instead of turning it into a string?

EDIT: looking at the comment, it looks like this is part of the scope for self-service error codes.

[gerolf-da]

Putting the transaction structure in a structured way into the error message is likely not going to work, because the error protocol has a size limit of the error. so we'll likely end up having to truncate the error message, which doesn't make sense for a large structure. See also google's documentation.

[stefanobaghino-da]

Putting the transaction structure in a structured way into the error message is likely not going to work, because the error protocol has a size limit of the error. so we'll likely end up having to truncate the error message, which doesn't make sense for a large structure. See also google's documentation.

Wouldn't this be true regardless of whether we expose the error in a structured way or as a string?