Stream error Error: 1 CANCELLED: Cancelled on client

[acheong08]

Expected Behavior

Run example

Actual Behavior

== APP == Instance 6cbc632a-6487-483a-a528-9772c8e12310 completed
== APP == Orchestration completed! Result: ["Hello Tokyo!","Hello Seattle!","Hello London!"]
== APP == Stream error Error: 1 CANCELLED: Cancelled on client
== APP ==     at callErrorFromStatus (/tmp/js-sdk/examples/workflow/authoring/node_modules/@grpc/grpc-js/build/src/call.js:31:19)
== APP ==     at Object.onReceiveStatus (/tmp/js-sdk/examples/workflow/authoring/node_modules/@grpc/grpc-js/build/src/client.js:358:73)
== APP ==     at Object.onReceiveStatus (/tmp/js-sdk/examples/workflow/authoring/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:323:181)
== APP ==     at /tmp/js-sdk/examples/workflow/authoring/node_modules/@grpc/grpc-js/build/src/resolving-call.js:129:78
== APP ==     at process.processTicksAndRejections (node:internal/process/task_queues:77:11)
== APP == for call at
== APP ==     at ServiceClientImpl.makeServerStreamRequest (/tmp/js-sdk/examples/workflow/authoring/node_modules/@grpc/grpc-js/build/src/client.js:341:32)
== APP ==     at ServiceClientImpl.getWorkItems (/tmp/js-sdk/examples/workflow/authoring/node_modules/@grpc/grpc-js/build/src/make-client.js:105:19)
== APP ==     at TaskHubGrpcWorker.start (/tmp/js-sdk/examples/workflow/authoring/node_modules/@microsoft/durabletask-js/worker/task-hub-grpc-worker.js:112:44)
== APP ==     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
== APP ==     at async WorkflowRuntime.start (/tmp/js-sdk/examples/workflow/authoring/node_modules/@dapr/dapr/workflow/runtime/WorkflowRuntime.js:110:9)
== APP ==     at async start (/tmp/js-sdk/examples/workflow/authoring/dist/activity-sequence.js:44:9) {
== APP ==   code: 1,
== APP ==   details: 'Cancelled on client',
== APP ==   metadata: Metadata { internalRepr: Map(0) {}, options: {} }
== APP == }

Steps to Reproduce the Problem

1. wget -q https://raw.githubusercontent.com/dapr/cli/master/install/install.sh -O - | DAPR_INSTALL_DIR="$HOME/.local/dapr" /bin/bash
2. dapr init --container-runtime podman
3. npm run start:dapr:activity-sequence (in examples)

Plausibly related: #221

CLI version: 1.13.0 
Runtime version: 1.13.2

[acheong08]

Simply removing await workflowRuntime.stop(); worked.

[acheong08]

This breaks tests because things aren't cleaned up properly

Could not process the event TASKSCHEDULED due to error NonDeterminismError: Failed to restore orchestration state due to a history mismatch: A previous execution called callActivity with ID=1, but the current execution is instead trying to call completeOrchestration as part of rebuilding it's history. This kind of mismatch can happen if an orchestration has non-deterministic logic or if the code was changed after an instance of this orchestration already started running.

[shubham1172]

@DeepanshuA can you help with this?

[dapr-bot]

This issue has been automatically marked as stale because it has not had activity in the last 60 days. It will be closed in the next 7 days unless it is tagged (pinned, good first issue, help wanted or triaged/resolved) or other activity occurs. Thank you for your contributions.

[paulyuk]

@acheong08 what about moving the stop the start loop like this, in a SIGTERM handler? this is working reliably for me now

process.on('SIGTERM', () => {
  workflowRuntime.stop();
}) 

please see the change I did for this in Quickstarts https://github.com/dapr/quickstarts/pull/1039/commits/74e2a59a5795de9f1b193aa52dadeb2d3e9be5c9