Getting an error in @trident/data-access-artifact

[josuelopesascent]

We are encountering an issue when running the github action.

It feels like something changed in an internal node package @trident/data-access-artifact, but I don't really understand what this is about.

Run Azure/Synapse-workspace-deployment@V1.9.0
Bundle source : https://web.azuresynapse.net/assets/cmd-api/main.js
Downloading asset file
Asset file downloaded at : /home/***/actions-runner2/_work/***/***/downloads/main.js
Starting export operation
Executing shell command
Command : node /home/***/actions-runner2/_work/***/***/downloads/main.js export "/home/***/actions-runner2/_work/***/***/synapse" synw-***-dev-gwc ExportedArtifacts
Stdout:  CmdApiApp: Initializing application.
Resource: synw-***-dev-gwc
RootFolder: /home/***/actions-runner2/_work/***/***/synapse
Stdout:  ModelService: synchronize - start
Stdout:  ModelService: Dynamic connector - Start registering dynamic connectors
Stdout:  DynamicConnectorService: Dynamic connector - Start registering connector: Dataworld
Stdout:  DynamicConnectorService: Dynamic connector - Finished registering connector: Dataworld
Stdout:  DynamicConnectorService: Dynamic connector - Start registering connector: Asana
Stdout:  DynamicConnectorService: Dynamic connector - Finished registering connector: Asana
 DynamicConnectorService: Dynamic connector - Start registering connector: Twilio
Stdout:  DynamicConnectorService: Dynamic connector - Finished registering connector: Twilio
Stdout:  DynamicConnectorService: Dynamic connector - Start registering connector: AppFigures
Stdout:  DynamicConnectorService: Dynamic connector - Finished registering connector: AppFigures
Stdout:  DynamicConnectorService: Dynamic connector - Start registering connector: GoogleSheets
Stdout:  DynamicConnectorService: Dynamic connector - Finished registering connector: GoogleSheets
Stdout:  LocalFileUtils: Reading pipeline files at : /home/***/actions-runner2/_work/***/***/synapse/pipeline
Stdout:  LocalFileUtils: Reading dataset files at : /home/***/actions-runner2/_work/***/***/synapse/dataset
Stdout:  WARNING === LocalFileUtils: Unable to read files for: dataset, error: Error: ENOENT: no such file or directory, scandir '/home/***/actions-runner2/_work/***/***/synapse/dataset'
Stdout:  LocalFileUtils: Saving file: /home/***/actions-runner2/_work/***/***/synapse/bigDataPool/sparkpool.json
Stdout:  LocalFileUtils: Dir: /home/***/actions-runner2/_work/***/***/synapse/bigDataPool does not exist, creating it....
Stdout:  LocalFileUtils: File saved.
Stdout:  CmdApiApp: Initializing resource registries...
Stdout:  CmdApiApp: Initializing model service...
Stdout:  ModelService: Dynamic connector - Finished registering dynamic connectors
Stdout:  ModelService: _createAndFetchEntities synchronizeInternal - start
 ModelService: _createAndFetchEntities synchronizeInternal - end
Stdout:  BaseFileResourceProviderService: populateAllResources - start
Stdout:  CmdApiApp: Initializing publish config service...
Stdout:  PublishConfigService: _getLatestPublishConfig - retrieving config file.
Stderr: /home/***/actions-runner2/_work/***/***/downloads/main.js:359820
          throw ex;
          ^
ReferenceError: Response is not defined
    at Module../node_modules/@trident/data-access-artifact/src/lib/models/http.js (/home/***/actions-runner2/_work/***/***/downloads/main.js:362608:40)
    at __webpack_require__ (/home/***/actions-runner2/_work/***/***/downloads/main.js:665765:42)
    at Module../node_modules/@trident/data-access-artifact/src/lib/models/index.js (/home/***/actions-runner2/_work/***/***/downloads/main.js:362698:63)
    at __webpack_require__ (/home/***/actions-runner2/_work/***/***/downloads/main.js:665765:42)
    at Module../node_modules/@trident/data-access-artifact/src/lib/clients/artifact-client.js (/home/***/actions-runner2/_work/***/***/downloads/main.js:361495:65)
    at __webpack_require__ (/home/***/actions-runner2/_work/***/***/downloads/main.js:665765:42)
    at Module../node_modules/@trident/data-access-artifact/src/lib/clients/index.js (/home/***/actions-runner2/_work/***/***/downloads/main.js:361976:74)
    at __webpack_require__ (/home/***/actions-runner2/_work/***/***/downloads/main.js:665765:42)
    at Module../node_modules/@trident/data-access-artifact/src/index.js (/home/***/actions-runner2/_work/***/***/downloads/main.js:361472:70)
    at __webpack_require__ (/home/***/actions-runner2/_work/***/***/downloads/main.js:665765:42)
Shell execution failed.
Action failed -> Error

[keen85]

We are seeing the same problem in Azure DevOps

[TechnologyHH340b]

We are also experiencing this issue and our builds are failing.

[josuelopesascent]

Changing to node version 18 solved our issues, hope it helps

[TechnologyHH340b]

Thanks for the tip @josuelopesascent - We were at 14.x and bumping to 18.x solved our issue.

[keen85]

Thanks @josuelopesascent for sharing your findings! It worked for us as well.

[keen85]

@josuelopesascent & @TechnologyHH340b , after upgrading to Node.js 18 the error at start of the workspace deployment does not show any more. However, we are experiencing frequent timeouts:

...
2024-06-18T13:56:03.0289222Z Getting Artifacts from workspace: xxx.
2024-06-18T13:56:10.7470512Z Found 217 artifacts in the workspace.
2024-06-18T13:56:10.7471534Z Getting Artifacts which should be deleted from workspace.
2024-06-18T13:56:10.7533018Z Found 0 artifacts in the workspace that many need to be deleted.
2024-06-18T13:56:10.7534986Z Computing dependancies for Artifacts which should be deleted from workspace.
2024-06-18T13:59:11.0058617Z Failed to fetch artifacts from workspace:  Error: Request timeout: /databases?api-version=2019-06-01-preview
2024-06-18T13:59:11.0059194Z     at ClientRequest.req.setTimeout (C:\xxx\_work\_tasks\Synapse workspace deployment_xxx\2.3.0\node_modules\typed-rest-client\HttpClient.js:324:26)
2024-06-18T13:59:11.0059865Z     at Object.onceWrapper (events.js:286:20)
2024-06-18T13:59:11.0060187Z     at ClientRequest.emit (events.js:198:13)
2024-06-18T13:59:11.0060460Z     at TLSSocket.emitRequestTimeout (_http_client.js:673:40)
2024-06-18T13:59:11.0060725Z     at Object.onceWrapper (events.js:286:20)
2024-06-18T13:59:11.0060979Z     at TLSSocket.emit (events.js:198:13)
2024-06-18T13:59:11.0061248Z     at TLSSocket.Socket._onTimeout (net.js:443:8)
2024-06-18T13:59:11.0061498Z     at ontimeout (timers.js:436:11)
2024-06-18T13:59:11.0061720Z     at tryOnTimeout (timers.js:300:5)
2024-06-18T13:59:11.0061951Z     at listOnTimeout (timers.js:263:5)
2024-06-18T13:59:11.0062227Z deploy operation failed
2024-06-18T13:59:11.0062433Z An error occurred during execution: Failed

Are you facing the same issue? We use self-hosted agents.

[josuelopesascent]

@keen85 We are also using self-hosted agents, but we haven't seen those timeouts. Should point out that changing to node 18 is not the solution, it's more like a workaround...

[lordozb]

It appears that this issue is happening for self hosted agents running on Node 14. We had already migrated to Node 16 last year. If running on GitHub's provided pool I am unable to reproduce this issue.

Can anyone here confirm if they are only seeing this issue on self-hosted agents?

[SpaceJacks]

We used Node 16 and had the same issue, but Node 18 did the work.

[morgan-jones-kr]

On Node 18 when I do a deploy operation I get this error

Bundle source : https://web.azuresynapse.net/assets/cmd-api/main.js Starting deploy operation Able to fetch service principal token: 200; status message: OK Able to fetch location of workspace: 200; status message: OK Able to fetch service principal token: 200; status message: OK /__w/_actions/Azure/Synapse-workspace-deployment/V1.9.0/dist/index.js:10332 var error = new Error('tunneling socket could not be established, ' + ^ Error: tunneling socket could not be established, statusCode=407 at ClientRequest.onConnect (/__w/_actions/Azure/Synapse-workspace-deployment/V1.9.0/dist/index.js:10332:19) at Object.onceWrapper (node:events:6[28](https://github.com/krogertechnology/fst-falconforce-az-synapse-studio/actions/runs/10309613012/job/28539616955#step:8:28):26) at ClientRequest.emit (node:events:513:28) at Socket.socketOnData (node:_http_client:574:11) at Socket.emit (node:events:513:28) at addChunk (node:internal/streams/readable:[31](https://github.com/krogertechnology/fst-falconforce-az-synapse-studio/actions/runs/10309613012/job/28539616955#step:8:31)5:12) at readableAddChunk (node:internal/streams/readable:289:9) at Socket.Readable.push (node:internal/streams/readable:228:10) at TCP.onStreamRead (node:internal/stream_base_commons:190:23) { code: 'ECONNRESET' }