addItemPart returning HTML

addy commented 1 year ago

Describe the bug

When calling the addItemPart function from @esri/arcgis-rest-portal, HTML is returned. If allowed to resort to its default response format of 'json' a JSON parsing issue will occur here: https://github.com/Esri/arcgis-rest-js/blob/805844b1686449aad0a286d2d2e397626cc388ea/packages/arcgis-rest-request/src/request.ts#L431

Returned HTML is in the following format:

<!DOCTYPE html>
<html>
  <head>
    <title>ArcGIS Portal Directory</title>
    <link href="/sharing/rest/files/gw.css" rel="stylesheet" type="text/css" />
  </head>
  <body>
    <div class="gwDiv">
      <table class="navTable" width="100%">
        <tr>
          <td class="breadcrumbs">ArcGIS Portal Directory</td>
        </tr>
      </table>
      <tr valign="top">
        <a href="http://resources.arcgis.com/en/help/arcgis-rest-api/"
          >API Reference</a
        >
      </tr>
    </div>
  </body>
</html>

REST calls are made in what we believe is the correct order:

createItem
addItemPart
commit

We have multipart set to true as well as async set to true.

We are using the async/await syntax, rather than callback form (for what that's worth).

We are using the ArcGISIdentityManager to perform authentication.

A sample of the workflow looks like this: https://gist.github.com/addy/ba7ce5b69d3c2f3677021af213880510

It's worth noting that we are not using a custom Portal, so the base API URL for these functions is https://arcgis.com/sharing/rest/content/users/{username}/items/{itemId}/<function>.

Reproduction

See the above Gist: https://gist.github.com/addy/ba7ce5b69d3c2f3677021af213880510

I can work on getting a Codepen set up tomorrow if needed. We are mostly wondering if we are missing something about Portal vs public accounts, or if the API has changed recently.

Logs

See above logs

System Info

System:
    OS: macOS 13.1
    CPU: (16) x64 Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
    Memory: 61.08 MB / 32.00 GB
    Shell: 5.8.1 - /bin/zsh
  Binaries:
    Node: 18.8.0 - ~/Library/Caches/fnm_multishells/50696_1675228496436/bin/node
    Yarn: 1.22.19 - /usr/local/bin/yarn
    npm: 8.18.0 - ~/Library/Caches/fnm_multishells/50696_1675228496436/bin/npm
  Browsers:
    Chrome: 109.0.5414.119
    Safari: 16.2
  npmPackages:
    @esri/arcgis-rest-feature-service: ^4.0.4 => 4.0.4
    @esri/arcgis-rest-portal: ^4.1.0 => 4.1.0
    @esri/arcgis-rest-request: ^4.1.1 => 4.1.1
    @esri/arcgis-rest-routing: ^4.0.3 => 4.0.3

Additional Information

No response

addy commented 1 year ago

I've got an update that this seems likely to be related to node 18 and its experimental native fetch implementation. Downgrading to node 16 was enough to get successes back from addItemPart and commit. Seems to be working now.

I imagine this hasn't been noticed much because of #1052 but could be due to other circumstances I'm not accounting for.

patrickarlt commented 1 year ago

@addy I think this is probably related to #1052 becuase we default to using global fetch if it is defined. In Node 18 there is a global fetch and it is no longer experimental. I think the best bet would be to continue to use Node 16 if possible while we work out how to cleanly switch between Node 18 native fetch and node-fetch for Node < 16.

gavinr commented 1 year ago

@addy I am attempting to replicate your issue. I am using the script that you posted: https://gist.github.com/addy/ba7ce5b69d3c2f3677021af213880510 ... Even if I use Node v16, I still get error ArcGISRequestError: 400: Unable to add part -

... is this the same issue you're reporting in this issue?

addy commented 1 year ago

@gavinr Sorry, I should have amended that after getting past the 400s, it's not 100% correct.

https://gist.github.com/addy/2f1723e51724cf0ca05f0f5eae6e86a4 Should get you around the 400 -- basically just switching to synchronous file Buffer types and wrapping them in a Blob vs asynchronous file streams.

You will still run into the node 18 issue, where downgrading to node 16 should fix the entire workflow.

gavinr commented 1 year ago

I am following your new code using Node v16, and getting an error on the addItemPart() call:

PS C:\Users\***\github\new-project> node .\index.js
item.id 9ac0b94afc0b465cbabded02d00a0607
node:internal/process/esm_loader:97
    internalBinding('errors').triggerUncaughtException(
                              ^

TypeError [ERR_INVALID_STATE]: Invalid state: chunk ArrayBuffer is zero-length or detached
    at new NodeError (node:internal/errors:387:5)
    at ReadableByteStreamController.enqueue (node:internal/webstreams/readablestream:1111:13)
    at Object.pull (file:///C:/Users/***/github/new-project/node_modules/fetch-blob/index.js:161:42) {
  code: 'ERR_INVALID_STATE'
}

Is this the same issue that you're reporting here?

Replication Steps

Here are my replication steps:

Install Node v16.19.1
mkdir new-project
cd new-project
npm init -y
Add "type": "module", to package.json.
npm install @esri/arcgis-rest-request @esri/arcgis-rest-portal
Create index.js and copy contents of this file to index.js.
Replace your username/password in the top of the file
Download this geojson file and save to test.geojson in the same directory.
run node index.js
- Expected: no error or same error as reported in original issue
- Actual: TypeError [ERR_INVALID_STATE]: Invalid state: chunk ArrayBuffer is zero-length or detached error

Esri / arcgis-rest-js