Files uploaded using FormData placed incorrectly within collection.

[pythonicode]

UPDATE

I'm getting the expected behavior when running estuary locally and making requests to localhost:3004 from my app, but when I make requests to api.estuary.tech I'm getting the extra directory issue.

The only difference I can tell between the requests is that the request to api.estuary.tech is using an perms=upload API key so I will try testing to see if that makes any difference.

This is also reproducible behavior if you use the Estuary docs.

ISSUE

Describe the bug When uploading files using FormData to /content/add?coluuid=[coluuid]&overwrite=true&dir=/ causes to files not to exist in the root directory / but instead under a subdirectory with the file name for example under /[file name].

To Reproduce Upload a file using FormData and dir=/ and coluuid=[coluuid] then check result of /collection/:coluuid.

Expected behavior Uploaded file should be under root directory as specified by search param dir

Actual behavior Uploaded file is under a subdirectory with the same name as the file.

Additional context Slack discussion: https://filecoinproject.slack.com/archives/C016APFREQK/p1677178196959089

[pythonicode]

Testing this locally using a cloned version of this repository and I'm not getting the same issue. Will try to reproduce locally.

UPDATE I'm getting the expected behavior when running estuary locally and making requests to localhost:3004 from my app, but when I make requests to api.estuary.tech I'm getting the extra directory issue.

The only difference I can tell between the requests is that the request to api.estuary.tech is using an perms=upload API key so I will try testing to see if that makes any difference.

[pythonicode]

EDIT: I now realize that I should be testing using the shuttles. Having issues building, but I'm sure the issue can be found somewhere in cmd/estuary-shuttle

I tried helplessly to recreate the issue on localhost with no luck (this is using the most recent main branch of estuary). Not sure why when using the actual endpoint I'm getting the extra directory, here are the exact same requests and the results of each:

const response = await fetch(`https://api.estuary.tech/content/add?coluuid=${user.collectionId}&overwrite=true&dir=/`, {
    method: 'POST',
    body: form,
    headers: {
        'Accept': 'application/json',
        'Authorization': `Bearer ${user.apiKey}`
    }
});

[
  {
    id: 142858874,
    createdAt: '2023-02-24T19:55:39.613238Z',
    updatedAt: '2023-02-24T19:55:39.613238Z',
    cid: 'bafkreid7dky6z7x4nr7vqbwnqn2roop76nfpxcpjv5qk3giahwbuq5bpqu',
    name: 'subfile.png',
    userId: 341,
    description: '',
    size: 8848,
    type: 0,
    active: true,
    offloaded: false,
    replication: 6,
    aggregatedIn: 0,
    aggregate: false,
    pinning: true,
    pinMeta: '',
    replace: false,
    origins: '',
    failed: false,
    location: 'SHUTTLEdd6d5754-eb26-439a-a753-7d6e824325e7HANDLE',
    dagSplit: false,
    splitFrom: 0,
    pinningStatus: '',
    dealStatus: '',
    path: '/subfile.png/subfile.png'
  }
]

const response = await fetch(`http://localhost:3004/content/add?coluuid=685cd56d-93d3-4f61-89ee-e5ea036d2bd7&overwrite=true&dir=/`, {
    method: 'POST',
    body: form,
    headers: {
        'Accept': 'application/json',
        'Authorization': `Bearer {API key with perms=upload}`
    }
});

[
  {
    "id": 6,
    "createdAt": "2023-02-25T03:28:18.311279386Z",
    "updatedAt": "2023-02-25T03:28:18.311279386Z",
    "cid": "bafkreiehzremeji2voxc4mxsoqw4pry736s6z2t5nh23win272fo3rert4",
    "name": "subfile.png",
    "userId": 1,
    "description": "",
    "size": 8848,
    "type": 0,
    "active": true,
    "offloaded": false,
    "replication": 6,
    "aggregatedIn": 0,
    "aggregate": false,
    "pinning": false,
    "pinMeta": "",
    "replace": false,
    "origins": "[{\"ID\":\"12D3KooWCNSYCRFnPtf4WSdB2oXKJhkStPSiW9NdEthGWgKJLLBG\",\"Addrs\":[\"/ip4/10.129.11.192/tcp/6744\"]},{\"ID\":\"12D3KooWCNSYCRFnPtf4WSdB2oXKJhkStPSiW9NdEthGWgKJLLBG\",\"Addrs\":[\"/ip4/127.0.0.1/tcp/6744\"]},{\"ID\":\"12D3KooWCNSYCRFnPtf4WSdB2oXKJhkStPSiW9NdEthGWgKJLLBG\",\"Addrs\":[\"/ip4/128.12.123.59/tcp/54531\"]},{\"ID\":\"12D3KooWCNSYCRFnPtf4WSdB2oXKJhkStPSiW9NdEthGWgKJLLBG\",\"Addrs\":[\"/ip4/128.12.123.59/tcp/54639\"]}]",
    "failed": false,
    "location": "local",
    "dagSplit": false,
    "splitFrom": 0,
    "pinningStatus": "",
    "dealStatus": "",
    "path": "/subfile.png"
  }
]

The only difference I see is with the location being local vs. shuttle and of course the path is screwed up for the first one, @gmelodie I know you did some work with collections, do you have any idea what's going on?