Silently ignores partly failing uploads, providing wrong results

[TripleSpeeder]

After experiencieng issues with infura hashes not matching the hashes from other pinners i looked a bit deeper into the issue to understand what is happening.

Test case is the following directory structure:

public/:
root.txt  sub

public/sub:
sub1.txt  sub2.txt  subsub

public/sub/subsub:
subsub1.txt  subsub2.txt

Then i tried to upload with infura and dappnode pinner, and check the response from api.addFromFs in pinners/infura.js and pinners/dappnode.js (via debugger):

Response from dappnode pinner:

[ { path: 'public/root.txt',
    hash: 'QmQJRuzSnVj3ZtyTkuLqdhBL7QBzpDMwUSsuX2ZvvZqK9X',
    size: 34 },
  { path: 'public/sub/sub1.txt',
    hash: 'QmYXAWSYoyDVAwGuUe5X9B4F9e4r1G8E66meVk12U6MZSQ',
    size: 44 },
  { path: 'public/sub/sub2.txt',
    hash: 'Qma8NBewQpX2GZafD71zYUjVX7Ve5LZ44bYaEcTCHYY3hE',
    size: 44 },
  { path: 'public/sub/subsub/subsub1.txt',
    hash: 'QmZxES63SLnndw2vASMwYpXggqpUJu1wNhbNLmMti1GGq1',
    size: 53 },
  { path: 'public/sub/subsub/subsub2.txt',
    hash: 'Qma9Y3hHFb7Xt1kYoLZ6emtGRZym8wooTHokJ9NYhD2DDN',
    size: 53 },
  { path: 'public/sub/subsub',
    hash: 'QmUnRVEPAPnsm3cTWLy6cA9RrgdHajSbuuVaJb8WRpvyXr',
    size: 216 },
  { path: 'public/sub',
    hash: 'Qmf9EDMfav9WBmM5reng5H4igoMnkFJP5eiN4pTUD12hV5',
    size: 457 },
  { path: 'public',
    hash: 'QmWJJU8G7SDBYvu3dYZ9qdDHNgLkbQLGwAREEx3Djc5LFd',
    size: 591 } ]

Response from infura pinner:

[ { path: 'public/root.txt',
    hash: 'QmQJRuzSnVj3ZtyTkuLqdhBL7QBzpDMwUSsuX2ZvvZqK9X',
    size: 34 } ]

Obviously, the responses are different. Dappnode response object contains all files, with the root folder element being the last element. Infura response object contains just one (random?) file from the directory.

While the root cause of this issue might be instability of Infura service (it also took about 20 seconds to get a response from infura API at all, so they clearly had problems at this time), this error needs to result in ipfs-deploy exiting with an error instead of providing wrong results.

In only had a brief look so far at the implementation, but I'm wondering if this needs to be investigated more inside ipfs-http-client? After all we're just using their "addFromFs" API.

Edit/addendum: Looking at https://github.com/ipfs/interface-js-ipfs-core/blob/master/SPEC/FILES.md#addfromfs there is no mention of a specifc ordering of the results. So the current approach of assuming that the last array element is the root node seems quite dangerous.

[TripleSpeeder]

I think this is also the root cause or at least related to issues #102 and #55.

[tobowers]

I'm seeing this also. It can give you random results from upload.