TritonDataCenter / node-manta

Node.js SDK for Manta
75 stars 54 forks source link

client should detect if MPU is enabled #320

Closed jordanhendricks closed 7 years ago

jordanhendricks commented 7 years ago

If you use the client to try a multipart upload operation against a Manta that does not have MPU enabled, it reports a ResourceNotFoundError:

mmpu create ~~/stor/foo -v 2>&1 | bunyan                         
[2017-08-10T18:06:30.643Z] DEBUG: mmpu/MantaClient/3841 on joyadmins-MacBook-Pro-4.local (/opt/pkg/lib/node_modules/manta/lib/client.js:3352 in createUpload): createUpload: entered (req_id=03d952b4-dadc-4962-a73b-8eb7142dadc2, path=/jhendricks/uploads)
[2017-08-10T18:06:30.681Z] TRACE: mmpu/MantaClient/3841 on joyadmins-MacBook-Pro-4.local (/opt/pkg/lib/node_modules/manta/node_modules/restify-clients/lib/HttpClient.js:314 in rawRequest): request sent
    POST /jhendricks/uploads HTTP/1.1
    Host: us-east.manta.joyent.com
    accept: application/json
    content-type: application/json
    x-request-id: 03d952b4-dadc-4962-a73b-8eb7142dadc2
    date: Thu, 10 Aug 2017 18:06:30 GMT
    authorization: Signature keyId="/jhendricks/keys/15:4d:55:b0:e4:ae:82:ef:8e:e5:2d:08:2c:38:2c:90",algorithm="ecdsa-sha256",headers="date",signature="MEYCIQCYRuUcVX5+KNoxijHXq7AsBHHM+dTI7d9qHKShy/tZ4wIhALlGsGQ8ki8CFbp95lo8nnTdNERy6fzMf6id9jYTWpIZ"
    user-agent: restify/1.4.1 (x64-darwin; v8/3.28.71.19; OpenSSL/1.0.2j) node/0.12.17
    accept-version: ~1.0
    content-length: 70
    content-md5: z96N6HTBFeZjC3sH2pNy2g==
[2017-08-10T18:06:31.045Z] TRACE: mmpu/MantaClient/3841 on joyadmins-MacBook-Pro-4.local (/opt/pkg/lib/node_modules/manta/node_modules/restify-clients/lib/HttpClient.js:210 in onResponse): Response received
    HTTP/1.1 404 Not Found
    content-type: application/json
    content-length: 74
    content-md5: HMYqC45c5CPsn62YKkAJdQ==
    date: Thu, 10 Aug 2017 18:06:30 GMT
    connection: keep-alive
    x-request-received: 1502388390671
    x-request-processing-time: 374
[2017-08-10T18:06:31.049Z] TRACE: mmpu/MantaClient/3841 on joyadmins-MacBook-Pro-4.local (/opt/pkg/lib/node_modules/manta/node_modules/restify-clients/lib/StringClient.js:193 in done):
    body received:
    {"code":"ResourceNotFound","message":"/jhendricks/uploads does not exist"}
[2017-08-10T18:06:31.052Z] DEBUG: mmpu/MantaClient/3841 on joyadmins-MacBook-Pro-4.local (/opt/pkg/lib/node_modules/manta/lib/client.js:3370): /jhendricks/uploads does not exist (req_id=03d952b4-dadc-4962-a73b-8eb7142dadc2, path=/jhendricks/uploads)
    ResourceNotFoundError: /jhendricks/uploads does not exist
        at parseResponse (/opt/pkg/lib/node_modules/manta/node_modules/restify-clients/lib/JsonClient.js:73:26)
        at IncomingMessage.done (/opt/pkg/lib/node_modules/manta/node_modules/restify-clients/lib/StringClient.js:207:13)
        at IncomingMessage.g (events.js:199:16)
        at IncomingMessage.emit (events.js:129:20)
        at _stream_readable.js:908:16
        at process._tickCallback (node.js:355:11)
mmpu create: error: /jhendricks/uploads does not exist

The client can easily detect if the Manta it is connecting to has multipart upload enabled by querying for the /:account/uploads directory. If that directory does not exist, then MPU is not enabled. In this case, we should report a crisp error to the client indicating this is the case.

We should be able to preserve the statusCode of the error, but reporting a new type of error will break clients switching on the "ResourceNotFound" error code. This change will thus require a major version bump.

jordanhendricks commented 7 years ago

Adding this functionality will also make joyent/node-manta#319 simpler to solve.

jordanhendricks commented 7 years ago

CR at https://cr.joyent.us/#/c/2417/.

Testing Notes As part of the testing for #319, I ran the test suite both on a Manta with MPU enabled as well as one without MPU enabled. The output of the tests are in #319.

I also ran manual mmpu commands against a Manta without MPU enabled. Here is the output of those manual tests:

# create-mpu
$ bin/mmpu create ~~/stor/foo                                           
mmpu create: error: multipart upload is not supported for this Manta deployment

# upload-part
$ bin/mmpu upload 359d0a6c-0bbd-6b49-d976-9a589f999fc4 0 -f package.json
mmpu upload: error: multipart upload is not supported for this Manta deployment

# get-mpu
$ bin/mmpu get 359d0a6c-0bbd-6b49-d976-9a589f999fc4                     
mmpu get: error: multipart upload is not supported for this Manta deployment

# abort
$ bin/mmpu abort 359d0a6c-0bbd-6b49-d976-9a589f999fc4                   
mmpu abort: error: multipart upload is not supported for this Manta deployment

# commit
$ bin/mmpu commit 359d0a6c-0bbd-6b49-d976-9a589f999fc4
mmpu commit: error: multipart upload is not supported for this Manta deployment

Other commands in the mmpu CLI do not print this error message, as they are not implemented with MPU-related methods in the client library. We may want to create another issue to update mmpu for this purpose. For reference, here is the output of those commands:

# list-mpu
$ bin/mmpu list
mmpu list: error: 

# list-parts
$ bin/mmpu parts 359d0a6c-0bbd-6b49-d976-9a589f999fc4
mmpu parts: error: 
davepacheco commented 7 years ago

Those blank errors look like #198.