Feature/UploadFile Service Implementation

[faraonc]

Description

As a user, I want to upload a file to Azure Blob Storage.

Story Points

8

Definitions of Done

• [x] Implementation (Uploading to Azure Blob Storage)
• [x] Refactor with service.py and utility.py
• [x] Please refer to the following links and examples on how to return an error. If in doubt, look at https://github.com/faraonc/hwsc-document-svc/blob/master/service/service.go
  • https://github.com/faraonc/hwsc-file-transaction-svc/blob/downloadzip/hwsc_file_transaction_svc_pb2_grpc.py
  • http://avi.im/grpc-errors/#python
• [ ] Unit Testing
• [ ] Integration Testing

Algorithms & Resources

• [OPTION 1 (PREFERRED)] Learn how to use createReadStream, link on how to pipe, link on how to set the chunk size. Code may look like the following:

  const fs = require('fs');
  const server = client.UploadFile....
  server.send({name : "some name"})
  const readStream = fs.createReadStream('myfile.txt'); //read the doc and learn how to use highWaterMark
  readStream
  .on('readable', function () {
  let chunk;
  while (null !== (chunk = readStream.read())) {
  //do something  like server.send({buffer :  chunk)
  }
  })
  .on('end', function () {
  //do something server.end() or is it close()?
  });
  .on('error', function () {
  //do something server.cancel()
  })

• [OPTION 2] Learn how to read a file in node.js in 1MB chunks; sample 1, sample 2, more npm library for reading file in chunks link.
• The algorithm for the UploadFile in the client should be the following:
  1. Send the name. If you use oneof, then you may have to check how to use it doc because of oneof you can only send 1 at a time! From the doc, unlike regular fields, at most one of the fields in a oneof can be set at a time, so setting one field will clear the others. Also note that if you are using proto3, the compiler generates has.. and clear.. accessors for oneof fields, even for scalar types.
  2. Open the file, and read/pipe the first 1024 bytes of the file. The ideal way is to pipe and to send the 1024 bytes, instead of reading before sending 1024 bytes. The reading is technically unnecessary.
  3. Set buffer=1st 1024 bytes.
  4. Send the buffer.
  5. Read/pipe and send the next 1024 bytes. Repeat step iii.
  6. Obviously and highly possible, the last chunk will not be 1024 bytes. We may or may NOT have to add extra logic for the last read/pipe and send.

[faraonc]

Please refer to https://github.com/faraonc/hwsc-gateway-svc/issues/43

[faraonc]

Error types.

• File not supported - INVALID_ARGUMENT
• Max is 32MB - INVALID_ARGUMENT
• Check for valid UUID using regex ^[[:digit:][:lower:]]{26}$ - INVLID_ARGUMENT
• What if the UUID does not exist in Azure Blob Storage? - INVALID_ARGUMENT
• Check for max length of file name in Azure Blob Storage, if max > 100, we go for 100, else we go with Azure's max. - INVALID_ARGUMENT
• Azure is down - INTERNAL
• What if something goes wrong with the USER? (Different size) - ABORTED