Allow specifying custom metadata when uploading Google Cloud Storage files

[jannisy]

The resumableUpload in the Google Cloud Storage connector allows specifying metadata of type Map[String, String] which are added to the uploaded file. As per the GCS documentation, metadata properties like contentType or cacheControl are predefined by GCS.

However, there is also custom metadata, which is arbitrary user-provided key-value pairs. They are sent in the HttpRequest in the same way, but the value is not a string but a json object:

{
    // metadata
    "contentType": "...",
    // custom metadata
    "metadata": {
        "customKey": "customData"
    }
}

However this cannot be modelled currently because metadata is Map[String, String] - the value needs to be specified as Map[String, String] instead of String to handle this.

(According to the documentatoin, GCS should also allow the notation metadata.customKey = customData but this did not work for me when I tried)

[pjfanning]

If the GCS docs say that metadata.customKey is a valid key, I think we really need to track down why that doesn't work.

It is not trivial for us to change the connector to modify the metadata support.

[jannisy]

Yes, makes sense. I will check again if I can find out what's the issue.