search

snowplow / igluctl

A command-line tool for working with Iglu schema registries
8 stars 6 forks source link

Set content type to appplication/json when pushing schemas to S3 (clo… #108

Closed lmath closed 2 years ago

lmath commented 2 years ago

See: https://github.com/snowplow-incubator/igluctl/issues/87

static s3cp sends json schema to an s3 bucket. When you upload objects to S3, there is associated metadata. Since igluctl static s3cp will only ever send json, in this PR we ensure we are setting ContentType to application/json.

To test this code change I compared what happens currently with what happens with the change in this PR.

Currently

I made a bucket called lm-s3cp-test-bucket-deleteme via the aws console

I sent a schema to the bucket using s3cp via the igluctl executable I downloaded here

./igluctl static s3cp /Users/leigh-anne/code/iglu-central/schemas/com.maxmind/anonymous_ip/jsonschema/1-0-0 lm-s3cp-test-bucket-deleteme --region us-east-1 --profile sandbox

I can look at the schema's metadata:

leigh-anne@Leigh-Annes-MacBook-Pro iglu-central % aws s3api head-object --bucket lm-s3cp-test-bucket-deleteme --key schemas/com.maxmind/anonymous_ip/jsonschema/1-0-0 

{
    "AcceptRanges": "bytes",
    "LastModified": "2022-05-12T09:36:37+00:00",
    "ContentLength": 1114,
    "ETag": "\"987f15dfe31b34455516999bcb55bd75\"",
    "ContentType": "text/plain; charset=UTF-8",
    "Metadata": {}
}

And see that it is set to text/plain

with the changes in this PR

I set up a run configuration in my IDE to run the scala main method and passed these args static s3cp /Users/leigh-anne/code/iglu-central/schemas/com.pagerduty/incident/jsonschema/1-0-0 lm-s3cp-test-bucket-deleteme --region us-east-1 --profile sandbox i.e. sent a different schema to compare image

and then if I fetch that object's metadata:

aws s3api head-object --bucket lm-s3cp-test-bucket-deleteme --key schemas/com.pagerduty/incident/jsonschema/1-0-0  
{
    "AcceptRanges": "bytes",
    "LastModified": "2022-05-12T12:49:03+00:00",
    "ContentLength": 2385,
    "ETag": "\"6ff147b21df0ab3832f0c2380e387f64\"",
    "ContentType": "application/json; charset=utf-8",
    "Metadata": {}
}

I can see that it has ContentType as application/json; charset=utf-8 as we expect