snowplow / igluctl

A command-line tool for working with Iglu schema registries
8 stars 6 forks source link

Use http4s #118

Closed lmath closed 2 years ago

lmath commented 2 years ago

scalaj-http is deprecated, so here we replace it with http4s, which is widely used elsewhere in other snowplow projects.

See https://github.com/snowplow-incubator/igluctl/issues/119

lmath commented 2 years ago

notes on testing

Currently, we have unit tests that test contained business logic introduced within igluctl, as is sensible. With this PR, I am not introducing any new functionality, and primarily need existing functionality to work. The existing unit tests work.

Still, I did run the commands that make http requests locally, and here are notes on that.

I had an iglu server up from when I was trying out the open source quick start that I have kept around until this latest igluctl release goes out, just because it has been handy for testing. That is: sp-lm-iglu-lb-195567083.us-east-1.elb.amazonaws.com

It knows about these schemas

igluctl push

igluctl static push /Users/leigh-anne/schemas sp-lm-iglu-lb-195567083.us-east-1.elb.amazonaws.com/api/schemas <apikey>

where /Users/leigh-anne/schemas has is a directory where I added a new schema iglu:com.test4/anonymous_ip/jsonschema/1-0-0 that I was able to see with this request

curl --location --request GET 'http://sp-lm-iglu-lb-195567083.us-east-1.elb.amazonaws.com/api/schemas' \
--header 'apikey: <key>'

output:

{"message":"Inconsistent schema availability. Cannot add private schema, previous versions are public"}
Schema created at iglu:com.test4/anonymous_ip/jsonschema/1-0-0  (201)
{"message":"Inconsistent schema availability. Cannot add private schema, previous versions are public"}
{"message":"Inconsistent schema availability. Cannot add private schema, previous versions are public"}
{"message":"Inconsistent schema availability. Cannot add private schema, previous versions are public"}
{"message":"Inconsistent schema availability. Cannot add private schema, previous versions are public"}

(seems sensible - I previously pushed the other 4 schemas as public)

igluctl pull

igluctl static pull /Users/leigh-anne/pull/schemas sp-lm-iglu-lb-195567083.us-east-1.elb.amazonaws.com <key> gave me the output I expected.

igluctl server keygen

compared running igluctl server keygen --vendor-prefix com.acme sp-lm-iglu-lb-195567083.us-east-1.elb.amazonaws.com <key> on this branch use-http4s and then used the output keys to read/write schemas with sample requests like the ones here.

Read key: <a uuid that allowed me to read>
Write key: <a uuid that allowed me to write>

and was able to fetch schemas with the read key, and write a schema with the write key.

igluctl table-check (the multiple one)

./igluctl rdbms table-check --server sp-lm-iglu-lb-195567083.us-east-1.elb.amazonaws.com --apikey <read api key> --host <aws_redshift_prod1 host> --dbname snowplow --username <user> --password <password>

I ran the table check command above on this branch use-http-4s and compared with master.

Both outputs were the same, ending in Unmatched: 0, Comment issues: 0, Matched: 6, Not Deployed: 364