search

snowplow / kinesis-tee

Unix tee, but for Kinesis streams
http://snowplowanalytics.com/
12 stars 5 forks source link

Validate the schema URI in the configuration file #17

Open alexanderdean opened 8 years ago

alexanderdean commented 8 years ago

was: iglu: URIs in documentation are invalid

Looking at https://github.com/snowplow/kinesis-tee/wiki/Guide-for-devops-users :

{
  "schema": "iglu:com.snowplowanalytics.kinesistee.config.Configuration",
  "data": {
    "name": "My Kinesis Tee example",
    ...

That's not a valid Iglu URI - it should be:

{
  "schema": "iglu:com.snowplowanalytics.kinesistee.config/Configuration/avro/1-0-0",
  "data": {
    "name": "My Kinesis Tee example",
    ...

Creating this as a ticket in case the wiki page is indicative of a bug in the underlying implementation...

/cc @chuwy as he has been doing a lot with self-describing Avro in Sauna recently...

ninjabear commented 8 years ago

Fixed the wiki. Kinesis Tee currently doesn't check this field at all, on the premise that:

1) it's very strongly implied already 2) if the payload validates, it doesn't really matter what envelope it's in

I don't think it'd give us much right now adding the check, which is why it's not in there already. Happy to discuss!

alexanderdean commented 8 years ago

Thanks @ninjabear! I'll rename the ticket and deschedule, agree it's not urgent. I got some interesting feedback from @chuwy on using self-describing Avro for configs earlier today...