meltano / tap-smoke-test

Singer.io tap that can be used for use in functional/smoke tests. Migrated from GitLab: https://gitlab.com/meltano/tap-smoke-test
Apache License 2.0
2 stars 2 forks source link

meltano config tap-smoke-test set --interactive doesnt work well #95

Open jfo8001 opened 11 months ago

jfo8001 commented 11 months ago

I was trying to use this for some sample data and it was nearly impossible to choose a stream inside of the interactive configurator - I had to fall back to using meltano to copy paste.

meltano config tap-smoke-test set --interactive Choose option 10 (streams)

image

[ {"stream_name": "animals"}, {"input_filename": "https://raw.githubusercontent.com/meltano/tap-smoke-test/main/demo-data/animals-data.jsonl"}]

image

When you meltano invoke tap-smoke-test it blows chunks because there's an extra dash in front of input_filename in meltano.yml

In the end I had to scrape the config out of the getting started guide https://docs.meltano.com/cloud/getting_started/#new-to-meltano

Given this is in the getting started guide - perhaps some love and attention to the interactive config would be nice?

edgarrmondragon commented 11 months ago

Thanks for reporting @jfo8001!

I think the error here is the input format, since you're inputting two array elements each with one key, instead of a single element with two keys:

- [ {"stream_name": "animals"}, {"input_filename": "https://raw.githubusercontent.com/meltano/tap-smoke-test/main/demo-data/animals-data.jsonl"}]
+ [ {"stream_name": "animals", "input_filename": "https://raw.githubusercontent.com/meltano/tap-smoke-test/main/demo-data/animals-data.jsonl"}]

This produces what I think is the correct setting value:

extractors:
  - name: tap-smoke-test
    variant: meltano
    pip_url: git+https://github.com/meltano/tap-smoke-test.git
    config:
      streams:
        - stream_name: animals
          input_filename: https://raw.githubusercontent.com/meltano/tap-smoke-test/main/demo-data/animals-data.jsonl

The "Getting Started" guide could certainly be improved, PRs welcome!