dataverbinders / statline-bq

Library to fetch CBS open datasets into parquet and optionally load into Google Cloud Storage and BigQuery
MIT License
0 stars 0 forks source link

Acceptance testing #35

Closed dkapitan closed 3 years ago

dkapitan commented 3 years ago

@galamit86

I have started to do some acceptance testing. Running statline-bq from main branch with config as-is yields the following error:

Traceback (most recent call last):
  File "cli.py", line 57, in <module>
    upload_datasets()
  File "/Users/dkapitan/.pyenv/versions/3.8.5/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/Users/dkapitan/.pyenv/versions/3.8.5/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/Users/dkapitan/.pyenv/versions/3.8.5/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/dkapitan/.pyenv/versions/3.8.5/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "cli.py", line 53, in upload_datasets
    main(id=id, config=config, gcp_env=gcp_env)
  File "/Users/dkapitan/.pyenv/versions/3.8.5/lib/python3.8/site-packages/statline_bq-0.1.0-py3.8.egg/statline_bq/utils.py", line 1482, in main
    cbsodata_to_gbq(
  File "/Users/dkapitan/.pyenv/versions/3.8.5/lib/python3.8/site-packages/statline_bq-0.1.0-py3.8.egg/statline_bq/utils.py", line 1006, in cbsodata_to_gbq
    desc_dict = get_col_descs_from_gcs(
TypeError: get_col_descs_from_gcs() got an unexpected keyword argument 'gcp'

Should I run a different branch?

galamit86 commented 3 years ago

@dkapitan Not sure when I messed it up, but main should have worked, of course.

It's a small issue, handing config + gcp_env instead of gcp. PR #36 fixes it, and it should run properly.

dkapitan commented 3 years ago

@galamit86

So, I have setup a completely new working laptop:

If I git clone this repo, do poetry install and then .../statline_bq/statline-bq I get an error:

zsh: command not found: statline-bq

I have never used poetry before to actually install the commandline scripts. Am I missing something?

dkapitan commented 3 years ago

@galamit86

Never mind, me being a noob. Of course it should be:

poetry run statline-bq

Will update README.md immediately