Powerspace / pg2bq

Export PostgreSQL tables to Google BigQuery
MIT License
37 stars 11 forks source link

WrongType & KeyNotFound Errors #4

Open ZonerDan opened 6 years ago

ZonerDan commented 6 years ago

I have modified my configuration file in multiple ways and I continue to get one of the following errors regardless of the changes I make:

When using JSON format:

[main] ERROR com.powerspace.pg2bq.Main$ - Incorrect configuration: KeyNotFound(jdbc,None,Set("jdbc")), KeyNotFound(gcloud,None,Set("gcloud"))

When using HOCON format that matches the example on your README:

[main] ERROR com.powerspace.pg2bq.Main$ - Incorrect configuration: WrongType(STRING,Set(OBJECT),None,jdbc), WrongType(STRING,Set(OBJECT),None,gcloud)

sderosiaux commented 6 years ago

Hi @ZonerDan,

Can you paste the content of your configuration file? It seems you're passing a string to jdbc instead of an object jdbc { ... }, same for gcloud. Or some odd character is present?

ZonerDan commented 6 years ago

JSON Version:

{ "jdbc": { "url": "", "user": "", "password": "", "tables": ["people"] }, "gcloud": { "project": "", "service-account-key-path": "", "bq.dataset": "", "gcs.tmp-bucket": "pg-export" } }

HOCON Version:

jdbc { url = "" user = "" password = "" tables = ["people"] }

gcloud { project = "" service-account-key-path = "" bq.dataset = "" gcs.tmp-bucket = "pg-export" }

ZonerDan commented 6 years ago

Other things I have tried:

ZonerDan commented 6 years ago

Provided in comments

sderosiaux commented 6 years ago

I just copy/pasted your HOCON (with all the placeholders ), the program starts:

/bin/pg2bq -Dconfig.file=/Users/sderosiaux/pg2bq.conf
2018-02-20 16:34:04,336  [main] ...
ZonerDan commented 6 years ago

:( I'm at a loss. I just tried with all the placeholders as well and still receive the WrongType error.

ZonerDan commented 6 years ago

Ok. I think its because in my original testing I received an error stating that I should change the file extension to .properties instead of .conf. Now I've changed it back and the program is starting.

sderosiaux commented 6 years ago

Yeah, I think when the file ends by .properties, the config expects to find the Java properties format. Happy you got it fixed!

ZonerDan commented 6 years ago

Thanks for your help!