search

RADar-AZDelta / Rabbit-in-a-Blender

An ETL pipeline to transform your EMP data to OMOP.
https://radar-azdelta.github.io/Rabbit-in-a-Blender/
GNU General Public License v3.0
11 stars 3 forks source link

Parquet files in bucket not found due to backslash #97

Closed annabel-uzl closed 1 month ago

annabel-uzl commented 2 months ago

The files in the bucket cannot be found anymore due to use of backslash instead of forward slash. Temporary solution is to rename and move the files once they appear in the bucket (but the concepts will not be swapped)

image

pjlammertyn commented 2 months ago

Did you use the correct uri format for the bucket? 'gs://{bucket_name}/{bucket_path}'

ex: 

[bigquery]
bucket=gs://my_riab_bucket/upload
annabel-uzl commented 2 months ago

Yes: image

I did not have any problems previously and i did not change anything to my ini file. I do use Windows.

pjlammertyn commented 2 months ago

I can't simulate this problem. In the code, the blob name is also generated by using Pythons buildin Path.name function. This function doesn't generate a \ character.