search

apache / iceberg-python

Apache PyIceberg
https://py.iceberg.apache.org/
Apache License 2.0
314 stars 116 forks source link

Snowflke can't read tables generated by pyicberg ? #723

Open djouallah opened 1 month ago

djouallah commented 1 month ago

Question

I keep getting this error ?

Failed to read from Iceberg file 'abfss://xxxxxx.dfs.core.windows.net/data/iceberg_dwh/scada/metadata/snap-2728627078701324745-0-7c1d442e-7321-46f8-aa06-5d5f94cde607.avro': Failed to open file: abfss://xxxxxx.dfs.core.windows.net/data/iceberg_dwh/scada/metadata/snap-2728627078701324745-0-7c1d442e-7321-46f8-aa06-5d5f94cde607.avro

djouallah commented 1 month ago

i see, it is an upstream issue https://github.com/apache/iceberg/issues/10127

sfc-gh-dhuo commented 1 month ago

Normally the abfss URI is expected to look like this, according to azure docs:

abfs[s]://<file_system>@<account_name>.dfs.core.windows.net/<path>/<file_name>

In your example, did your files contain the <file_system> component of the URI? (Or what would be the "container" in Azure Blob syntax)

djouallah commented 1 month ago

the URI is 'abfss://account_name.dfs.core.windows.net/data/iceberg_dwh/scada/metadata/snap-2728627078701324745-0-7c1d442e-7321-46f8-aa06-5d5f94cde607.avro'