Closed sananguliyev closed 4 years ago
Hi Sanan,
The problem isn't with lists as I have successful unit tests and sample files containing lists. It looks like this Parquet file was made with parquet-go, and that the Parquet implementation in that project creates a file that isn't compatible with the org.apache.parquet:parquet-avro:1.11.0
library that this plugin uses to read Parquet files.
Avro has some strict rules for schema naming, and the parquet-go
library seems to introduce a field called parquet-go-root
that breaks these rules. Sadly I can't fix this issue, but you may want to raise an issue with the parquet-go
library, or check the way that this file is created.
Thank you very much for the quick reply and this amazing plugin.
P.S. I will check the differences but it works fine when I create a parquet file without a list. Most probably it's not because of the parquet-go-root
field.
No problem, thanks for giving a sample file and stack trace, that makes everything a lot easier. Let me know if you ever get anywhere with this or if I can help.
Just saw your edit - maybe it's something to do with how parquet-go
specifically handles lists?
I still do not know why the old parquet files I created without a list field but the main problem is -
. The library org.apache.parquet:parquet-avro:1.11.0
counts it as an illegal character. I do not know whether it is parquet rules but I will make pull request to the go library if it's.
Anyway, thanks again for noticing my edit and reply :)
Hi Ben,
The plugin throw and error when I try to view the parquet file with an array field. You can download the parquet file if you need to reproduce the case.
File: https://www.transfernow.net/grbC2h032020 IDE: GoLand 2019.3 Plugin version: 1.1.1 Error: