search

benwatson528 / intellij-avro-parquet-plugin

A Tool Window plugin for IntelliJ that displays Avro and Parquet files and their schemas in JSON.
Apache License 2.0
43 stars 9 forks source link

Working with INT96 #28

Closed pafelin closed 4 years ago

pafelin commented 4 years ago

Hi Ben,

Great job on making that plugin. I've received some files that i've tried opening and I get this error:

java.lang.IllegalArgumentException: INT96 not implemented and is deprecated
    at org.apache.parquet.avro.AvroSchemaConverter$1.convertINT96(AvroSchemaConverter.java:297)
    at org.apache.parquet.avro.AvroSchemaConverter$1.convertINT96(AvroSchemaConverter.java:282)
    at org.apache.parquet.schema.PrimitiveType$PrimitiveTypeName$7.convert(PrimitiveType.java:340)
    at org.apache.parquet.avro.AvroSchemaConverter.convertField(AvroSchemaConverter.java:281)
    at org.apache.parquet.avro.AvroSchemaConverter.convertFields(AvroSchemaConverter.java:259)
    at org.apache.parquet.avro.AvroSchemaConverter.convert(AvroSchemaConverter.java:248)
    at org.apache.parquet.avro.AvroReadSupport.prepareForRead(AvroReadSupport.java:130)
    at org.apache.parquet.hadoop.InternalParquetRecordReader.initialize(InternalParquetRecordReader.java:183)
    at org.apache.parquet.hadoop.ParquetReader.initReader(ParquetReader.java:156)
    at org.apache.parquet.hadoop.ParquetReader.read(ParquetReader.java:135)
    at uk.co.hadoopathome.intellij.viewer.fileformat.ParquetFileReader.getRecords(ParquetFileReader.java:40)
    at uk.co.hadoopathome.intellij.viewer.FileViewerToolWindow$2.doInBackground(FileViewerToolWindow.java:177)
    at uk.co.hadoopathome.intellij.viewer.FileViewerToolWindow$2.doInBackground(FileViewerToolWindow.java:168)
    at javax.swing.SwingWorker$1.call(SwingWorker.java:295)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at javax.swing.SwingWorker.run(SwingWorker.java:334)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

Do u think that's due to the older version of pycharm (PyCharm 2018.2.8) i have or due to the file's content ?

Best regards Pavel Genov

benwatson528 commented 4 years ago

Hi Pavel,

See https://github.com/benwatson528/intellij-avro-parquet-plugin/issues/19. I'm afraid this is out of my control.

Regards,

Ben

benwatson528 commented 4 years ago

Hi @pafelin, this is now supported.