search

Eugene-Mark / bigdata-file-viewer

A cross-platform (Windows, MAC, Linux) desktop application to view common bigdata binary format like Parquet, ORC, AVRO, etc. Support local file system, HDFS, AWS S3, Azure Blob Storage ,etc.
GNU General Public License v2.0
287 stars 54 forks source link

Error while loading Parquet local file #29

Closed javiruiz closed 2 years ago

javiruiz commented 2 years ago

Bigdata-file-viewer produces an exception [see below] while loading a Parquet local file (downloaded from a Databricks tutorial). I am using version 1.3 in Windows 10. Other Parquet files exported from BigQuery load just fine.

part-00000-tid-5817126155423616578-c942211a-3073-4a28-a76d-8abea3bc69c3-9-1.c000.snappy.parquet.zip

java.lang.IllegalArgumentException: INT96 is deprecated. As interim enable READ_INT96_AS_FIXED flag to read as byte array. at org.apache.parquet.avro.AvroSchemaConverter$1.convertINT96(AvroSchemaConverter.java:316) at org.apache.parquet.avro.AvroSchemaConverter$1.convertINT96(AvroSchemaConverter.java:298) at org.apache.parquet.schema.PrimitiveType$PrimitiveTypeName$7.convert(PrimitiveType.java:341) at org.apache.parquet.avro.AvroSchemaConverter.convertField(AvroSchemaConverter.java:297) at org.apache.parquet.avro.AvroSchemaConverter.convertFields(AvroSchemaConverter.java:275) at org.apache.parquet.avro.AvroSchemaConverter.convert(AvroSchemaConverter.java:264) at org.apache.parquet.avro.AvroReadSupport.prepareForRead(AvroReadSupport.java:134) at org.apache.parquet.hadoop.InternalParquetRecordReader.initialize(InternalParquetRecordReader.java:185) at org.apache.parquet.hadoop.ParquetReader.initReader(ParquetReader.java:156) at org.apache.parquet.hadoop.ParquetReader.read(ParquetReader.java:135) at org.eugene.core.parquet.ParquetReader.read(ParquetReader.java:26) at org.eugene.controller.ParquetDataParser.parseData(ParquetDataParser.java:18) at org.eugene.controller.Renderer.load(Renderer.java:55) at org.eugene.controller.Renderer.loadAndShow(Renderer.java:91) at org.eugene.ui.CustomizedMenuBar.lambda$new$0(CustomizedMenuBar.java:41) at com.sun.javafx.event.CompositeEventHandler.dispatchBubblingEvent(CompositeEventHandler.java:86) at com.sun.javafx.event.EventHandlerManager.dispatchBubblingEvent(EventHandlerManager.java:238) at com.sun.javafx.event.EventHandlerManager.dispatchBubblingEvent(EventHandlerManager.java:191) at com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(BasicEventDispatcher.java:58) at com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(EventDispatchChainImpl.java:114) at com.sun.javafx.event.EventUtil.fireEventImpl(EventUtil.java:74) at com.sun.javafx.event.EventUtil.fireEvent(EventUtil.java:49) at javafx.event.Event.fireEvent(Event.java:198) at javafx.scene.control.MenuItem.fire(MenuItem.java:465) at com.sun.javafx.scene.control.ContextMenuContent$MenuItemContainer.doSelect(ContextMenuContent.java:1380) at com.sun.javafx.scene.control.ContextMenuContent$MenuItemContainer.lambda$createChildren$12(ContextMenuContent.java:1333) at com.sun.javafx.event.CompositeEventHandler$NormalEventHandlerRecord.handleBubblingEvent(CompositeEventHandler.java:218) at com.sun.javafx.event.CompositeEventHandler.dispatchBubblingEvent(CompositeEventHandler.java:80) at com.sun.javafx.event.EventHandlerManager.dispatchBubblingEvent(EventHandlerManager.java:238) at com.sun.javafx.event.EventHandlerManager.dispatchBubblingEvent(EventHandlerManager.java:191) at com.sun.javafx.event.CompositeEventDispatcher.dispatchBubblingEvent(CompositeEventDispatcher.java:59) at com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(BasicEventDispatcher.java:58) at com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(EventDispatchChainImpl.java:114) at com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(BasicEventDispatcher.java:56) at com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(EventDispatchChainImpl.java:114) at com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(BasicEventDispatcher.java:56) at com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(EventDispatchChainImpl.java:114) at com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(BasicEventDispatcher.java:56) at com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(EventDispatchChainImpl.java:114) at com.sun.javafx.event.EventUtil.fireEventImpl(EventUtil.java:74) at com.sun.javafx.event.EventUtil.fireEvent(EventUtil.java:54) at javafx.event.Event.fireEvent(Event.java:198) at javafx.scene.Scene$MouseHandler.process(Scene.java:3851) at javafx.scene.Scene$MouseHandler.access$1200(Scene.java:3579) at javafx.scene.Scene.processMouseEvent(Scene.java:1849) at javafx.scene.Scene$ScenePeerListener.mouseEvent(Scene.java:2588) at com.sun.javafx.tk.quantum.GlassViewEventHandler$MouseEventNotification.run(GlassViewEventHandler.java:397) at com.sun.javafx.tk.quantum.GlassViewEventHandler$MouseEventNotification.run(GlassViewEventHandler.java:295) at java.base/java.security.AccessController.doPrivileged(Native Method) at com.sun.javafx.tk.quantum.GlassViewEventHandler.lambda$handleMouseEvent$2(GlassViewEventHandler.java:434) at com.sun.javafx.tk.quantum.QuantumToolkit.runWithoutRenderLock(QuantumToolkit.java:390) at com.sun.javafx.tk.quantum.GlassViewEventHandler.handleMouseEvent(GlassViewEventHandler.java:433) at com.sun.glass.ui.View.handleMouseEvent(View.java:556) at com.sun.glass.ui.View.notifyMouse(View.java:942) at com.sun.glass.ui.win.WinApplication._runLoop(Native Method) at com.sun.glass.ui.win.WinApplication.lambda$runLoop$3(WinApplication.java:174) at java.base/java.lang.Thread.run(Thread.java:829)

Eugene-Mark commented 2 years ago

@javiruiz Thanks for your feedback. The Int96 is not supported any more per this thread. So that this tool can't read parquet file with INT96 data type.