search

vmware-archive / pxf-field

Prototype PXF extensions for HAWQ
Apache License 2.0
36 stars 15 forks source link

JsonResolver class reports error no enum constant #1

Closed randomtask1155 closed 10 years ago

randomtask1155 commented 10 years ago

sample data: {"source":"web1"} {"source":"web2"}

hawq table definition: CREATE EXTERNAL TABLE ext_json ( source TEXT ) LOCATION ('pxf://pivhdsne:50070/testfile.json?FRAGMENTER=com.pivotal.pxf.plugins.hdfs.HdfsDataFragmenter&ACCESSOR=com.pivotal.pxf.plugins.json.JsonAccessor&RESOLVER=com.pivotal.pxf.plugins.json.JsonResolver&ANALYZER=com.pivotal.pxf.plugins.hdfs.HdfsAnalyzer') FORMAT 'CUSTOM' (FORMATTER='pxfwritable_import');

Hawq 1.2.0.1 error: ERROR: remote component error (500) from '192.168.220.129:50075': Problem accessing /gpdb/v10/Bridge/. Reason: Server Error Caused by: java.io.IOException: No enum constant com.pivotal.pxf.api.io.DataType.mysource (libchurl.c:753)

I was able to resolve this error after the following changes in the getFields() function call //DataType columnType = DataType.valueOf(cd.columnName()); DataType columnType = DataType.valueOf(cd.columnTypeName().toUpperCase());

ashook-pivotal commented 10 years ago

I take it this is PHD 2.0.1.0? If so, I haven't fully finished porting them to the new APIs and testing them. The stuff in the trunk is unstable at best. I'll work on it soon if not tomorrow. Let me know if it is urgent and I can get to it quicker.

randomtask1155 commented 10 years ago

Hi Ashook, yes this is phd 2.0.1.0 and latest pivhdsne VM. Actually it looks like the above fix does not work for INTEGER column type. Column descriptor converts integer to name "int4" and this breaks the json code.

I was able to get around this using the following fix. Call DataType.get from the column descriptor type code.

/com/pivotal/pxf/plugins/json/JsonResolver.java ColumnDescriptor cd = inputData.getColumn(i); DataType columnType = DataType.get(cd.columnTypeCode()); if ( columnType == DataType.UNSUPPORTED_TYPE ) { throw new IOException("Column Descriptor type is not supported typeName:" + cd.columnTypeName() + " index:" + cd.columnIndex() + " Name:" + cd.columnName() + " typeCode:" + cd.columnTypeCode()); }

ashook-pivotal commented 10 years ago

This should be resolved in the updated release for PHD 2.0.1.0. Thank you for bringing it up!