Closed 360d-io-labs closed 4 years ago
Could you share the SHOW CREATE TABLE result for elli_csv_presto
on both Hive and Presto?
I cannot do a SHOW CREATE TABLE from Presto (that's why I added the vision from Hive) for all text/csv tables (either created with Hive or Presto). It is generating the same exception. Though I can try to do it from Presto-314 if you want.
@360d-io-labs this is backwards-incompatibility issue on the metastore side.
The reason why it broke in your environment is that we added first-class (and correct) support for CSV tables in 316.
Before that version you could have all sorts of problems when CSV table was declared with types different than varchar
(hive allows that).
Can you add this to your Hive metastore configuration?
<property>
<!-- https://community.hortonworks.com/content/supportkb/247055/errorjavalangunsupportedoperationexception-storage.html -->
<name>metastore.storage.schema.reader.impl</name>
<value>org.apache.hadoop.hive.metastore.SerDeStorageSchemaReader</value>
</property>
(i added a note to https://github.com/prestosql/presto/issues/1841 too)
@findepi Thanks it works! I think that this should be added to the documentation regarding hive connector to make it clear as this is not the default configuration in HDP and other people may face the same problem (same for the default ACID, which let Presto write to the table but not read).
For anyone finding this issue, the solution is documented here:
https://prestosql.io/docs/current/connector/hive.html#metastore-configuration-for-avro-and-csv
I am having the similar issue with access / describing the tables created from a CSV File that is accessed using s3a.
Even after the table being created successfully, I get the localhost:9083 null error. I do not want to create it using Hive.
Is there a solution / workaround for this issue ?
After upgrading the cluster to 328/329 from 314, it is impossible to query CSV table stored on HDFS with Hive 3.1.0
Here is the exception raised:
I have no problem to do the same query with Presto 314 (the previous version installed on this cluster).
I tried to create an external table with Presto on the same data and the table is successfully created in Hive. However if I tried to query this presto created table I also get this exception.
from Hive (show create table):
I have no problem to access all the ORC tables.