search

treasure-data / embulk-input-td

Treasure Data Input Plugin for Embulk
8 stars 6 forks source link

ARRAY(varchar) in Presto is not supported #16

Closed toru-takahashi closed 6 years ago

toru-takahashi commented 6 years ago

Version: embulk-input-td 0.2.0

When Presto returns Array(varchar) type, the following error happens.

org.embulk.exec.PartialExecutionException: org.embulk.config.ConfigException: Unsupported presto type 'array(varchar)'
    at org.embulk.exec.BulkLoader$LoaderState.buildPartialExecuteException(BulkLoader.java:375)
    at org.embulk.exec.BulkLoader.doRun(BulkLoader.java:607)
    at org.embulk.exec.BulkLoader.access$000(BulkLoader.java:35)
    at org.embulk.exec.BulkLoader$1.run(BulkLoader.java:391)
    at org.embulk.exec.BulkLoader$1.run(BulkLoader.java:387)
    at org.embulk.spi.Exec.doWith(Exec.java:25)
    at org.embulk.exec.BulkLoader.run(BulkLoader.java:387)
    at org.embulk.EmbulkEmbed.run(EmbulkEmbed.java:180)
    at org.embulk.EmbulkRunner.runInternal(EmbulkRunner.java:331)
    at org.embulk.EmbulkRunner.run(EmbulkRunner.java:174)
    at org.embulk.cli.EmbulkRun.runSubcommand(EmbulkRun.java:468)
    at org.embulk.cli.EmbulkRun.run(EmbulkRun.java:100)
    at org.embulk.cli.Main.main(Main.java:28)
    Suppressed: java.lang.NullPointerException
        at org.embulk.exec.BulkLoader.doCleanup(BulkLoader.java:496)
        at org.embulk.exec.BulkLoader$3.run(BulkLoader.java:427)
        at org.embulk.exec.BulkLoader$3.run(BulkLoader.java:423)
        at org.embulk.spi.Exec.doWith(Exec.java:25)
        at org.embulk.exec.BulkLoader.cleanup(BulkLoader.java:423)
        at org.embulk.EmbulkEmbed.run(EmbulkEmbed.java:184)
        ... 5 more
Caused by: org.embulk.config.ConfigException: Unsupported presto type 'array(varchar)'
    at org.embulk.input.td.TdInputPlugin.convertPrestoColumnType(TdInputPlugin.java:307)
    at org.embulk.input.td.TdInputPlugin.convertColumnType(TdInputPlugin.java:284)
    at org.embulk.input.td.TdInputPlugin.convertSchema(TdInputPlugin.java:274)
    at org.embulk.input.td.TdInputPlugin.transaction(TdInputPlugin.java:153)
    at org.embulk.exec.BulkLoader.doRun(BulkLoader.java:544)
    ... 11 more

Error: org.embulk.config.ConfigException: Unsupported presto type 'array(varchar)'

Expected Results

Support Array(xxx) type

muga commented 6 years ago

Those features are included in v0.2.1 https://github.com/muga/embulk-input-td/pull/19

muga commented 6 years ago

https://rubygems.org/gems/embulk-input-td/versions/0.2.1 released. Please let me know if you have feedbacks.

toru-takahashi commented 6 years ago

Thank you!