search

Cascading / lingual

Stand-alone ANSI SQL for Cascading on Apache Hadoop
http://www.cascading.org/lingual/
48 stars 17 forks source link

Cannot specify location of catalog in HDFS with "lingual catalog --uri" #16

Open quux00 opened 10 years ago

quux00 commented 10 years ago

According to the help screen for "lingual catalog --help" one should be able to specify the directory for the catalog location with the --uri option:

--uri <directory>                    path to catalog location, defaults is
                                       current directory on current       
                                       platform (default: ./)

But my attempts to get this work are failing. Examples:

$ lingual catalog --platform hadoop --uri hdfenc/12345 --init
path: hdfs://michael-hadoop-9.foo.com:8020/user/diuser/.lingual has been initialized

$ hadoop fs -rmr -skipTrash .lingual
Deleted hdfs://michael-hadoop-9.foo.com:8020/user/diuser/.lingual

$ lingual catalog --platform hadoop --uri /user/diuser/hdfenc/12345 --init
path: hdfs://michael-hadoop-9.foo.com:8020/user/diuser/.lingual has been initialized

$ hadoop fs -rmr -skipTrash .lingual
Deleted hdfs://michael-hadoop-9.foo.com:8020/user/diuser/.lingual

lingual catalog --platform hadoop \
  --uri hdfs://michael-hadoop-9.foo.com:8020/user/diuser/hdfenc/12345/.lingual \
  --init

In every case it always creates it in the top level HDFS directory for my HDFS user.

Note: you can specify the catalog location with:

lingual catalog --platform hadoop --config catalog=hdfenc/12345 --init

So either the documentation should be updated or the code should be adjusted to allow the --uri switch to work as well.

Tested on: Hadoop 1 (Hortonworks Data Platform 3.2)

$ lingual catalog --version
Concurrent, Inc - Lingual:1.0.3-wip-320, Cascading:2.2.0