search

thinkaurelius / titan

Distributed Graph Database
http://titandb.io
Apache License 2.0
5.25k stars 1.01k forks source link

Problem connecting titan (0.5.1) to an hbase backend #832

Closed levin-royl closed 9 years ago

levin-royl commented 9 years ago

Hi,

In my configuration I run my code (including a titan database) from a 64bit windows machine (inside eclipse) which uses an hbase backend installed on a 64bit remote linux machine. I am initializing the graph as follows and with the following config file.

TitanGraph tg = TitanFactory.open(config);

config file:

storage.backend=hbase storage.hostname=rcc-hrl-kvg-469 storage.hbase.ext.hbase.zookeeper.property.clientPort=21810 storage.hbase.table=testdata fs.default.name=hdfs://rcc-hrl-kvg-469:9100/user/royl

Note that a similar setting worked well when I used titan 0.4.4. It seems from the error that titan is trying to run hadoop from my local (windows) machine for some reason.

Output (including error):

0 [main] INFO com.thinkaurelius.titan.diskstorage.hbase.HBaseCompatLoader - Instantiated HBase compatibility layer supporting runtime HBase version 0.98.2-hadoop2: com.thinkaurelius.titan.diskstorage.hbase.HBaseCompat0_98 200 [main] INFO com.thinkaurelius.titan.diskstorage.hbase.HBaseStoreManager - HBase configuration: setting hbase.zookeeper.property.clientPort=21810 200 [main] INFO com.thinkaurelius.titan.diskstorage.hbase.HBaseStoreManager - Copied host list from root.storage.hostname to hbase.zookeeper.quorum: rcc-hrl-kvg-469 1700 [main] WARN org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2108 [main] ERROR org.apache.hadoop.util.Shell - Failed to locate the winutils binary in the hadoop binary path java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries. at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:278) at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:300) at org.apache.hadoop.util.Shell.(Shell.java:293) at org.apache.hadoop.util.StringUtils.(StringUtils.java:76) at org.apache.hadoop.conf.Configuration.getStrings(Configuration.java:1514) at org.apache.hadoop.hbase.zookeeper.ZKConfig.makeZKProps(ZKConfig.java:113) at org.apache.hadoop.hbase.zookeeper.ZKConfig.getZKQuorumServersString(ZKConfig.java:265) at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.(ZooKeeperWatcher.java:159) at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.(ZooKeeperWatcher.java:134) at org.apache.hadoop.hbase.client.ZooKeeperKeepAliveConnection.(ZooKeeperKeepAliveConnection.java:43) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveZooKeeperWatcher(HConnectionManager.java:1768) at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:82) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.retrieveClusterId(HConnectionManager.java:857) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.(HConnectionManager.java:662) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:525) at org.apache.hadoop.hbase.client.HConnectionManager.createConnection(HConnectionManager.java:414) at org.apache.hadoop.hbase.client.HConnectionManager.createConnection(HConnectionManager.java:309) at com.thinkaurelius.titan.diskstorage.hbase.HBaseStoreManager.(HBaseStoreManager.java:323) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:525) at com.thinkaurelius.titan.util.system.ConfigurationUtil.instantiate(ConfigurationUtil.java:44) at com.thinkaurelius.titan.diskstorage.Backend.getImplementationClass(Backend.java:425) at com.thinkaurelius.titan.diskstorage.Backend.getStorageManager(Backend.java:366) at com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration.(GraphDatabaseConfiguration.java:1250) at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:91) at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:71) at com.ibm.titanlytics.Main.fillEmptyRandomGraph(Main.java:69) at com.ibm.titanlytics.Main.main(Main.java:30) 2137 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT 2137 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:host.name=ROYL-TP.haifa.ibm.com 2137 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.version=1.7.0 2138 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.vendor=Oracle Corporation 2138 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.home=C:\Program Files\Java\jdk1.7.0\jre 2138 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.class.path=D:\workspace\titandatabase\titan-analytics\target\classes;C:\Program Files\Java\jdk1.7.0\lib\tools.jar;C:\Users\royl.m2\repository\com\thinkaurelius\titan\titan-all\0.5.1\titan-all-0.5.1.jar;C:\Users\royl.m2\repository\com\thinkaurelius\titan\titan-core\0.5.1\titan-core-0.5.1.jar;C:\Users\royl.m2\repository\com\tinkerpop\blueprints\blueprints-core\2.5.0\blueprints-core-2.5.0.jar;C:\Users\royl.m2\repository\colt\colt\1.2.0\colt-1.2.0.jar;C:\Users\royl.m2\repository\concurrent\concurrent\1.3.4\concurrent-1.3.4.jar;C:\Users\royl.m2\repository\com\tinkerpop\frames\2.5.0\frames-2.5.0.jar;C:\Users\royl.m2\repository\org\javassist\javassist\3.18.0-GA\javassist-3.18.0-GA.jar;C:\Users\royl.m2\repository\org\glassfish\javax.json\1.0\javax.json-1.0.jar;C:\Users\royl.m2\repository\com\codahale\metrics\metrics-core\3.0.1\metrics-core-3.0.1.jar;C:\Users\royl.m2\repository\com\codahale\metrics\metrics-ganglia\3.0.1\metrics-ganglia-3.0.1.jar;C:\Users\royl.m2\repository\info\ganglia\gmetric4j\gmetric4j\1.0.3\gmetric4j-1.0.3.jar;C:\Users\royl.m2\repository\org\acplt\oncrpc\1.0.7\oncrpc-1.0.7.jar;C:\Users\royl.m2\repository\com\codahale\metrics\metrics-graphite\3.0.1\metrics-graphite-3.0.1.jar;C:\Users\royl.m2\repository\org\reflections\reflections\0.9.9-RC1\reflections-0.9.9-RC1.jar;C:\Users\royl.m2\repository\dom4j\dom4j\1.6.1\dom4j-1.6.1.jar;C:\Users\royl.m2\repository\xml-apis\xml-apis\1.0.b2\xml-apis-1.0.b2.jar;C:\Users\royl.m2\repository\com\spatial4j\spatial4j\0.4.1\spatial4j-0.4.1.jar;C:\Users\royl.m2\repository\commons-collections\commons-collections\3.2.1\commons-collections-3.2.1.jar;C:\Users\royl.m2\repository\com\google\code\findbugs\jsr305\1.3.9\jsr305-1.3.9.jar;C:\Users\royl.m2\repository\com\carrotsearch\hppc\0.6.0\hppc-0.6.0.jar;C:\Users\royl.m2\repository\com\github\stephenc\high-scale-lib\high-scale-lib\1.1.4\high-scale-lib-1.1.4.jar;C:\Users\royl.m2\repository\com\esotericsoftware\kryo\kryo\2.22\kryo-2.22.jar;C:\Users\royl.m2\repository\com\thinkaurelius\titan\titan-cassandra\0.5.1\titan-cassandra-0.5.1.jar;C:\Users\royl.m2\repository\com\carrotsearch\junit-benchmarks\0.7.0\junit-benchmarks-0.7.0.jar;C:\Users\royl.m2\repository\com\carrotsearch\randomizedtesting\randomizedtesting-runner\2.0.8\randomizedtesting-runner-2.0.8.jar;C:\Users\royl.m2\repository\org\apache\cassandra\cassandra-all\2.0.8\cassandra-all-2.0.8.jar;C:\Users\royl.m2\repository\net\jpountz\lz4\lz4\1.2.0\lz4-1.2.0.jar;C:\Users\royl.m2\repository\com\ning\compress-lzf\0.8.4\compress-lzf-0.8.4.jar;C:\Users\royl.m2\repository\org\apache\commons\commons-lang3\3.1\commons-lang3-3.1.jar;C:\Users\royl.m2\repository\com\googlecode\concurrentlinkedhashmap\concurrentlinkedhashmap-lru\1.3\concurrentlinkedhashmap-lru-1.3.jar;C:\Users\royl.m2\repository\org\antlr\antlr\3.2\antlr-3.2.jar;C:\Users\royl.m2\repository\jline\jline\1.0\jline-1.0.jar;C:\Users\royl.m2\repository\com\googlecode\json-simple\json-simple\1.1\json-simple-1.1.jar;C:\Users\royl.m2\repository\org\yaml\snakeyaml\1.11\snakeyaml-1.11.jar;C:\Users\royl.m2\repository\edu\stanford\ppl\snaptree\0.1\snaptree-0.1.jar;C:\Users\royl.m2\repository\org\mindrot\jbcrypt\0.3m\jbcrypt-0.3m.jar;C:\Users\royl.m2\repository\com\yammer\metrics\metrics-core\2.2.0\metrics-core-2.2.0.jar;C:\Users\royl.m2\repository\com\addthis\metrics\reporter-config\2.1.0\reporter-config-2.1.0.jar;C:\Users\royl.m2\repository\org\hibernate\hibernate-validator\4.3.0.Final\hibernate-validator-4.3.0.Final.jar;C:\Users\royl.m2\repository\javax\validation\validation-api\1.0.0.GA\validation-api-1.0.0.GA.jar;C:\Users\royl.m2\repository\org\jboss\logging\jboss-logging\3.1.0.CR2\jboss-logging-3.1.0.CR2.jar;C:\Users\royl.m2\repository\com\thinkaurelius\thrift\thrift-server\0.3.3\thrift-server-0.3.3.jar;C:\Users\royl.m2\repository\com\lmax\disruptor\3.0.1\disruptor-3.0.1.jar;C:\Users\royl.m2\repository\net\sf\supercsv\super-csv\2.1.0\super-csv-2.1.0.jar;C:\Users\royl.m2\repository\org\apache\thrift\libthrift\0.9.1\libthrift-0.9.1.jar;C:\Users\royl.m2\repository\org\apache\cassandra\cassandra-thrift\2.0.8\cassandra-thrift-2.0.8.jar;C:\Users\royl.m2\repository\com\github\stephenc\jamm\0.2.5\jamm-0.2.5.jar;C:\Users\royl.m2\repository\commons-pool\commons-pool\1.6\commons-pool-1.6.jar;C:\Users\royl.m2\repository\org\jboss\netty\netty\3.2.7.Final\netty-3.2.7.Final.jar;C:\Users\royl.m2\repository\com\netflix\astyanax\astyanax-core\1.56.48\astyanax-core-1.56.48.jar;C:\Users\royl.m2\repository\com\eaio\uuid\uuid\3.2\uuid-3.2.jar;C:\Users\royl.m2\repository\joda-time\joda-time\1.6.2\joda-time-1.6.2.jar;C:\Users\royl.m2\repository\com\netflix\astyanax\astyanax-thrift\1.56.48\astyanax-thrift-1.56.48.jar;C:\Users\royl.m2\repository\com\netflix\astyanax\astyanax-cassandra\1.56.48\astyanax-cassandra-1.56.48.jar;C:\Users\royl.m2\repository\org\apache\servicemix\bundles\org.apache.servicemix.bundles.commons-csv\1.0-r706900_3\org.apache.servicemix.bundles.commons-csv-1.0-r706900_3.jar;C:\Users\royl.m2\repository\com\netflix\astyanax\astyanax-recipes\1.56.48\astyanax-recipes-1.56.48.jar;C:\Users\royl.m2\repository\org\xerial\snappy\snappy-java\1.0.5-M3\snappy-java-1.0.5-M3.jar;C:\Users\royl.m2\repository\com\thinkaurelius\titan\titan-berkeleyje\0.5.1\titan-berkeleyje-0.5.1.jar;C:\Users\royl.m2\repository\com\sleepycat\je\5.0.73\je-5.0.73.jar;C:\Users\royl.m2\repository\com\thinkaurelius\titan\titan-hbase\0.5.1\titan-hbase-0.5.1.jar;C:\Users\royl.m2\repository\org\apache\hbase\hbase-client\0.98.2-hadoop2\hbase-client-0.98.2-hadoop2.jar;C:\Users\royl.m2\repository\org\apache\hbase\hbase-common\0.98.2-hadoop2\hbase-common-0.98.2-hadoop2.jar;C:\Users\royl.m2\repository\org\apache\hbase\hbase-protocol\0.98.2-hadoop2\hbase-protocol-0.98.2-hadoop2.jar;C:\Users\royl.m2\repository\org\cloudera\htrace\htrace-core\2.04\htrace-core-2.04.jar;C:\Users\royl.m2\repository\org\apache\hadoop\hadoop-mapreduce-client-core\2.2.0\hadoop-mapreduce-client-core-2.2.0.jar;C:\Users\royl.m2\repository\com\github\stephenc\findbugs\findbugs-annotations\1.3.9-1\findbugs-annotations-1.3.9-1.jar;C:\Users\royl.m2\repository\junit\junit\4.11\junit-4.11.jar;C:\Users\royl.m2\repository\org\hamcrest\hamcrest-core\1.3\hamcrest-core-1.3.jar;C:\Users\royl.m2\repository\com\thinkaurelius\titan\titan-hadoop\0.5.1\titan-hadoop-0.5.1.jar;C:\Users\royl.m2\repository\com\tinkerpop\blueprints\blueprints-sail-graph\2.5.0\blueprints-sail-graph-2.5.0.jar;C:\Users\royl.m2\repository\org\openrdf\sesame\sesame-sail-api\2.7.10\sesame-sail-api-2.7.10.jar;C:\Users\royl.m2\repository\org\openrdf\sesame\sesame-query\2.7.10\sesame-query-2.7.10.jar;C:\Users\royl.m2\repository\org\openrdf\sesame\sesame-queryalgebra-model\2.7.10\sesame-queryalgebra-model-2.7.10.jar;C:\Users\royl.m2\repository\org\openrdf\sesame\sesame-queryparser-sparql\2.7.10\sesame-queryparser-sparql-2.7.10.jar;C:\Users\royl.m2\repository\org\openrdf\sesame\sesame-queryparser-api\2.7.10\sesame-queryparser-api-2.7.10.jar;C:\Users\royl.m2\repository\org\openrdf\sesame\sesame-sail-memory\2.7.10\sesame-sail-memory-2.7.10.jar;C:\Users\royl.m2\repository\org\openrdf\sesame\sesame-sail-inferencer\2.7.10\sesame-sail-inferencer-2.7.10.jar;C:\Users\royl.m2\repository\org\openrdf\sesame\sesame-queryparser-serql\2.7.10\sesame-queryparser-serql-2.7.10.jar;C:\Users\royl.m2\repository\org\openrdf\sesame\sesame-queryalgebra-evaluation\2.7.10\sesame-queryalgebra-evaluation-2.7.10.jar;C:\Users\royl.m2\repository\org\openrdf\sesame\sesame-repository-sparql\2.7.10\sesame-repository-sparql-2.7.10.jar;C:\Users\royl.m2\repository\org\openrdf\sesame\sesame-queryresultio-sparqlxml\2.7.10\sesame-queryresultio-sparqlxml-2.7.10.jar;C:\Users\royl.m2\repository\org\openrdf\sesame\sesame-http-client\2.7.10\sesame-http-client-2.7.10.jar;C:\Users\royl.m2\repository\org\openrdf\sesame\sesame-http-protocol\2.7.10\sesame-http-protocol-2.7.10.jar;C:\Users\royl.m2\repository\org\openrdf\sesame\sesame-queryresultio-api\2.7.10\sesame-queryresultio-api-2.7.10.jar;C:\Users\royl.m2\repository\org\openrdf\sesame\sesame-sail-nativerdf\2.7.10\sesame-sail-nativerdf-2.7.10.jar;C:\Users\royl.m2\repository\net\fortytwo\linked-data-sail\1.1\linked-data-sail-1.1.jar;C:\Users\royl.m2\repository\net\fortytwo\ripple\ripple-flow-rdf\1.1\ripple-flow-rdf-1.1.jar;C:\Users\royl.m2\repository\net\fortytwo\ripple\ripple-flow\1.1\ripple-flow-1.1.jar;C:\Users\royl.m2\repository\net\fortytwo\ripple\ripple-base\1.1\ripple-base-1.1.jar;C:\Users\royl.m2\repository\org\restlet\jse\org.restlet\2.1.1\org.restlet-2.1.1.jar;C:\Users\royl.m2\repository\org\openrdf\sesame\sesame-rio-binary\2.7.10\sesame-rio-binary-2.7.10.jar;C:\Users\royl.m2\repository\org\openrdf\sesame\sesame-rio-nquads\2.7.10\sesame-rio-nquads-2.7.10.jar;C:\Users\royl.m2\repository\org\openrdf\sesame\sesame-rio-rdfjson\2.7.10\sesame-rio-rdfjson-2.7.10.jar;C:\Users\royl.m2\repository\org\semarglproject\semargl-sesame\0.4\semargl-sesame-0.4.jar;C:\Users\royl.m2\repository\org\semarglproject\semargl-core\0.4\semargl-core-0.4.jar;C:\Users\royl.m2\repository\org\semarglproject\semargl-rdfa\0.4\semargl-rdfa-0.4.jar;C:\Users\royl.m2\repository\org\semarglproject\semargl-rdf\0.4\semargl-rdf-0.4.jar;C:\Users\royl.m2\repository\com\github\jsonld-java\jsonld-java-sesame\0.3\jsonld-java-sesame-0.3.jar;C:\Users\royl.m2\repository\com\github\jsonld-java\jsonld-java\0.3\jsonld-java-0.3.jar;C:\Users\royl.m2\repository\org\apache\httpcomponents\httpclient-cache\4.2.5\httpclient-cache-4.2.5.jar;C:\Users\royl.m2\repository\net\fortytwo\sesametools\repository-sail\1.8\repository-sail-1.8.jar;C:\Users\royl.m2\repository\net\fortytwo\sesametools\common\1.8\common-1.8.jar;C:\Users\royl.m2\repository\org\openrdf\sesame\sesame-repository-api\2.7.10\sesame-repository-api-2.7.10.jar;C:\Users\royl.m2\repository\com\tinkerpop\gremlin\gremlin-groovy\2.5.0\gremlin-groovy-2.5.0.jar;C:\Users\royl.m2\repository\com\tinkerpop\gremlin\gremlin-java\2.5.0\gremlin-java-2.5.0.jar;C:\Users\royl.m2\repository\org\apache\ivy\ivy\2.3.0\ivy-2.3.0.jar;C:\Users\royl.m2\repository\com\tinkerpop\pipes\2.5.0\pipes-2.5.0.jar;C:\Users\royl.m2\repository\org\codehaus\groovy\groovy\1.8.9\groovy-1.8.9.jar;C:\Users\royl.m2\repository\antlr\antlr\2.7.7\antlr-2.7.7.jar;C:\Users\royl.m2\repository\asm\asm-commons\3.2\asm-commons-3.2.jar;C:\Users\royl.m2\repository\asm\asm-util\3.2\asm-util-3.2.jar;C:\Users\royl.m2\repository\asm\asm-analysis\3.2\asm-analysis-3.2.jar;C:\Users\royl.m2\repository\asm\asm-tree\3.2\asm-tree-3.2.jar;C:\Users\royl.m2\repository\org\apache\ant\ant\1.8.3\ant-1.8.3.jar;C:\Users\royl.m2\repository\org\apache\ant\ant-launcher\1.8.3\ant-launcher-1.8.3.jar;C:\Users\royl.m2\repository\org\fusesource\jansi\jansi\1.5\jansi-1.5.jar;C:\Users\royl.m2\repository\org\elasticsearch\elasticsearch-hadoop-mr\2.0.0\elasticsearch-hadoop-mr-2.0.0.jar;C:\Users\royl.m2\repository\org\openrdf\sesame\sesame-rio-n3\2.7.10\sesame-rio-n3-2.7.10.jar;C:\Users\royl.m2\repository\org\openrdf\sesame\sesame-model\2.7.10\sesame-model-2.7.10.jar;C:\Users\royl.m2\repository\org\openrdf\sesame\sesame-rio-api\2.7.10\sesame-rio-api-2.7.10.jar;C:\Users\royl.m2\repository\org\openrdf\sesame\sesame-rio-datatypes\2.7.10\sesame-rio-datatypes-2.7.10.jar;C:\Users\royl.m2\repository\org\openrdf\sesame\sesame-rio-languages\2.7.10\sesame-rio-languages-2.7.10.jar;C:\Users\royl.m2\repository\org\openrdf\sesame\sesame-rio-ntriples\2.7.10\sesame-rio-ntriples-2.7.10.jar;C:\Users\royl.m2\repository\org\openrdf\sesame\sesame-rio-rdfxml\2.7.10\sesame-rio-rdfxml-2.7.10.jar;C:\Users\royl.m2\repository\org\openrdf\sesame\sesame-util\2.7.10\sesame-util-2.7.10.jar;C:\Users\royl.m2\repository\org\openrdf\sesame\sesame-rio-trig\2.7.10\sesame-rio-trig-2.7.10.jar;C:\Users\royl.m2\repository\org\openrdf\sesame\sesame-rio-trix\2.7.10\sesame-rio-trix-2.7.10.jar;C:\Users\royl.m2\repository\org\openrdf\sesame\sesame-rio-turtle\2.7.10\sesame-rio-turtle-2.7.10.jar;C:\Users\royl.m2\repository\org\apache\hadoop\hadoop-client\2.2.0\hadoop-client-2.2.0.jar;C:\Users\royl.m2\repository\org\apache\hadoop\hadoop-hdfs\2.2.0\hadoop-hdfs-2.2.0.jar;C:\Users\royl.m2\repository\org\apache\hadoop\hadoop-mapreduce-client-app\2.2.0\hadoop-mapreduce-client-app-2.2.0.jar;C:\Users\royl.m2\repository\org\apache\hadoop\hadoop-mapreduce-client-common\2.2.0\hadoop-mapreduce-client-common-2.2.0.jar;C:\Users\royl.m2\repository\org\apache\hadoop\hadoop-yarn-client\2.2.0\hadoop-yarn-client-2.2.0.jar;C:\Users\royl.m2\repository\org\apache\hadoop\hadoop-yarn-server-common\2.2.0\hadoop-yarn-server-common-2.2.0.jar;C:\Users\royl.m2\repository\org\apache\hadoop\hadoop-mapreduce-client-shuffle\2.2.0\hadoop-mapreduce-client-shuffle-2.2.0.jar;C:\Users\royl.m2\repository\org\apache\hadoop\hadoop-mapreduce-client-jobclient\2.2.0\hadoop-mapreduce-client-jobclient-2.2.0.jar;C:\Users\royl.m2\repository\com\thinkaurelius\titan\titan-es\0.5.1\titan-es-0.5.1.jar;C:\Users\royl.m2\repository\org\elasticsearch\elasticsearch\1.2.1\elasticsearch-1.2.1.jar;C:\Users\royl.m2\repository\org\apache\lucene\lucene-codecs\4.8.1\lucene-codecs-4.8.1.jar;C:\Users\royl.m2\repository\org\apache\lucene\lucene-queries\4.8.1\lucene-queries-4.8.1.jar;C:\Users\royl.m2\repository\org\apache\lucene\lucene-memory\4.8.1\lucene-memory-4.8.1.jar;C:\Users\royl.m2\repository\org\apache\lucene\lucene-highlighter\4.8.1\lucene-highlighter-4.8.1.jar;C:\Users\royl.m2\repository\org\apache\lucene\lucene-sandbox\4.8.1\lucene-sandbox-4.8.1.jar;C:\Users\royl.m2\repository\org\apache\lucene\lucene-suggest\4.8.1\lucene-suggest-4.8.1.jar;C:\Users\royl.m2\repository\org\apache\lucene\lucene-misc\4.8.1\lucene-misc-4.8.1.jar;C:\Users\royl.m2\repository\org\apache\lucene\lucene-join\4.8.1\lucene-join-4.8.1.jar;C:\Users\royl.m2\repository\org\apache\lucene\lucene-grouping\4.8.1\lucene-grouping-4.8.1.jar;C:\Users\royl.m2\repository\com\thinkaurelius\titan\titan-solr\0.5.1\titan-solr-0.5.1.jar;C:\Users\royl.m2\repository\org\apache\solr\solr-solrj\4.8.1\solr-solrj-4.8.1.jar;C:\Users\royl.m2\repository\org\apache\httpcomponents\httpclient\4.3.1\httpclient-4.3.1.jar;C:\Users\royl.m2\repository\org\apache\httpcomponents\httpcore\4.3\httpcore-4.3.jar;C:\Users\royl.m2\repository\org\apache\httpcomponents\httpmime\4.3.1\httpmime-4.3.1.jar;C:\Users\royl.m2\repository\org\codehaus\woodstox\wstx-asl\3.2.7\wstx-asl-3.2.7.jar;C:\Users\royl.m2\repository\org\noggit\noggit\0.5\noggit-0.5.jar;C:\Users\royl.m2\repository\org\apache\solr\solr-core\4.8.1\solr-core-4.8.1.jar;C:\Users\royl.m2\repository\org\apache\lucene\lucene-analyzers-kuromoji\4.8.1\lucene-analyzers-kuromoji-4.8.1.jar;C:\Users\royl.m2\repository\org\apache\lucene\lucene-analyzers-phonetic\4.8.1\lucene-analyzers-phonetic-4.8.1.jar;C:\Users\royl.m2\repository\org\apache\lucene\lucene-expressions\4.8.1\lucene-expressions-4.8.1.jar;C:\Users\royl.m2\repository\commons-fileupload\commons-fileupload\1.2.1\commons-fileupload-1.2.1.jar;C:\Users\royl.m2\repository\org\antlr\antlr-runtime\3.5\antlr-runtime-3.5.jar;C:\Users\royl.m2\repository\org\eclipse\jetty\jetty-continuation\8.1.10.v20130312\jetty-continuation-8.1.10.v20130312.jar;C:\Users\royl.m2\repository\org\eclipse\jetty\jetty-deploy\8.1.10.v20130312\jetty-deploy-8.1.10.v20130312.jar;C:\Users\royl.m2\repository\org\eclipse\jetty\jetty-http\8.1.10.v20130312\jetty-http-8.1.10.v20130312.jar;C:\Users\royl.m2\repository\org\eclipse\jetty\jetty-io\8.1.10.v20130312\jetty-io-8.1.10.v20130312.jar;C:\Users\royl.m2\repository\org\eclipse\jetty\jetty-jmx\8.1.10.v20130312\jetty-jmx-8.1.10.v20130312.jar;C:\Users\royl.m2\repository\org\eclipse\jetty\jetty-security\8.1.10.v20130312\jetty-security-8.1.10.v20130312.jar;C:\Users\royl.m2\repository\org\eclipse\jetty\jetty-server\8.1.10.v20130312\jetty-server-8.1.10.v20130312.jar;C:\Users\royl.m2\repository\org\eclipse\jetty\jetty-servlet\8.1.10.v20130312\jetty-servlet-8.1.10.v20130312.jar;C:\Users\royl.m2\repository\org\eclipse\jetty\jetty-util\8.1.10.v20130312\jetty-util-8.1.10.v20130312.jar;C:\Users\royl.m2\repository\org\eclipse\jetty\jetty-webapp\8.1.10.v20130312\jetty-webapp-8.1.10.v20130312.jar;C:\Users\royl.m2\repository\org\eclipse\jetty\jetty-xml\8.1.10.v20130312\jetty-xml-8.1.10.v20130312.jar;C:\Users\royl.m2\repository\org\eclipse\jetty\orbit\javax.servlet\3.0.0.v201112011016\javax.servlet-3.0.0.v201112011016.jar;C:\Users\royl.m2\repository\org\ow2\asm\asm\4.1\asm-4.1.jar;C:\Users\royl.m2\repository\org\ow2\asm\asm-commons\4.1\asm-commons-4.1.jar;C:\Users\royl.m2\repository\org\restlet\jee\org.restlet\2.1.1\org.restlet-2.1.1.jar;C:\Users\royl.m2\repository\org\restlet\jee\org.restlet.ext.servlet\2.1.1\org.restlet.ext.servlet-2.1.1.jar;C:\Users\royl.m2\repository\com\vividsolutions\jts\1.13\jts-1.13.jar;C:\Users\royl.m2\repository\com\thinkaurelius\titan\titan-lucene\0.5.1\titan-lucene-0.5.1.jar;C:\Users\royl.m2\repository\org\apache\lucene\lucene-core\4.8.1\lucene-core-4.8.1.jar;C:\Users\royl.m2\repository\org\apache\lucene\lucene-analyzers-common\4.8.1\lucene-analyzers-common-4.8.1.jar;C:\Users\royl.m2\repository\org\apache\lucene\lucene-queryparser\4.8.1\lucene-queryparser-4.8.1.jar;C:\Users\royl.m2\repository\org\apache\lucene\lucene-spatial\4.8.1\lucene-spatial-4.8.1.jar;C:\Users\royl.m2\repository\com\thinkaurelius\titan\titan-rexster\0.5.1\titan-rexster-0.5.1.jar;C:\Users\royl.m2\repository\com\tinkerpop\rexster\rexster-core\2.5.0\rexster-core-2.5.0.jar;C:\Users\royl.m2\repository\javax\servlet\javax.servlet-api\3.0.1\javax.servlet-api-3.0.1.jar;C:\Users\royl.m2\repository\org\codehaus\jackson\jackson-mapper-asl\1.9.2\jackson-mapper-asl-1.9.2.jar;C:\Users\royl.m2\repository\org\codehaus\jackson\jackson-core-asl\1.9.2\jackson-core-asl-1.9.2.jar;C:\Users\royl.m2\repository\org\codehaus\jackson\jackson-xc\1.9.2\jackson-xc-1.9.2.jar;C:\Users\royl.m2\repository\org\codehaus\jackson\jackson-jaxrs\1.9.2\jackson-jaxrs-1.9.2.jar;C:\Users\royl.m2\repository\com\fasterxml\jackson\core\jackson-core\2.2.3\jackson-core-2.2.3.jar;C:\Users\royl.m2\repository\com\fasterxml\jackson\core\jackson-databind\2.2.3\jackson-databind-2.2.3.jar;C:\Users\royl.m2\repository\com\fasterxml\jackson\datatype\jackson-datatype-json-org\2.2.3\jackson-datatype-json-org-2.2.3.jar;C:\Users\royl.m2\repository\org\json\json\20090211\json-20090211.jar;C:\Users\royl.m2\repository\com\fasterxml\jackson\core\jackson-annotations\2.2.3\jackson-annotations-2.2.3.jar;C:\Users\royl.m2\repository\org\slf4j\slf4j-api\1.7.5\slf4j-api-1.7.5.jar;C:\Users\royl.m2\repository\org\apache\hadoop\hadoop-common\2.2.0\hadoop-common-2.2.0.jar;C:\Users\royl.m2\repository\org\apache\hadoop\hadoop-annotations\2.2.0\hadoop-annotations-2.2.0.jar;C:\Users\royl.m2\repository\com\google\guava\guava\11.0.2\guava-11.0.2.jar;C:\Users\royl.m2\repository\commons-cli\commons-cli\1.2\commons-cli-1.2.jar;C:\Users\royl.m2\repository\org\apache\commons\commons-math\2.1\commons-math-2.1.jar;C:\Users\royl.m2\repository\xmlenc\xmlenc\0.52\xmlenc-0.52.jar;C:\Users\royl.m2\repository\commons-httpclient\commons-httpclient\3.1\commons-httpclient-3.1.jar;C:\Users\royl.m2\repository\commons-codec\commons-codec\1.4\commons-codec-1.4.jar;C:\Users\royl.m2\repository\commons-io\commons-io\2.1\commons-io-2.1.jar;C:\Users\royl.m2\repository\commons-net\commons-net\3.1\commons-net-3.1.jar;C:\Users\royl.m2\repository\javax\servlet\servlet-api\2.5\servlet-api-2.5.jar;C:\Users\royl.m2\repository\org\mortbay\jetty\jetty\6.1.26\jetty-6.1.26.jar;C:\Users\royl.m2\repository\org\mortbay\jetty\jetty-util\6.1.26\jetty-util-6.1.26.jar;C:\Users\royl.m2\repository\com\sun\jersey\jersey-core\1.9\jersey-core-1.9.jar;C:\Users\royl.m2\repository\com\sun\jersey\jersey-json\1.9\jersey-json-1.9.jar;C:\Users\royl.m2\repository\org\codehaus\jettison\jettison\1.1\jettison-1.1.jar;C:\Users\royl.m2\repository\stax\stax-api\1.0.1\stax-api-1.0.1.jar;C:\Users\royl.m2\repository\com\sun\xml\bind\jaxb-impl\2.2.3-1\jaxb-impl-2.2.3-1.jar;C:\Users\royl.m2\repository\javax\xml\bind\jaxb-api\2.2.2\jaxb-api-2.2.2.jar;C:\Users\royl.m2\repository\javax\activation\activation\1.1\activation-1.1.jar;C:\Users\royl.m2\repository\com\sun\jersey\jersey-server\1.9\jersey-server-1.9.jar;C:\Users\royl.m2\repository\asm\asm\3.1\asm-3.1.jar;C:\Users\royl.m2\repository\tomcat\jasper-compiler\5.5.23\jasper-compiler-5.5.23.jar;C:\Users\royl.m2\repository\tomcat\jasper-runtime\5.5.23\jasper-runtime-5.5.23.jar;C:\Users\royl.m2\repository\javax\servlet\jsp\jsp-api\2.1\jsp-api-2.1.jar;C:\Users\royl.m2\repository\commons-el\commons-el\1.0\commons-el-1.0.jar;C:\Users\royl.m2\repository\commons-logging\commons-logging\1.1.1\commons-logging-1.1.1.jar;C:\Users\royl.m2\repository\log4j\log4j\1.2.17\log4j-1.2.17.jar;C:\Users\royl.m2\repository\net\java\dev\jets3t\jets3t\0.6.1\jets3t-0.6.1.jar;C:\Users\royl.m2\repository\commons-lang\commons-lang\2.5\commons-lang-2.5.jar;C:\Users\royl.m2\repository\commons-configuration\commons-configuration\1.6\commons-configuration-1.6.jar;C:\Users\royl.m2\repository\commons-digester\commons-digester\1.8\commons-digester-1.8.jar;C:\Users\royl.m2\repository\commons-beanutils\commons-beanutils\1.7.0\commons-beanutils-1.7.0.jar;C:\Users\royl.m2\repository\commons-beanutils\commons-beanutils-core\1.8.0\commons-beanutils-core-1.8.0.jar;C:\Users\royl.m2\repository\org\slf4j\slf4j-log4j12\1.7.5\slf4j-log4j12-1.7.5.jar;C:\Users\royl.m2\repository\org\apache\avro\avro\1.7.4\avro-1.7.4.jar;C:\Users\royl.m2\repository\com\thoughtworks\paranamer\paranamer\2.3\paranamer-2.3.jar;C:\Users\royl.m2\repository\com\google\protobuf\protobuf-java\2.5.0\protobuf-java-2.5.0.jar;C:\Users\royl.m2\repository\org\apache\hadoop\hadoop-auth\2.2.0\hadoop-auth-2.2.0.jar;C:\Users\royl.m2\repository\com\jcraft\jsch\0.1.42\jsch-0.1.42.jar;C:\Users\royl.m2\repository\org\apache\zookeeper\zookeeper\3.4.5\zookeeper-3.4.5.jar;C:\Users\royl.m2\repository\org\apache\commons\commons-compress\1.4.1\commons-compress-1.4.1.jar;C:\Users\royl.m2\repository\org\tukaani\xz\1.0\xz-1.0.jar;C:\Users\royl.m2\repository\org\apache\hadoop\hadoop-yarn-common\2.2.0\hadoop-yarn-common-2.2.0.jar;C:\Users\royl.m2\repository\org\apache\hadoop\hadoop-yarn-api\2.2.0\hadoop-yarn-api-2.2.0.jar;C:\Users\royl.m2\repository\com\google\inject\extensions\guice-servlet\3.0\guice-servlet-3.0.jar;C:\Users\royl.m2\repository\io\netty\netty\3.6.2.Final\netty-3.6.2.Final.jar;C:\Users\royl.m2\repository\com\google\inject\guice\3.0\guice-3.0.jar;C:\Users\royl.m2\repository\javax\inject\javax.inject\1\javax.inject-1.jar;C:\Users\royl.m2\repository\aopalliance\aopalliance\1.0\aopalliance-1.0.jar;C:\Users\royl.m2\repository\com\sun\jersey\jersey-test-framework\jersey-test-framework-grizzly2\1.9\jersey-test-framework-grizzly2-1.9.jar;C:\Users\royl.m2\repository\com\sun\jersey\jersey-test-framework\jersey-test-framework-core\1.9\jersey-test-framework-core-1.9.jar;C:\Users\royl.m2\repository\com\sun\jersey\jersey-client\1.9\jersey-client-1.9.jar;C:\Users\royl.m2\repository\com\sun\jersey\jersey-grizzly2\1.9\jersey-grizzly2-1.9.jar;C:\Users\royl.m2\repository\org\glassfish\grizzly\grizzly-http\2.1.2\grizzly-http-2.1.2.jar;C:\Users\royl.m2\repository\org\glassfish\grizzly\grizzly-framework\2.1.2\grizzly-framework-2.1.2.jar;C:\Users\royl.m2\repository\org\glassfish\gmbal\gmbal-api-only\3.0.0-b023\gmbal-api-only-3.0.0-b023.jar;C:\Users\royl.m2\repository\org\glassfish\external\management-api\3.0.0-b012\management-api-3.0.0-b012.jar;C:\Users\royl.m2\repository\org\glassfish\grizzly\grizzly-http-server\2.1.2\grizzly-http-server-2.1.2.jar;C:\Users\royl.m2\repository\org\glassfish\grizzly\grizzly-rcm\2.1.2\grizzly-rcm-2.1.2.jar;C:\Users\royl.m2\repository\org\glassfish\grizzly\grizzly-http-servlet\2.1.2\grizzly-http-servlet-2.1.2.jar;C:\Users\royl.m2\repository\org\glassfish\javax.servlet\3.1\javax.servlet-3.1.jar;C:\Users\royl.m2\repository\com\sun\jersey\contribs\jersey-guice\1.9\jersey-guice-1.9.jar 2140 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.library.path=C:\Program Files\Java\jdk1.7.0\bin;C:\Windows\Sun\Java\bin;C:\Windows\system32;C:\Windows;C:\Program Files\Java\jdk1.7.0\jre\bin;C:\Program Files (x86)\PC Connectivity Solution\;C:\Python27\;C:\Python27\Scripts;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files (x86)\Intel\Services\IPT\;C:\Program Files\ThinkPad\Bluetooth Software\;C:\Program Files\ThinkPad\Bluetooth Software\syswow64;C:\Program Files (x86)\Lenovo\Access Connections\;C:\Program Files (x86)\IBM\Mobility Client\;C:\Program Files\MiKTeX 2.9\miktex\bin\x64\;C:\Program Files\TortoiseSVN\bin;C:\Program Files\Intel\WiFi\bin\;C:\Program Files\Common Files\Intel\WirelessCommon\;C:\Program Files (x86)\CloudFoundry;C:\Program Files (x86)\Git\cmd;C:\Python27\;C:\Python27\Scripts;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files (x86)\Intel\Services\IPT\;C:\Program Files\ThinkPad\Bluetooth Software\;C:\Program Files\ThinkPad\Bluetooth Software\syswow64;C:\Program Files (x86)\Lenovo\Access Connections\;C:\Program Files (x86)\IBM\Mobility Client\;C:\Program Files\MiKTeX 2.9\miktex\bin\x64\;C:\Program Files\TortoiseSVN\bin;C:\Users\royl\AppData\Roaming\Python\Scripts;C:\Program Files\Intel\WiFi\bin\;C:\Program Files\Common Files\Intel\WirelessCommon\;C:\Program Files (x86)\WinAnt\bin;C:\Program Files\Apache Software Foundation\apache-maven-3.2.3\bin;.;;.;;.;;.;;.;;.;;.;;. 2140 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.io.tmpdir=C:\Users\royl\AppData\Local\Temp\ 2140 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.compiler= 2140 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.name=Windows 7 2140 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.arch=amd64 2140 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.version=6.1 2140 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:user.name=royl 2140 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:user.home=C:\Users\royl 2140 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:user.dir=D:\workspace\titandatabase\titan-analytics 2142 [main] INFO org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=rcc-hrl-kvg-469:21810 sessionTimeout=90000 watcher=hconnection-0x2e920878, quorum=rcc-hrl-kvg-469:21810, baseZNode=/hbase 2182 [main] INFO org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper - Process identifier=hconnection-0x2e920878 connecting to ZooKeeper ensemble=rcc-hrl-kvg-469:21810 2186 [main-SendThread(rcc-hrl-kvg-469.haifa.ibm.com:21810)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server rcc-hrl-kvg-469.haifa.ibm.com/9.148.243.236:21810. Will not attempt to authenticate using SASL (unknown error) 2213 [main-SendThread(rcc-hrl-kvg-469.haifa.ibm.com:21810)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to rcc-hrl-kvg-469.haifa.ibm.com/9.148.243.236:21810, initiating session 2258 [main-SendThread(rcc-hrl-kvg-469.haifa.ibm.com:21810)] INFO org.apache.zookeeper.ClientCnxn - Session establishment complete on server rcc-hrl-kvg-469.haifa.ibm.com/9.148.243.236:21810, sessionid = 0x149b810442c001e, negotiated timeout = 90000 2706 [main] INFO org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=rcc-hrl-kvg-469:21810 sessionTimeout=90000 watcher=hconnection-0x1ea326f9, quorum=rcc-hrl-kvg-469:21810, baseZNode=/hbase 2708 [main] INFO org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper - Process identifier=hconnection-0x1ea326f9 connecting to ZooKeeper ensemble=rcc-hrl-kvg-469:21810 2708 [main-SendThread(rcc-hrl-kvg-469.haifa.ibm.com:21810)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server rcc-hrl-kvg-469.haifa.ibm.com/9.148.243.236:21810. Will not attempt to authenticate using SASL (unknown error) 2733 [main-SendThread(rcc-hrl-kvg-469.haifa.ibm.com:21810)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to rcc-hrl-kvg-469.haifa.ibm.com/9.148.243.236:21810, initiating session 2766 [main-SendThread(rcc-hrl-kvg-469.haifa.ibm.com:21810)] INFO org.apache.zookeeper.ClientCnxn - Session establishment complete on server rcc-hrl-kvg-469.haifa.ibm.com/9.148.243.236:21810, sessionid = 0x149b810442c001f, negotiated timeout = 90000 2795 [main] INFO org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=rcc-hrl-kvg-469:21810 sessionTimeout=90000 watcher=catalogtracker-on-hconnection-0x1ea326f9, quorum=rcc-hrl-kvg-469:21810, baseZNode=/hbase 2796 [main] INFO org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper - Process identifier=catalogtracker-on-hconnection-0x1ea326f9 connecting to ZooKeeper ensemble=rcc-hrl-kvg-469:21810 2796 [main-SendThread(rcc-hrl-kvg-469.haifa.ibm.com:21810)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server rcc-hrl-kvg-469.haifa.ibm.com/9.148.243.236:21810. Will not attempt to authenticate using SASL (unknown error) 2812 [main-SendThread(rcc-hrl-kvg-469.haifa.ibm.com:21810)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to rcc-hrl-kvg-469.haifa.ibm.com/9.148.243.236:21810, initiating session 2842 [main-SendThread(rcc-hrl-kvg-469.haifa.ibm.com:21810)] INFO org.apache.zookeeper.ClientCnxn - Session establishment complete on server rcc-hrl-kvg-469.haifa.ibm.com/9.148.243.236:21810, sessionid = 0x149b810442c0020, negotiated timeout = 90000 2883 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - hadoop.native.lib is deprecated. Instead, use io.native.lib.available

based on some help I found in the internet I have tried adding the following params to the config file, but they didn't help: mapred.remote.os=Linux mapred.job.tracker=rcc-hrl-kvg-469:9101 mapred.max.split.size=5242880 mapred.job.reuse.jvm.num.tasks=-1

levin-royl commented 9 years ago

FYI --- here is my pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"

<modelVersion>4.0.0</modelVersion>
<groupId>com.ibm</groupId>
<artifactId>titan-analytics</artifactId>
<version>0.5.1.1</version>
<name>Titan Analytics</name>
<description>Analytics of Titan Graph</description>

<dependencies>
    <dependency>
        <groupId>jdk.tools</groupId>
        <artifactId>jdk.tools</artifactId>
        <version>1.7.0_05</version>
        <scope>system</scope>
        <systemPath>${JAVA_HOME}/lib/tools.jar</systemPath>
    </dependency>

    <dependency>
        <groupId>com.thinkaurelius.titan</groupId>
        <artifactId>titan-all</artifactId>
        <version>0.5.1</version>
    </dependency>
</dependencies>

dalaro commented 9 years ago 
java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries

Do you have winutils.exe installed? I have read that the Apache distro stopped shipping it around 2.2.0 and now users have to build it, but I'm not certain. It also looks like HADOOP_HOME is unset (null should probably be your HADOOP_HOME).

levin-royl commented 9 years ago

Thanks for your reply.

Note that I am not using hadoop at all with this configuration --- I am merely using the remote linux machine as my hbase backend. Furthermore, a similar configuration worked well when I used tital-all-0.4.4 (i.e., running titan locally and connecting to a remote linux data backend).

Also, (see my comment details in #833) the issues are not the same since here I am not using titan-hadoop at all and in the other issue (#833) I am.

dalaro commented 9 years ago

@levin-royl I think you're suggesting that winutils.exe is optional. I'm not a Windows Hadoop expert and I could be mistaken, but I think winutils.exe is required. Enis Söztutar is a committer on HBase, and he posted on the hbase-user list about this:

winutils.exe is the native implementation for some of the utilities that hdfs and hadoop clients require. For the hbase client and hadoop client to work properly, you have to have it installed properly.

Also, the HBase client is not so thin. HBase's client runs a ZK client for metainformation about the cluster, and both HBase's client code and the ZK client depend on a working set of Hadoop libraries and XML config files. You certainly don't have to run Hadoop or HBase daemons (such as a regionserver) on the client, but Hadoop still has to be on the runtime classpath and configured appropriately, even just to use the HBase client.

Some thoughts on 0.4.4 vs 0.5.1. 0.4.4 was built against Hadoop 1.0.4, whereas 0.5.1 was built to work with either Hadoop 1.x.y or 2.x.y (but best-tested with 1.2.1 and 2.2.0). The Apache wiki page linked by Enis's posting implies that winutils.exe was introduced in 2.2, though I'm not certain of that. Hadoop 2.2.0 appears in the classpath snippet pasted above. I'm getting into speculation now, but perhaps this error didn't appear under Titan 0.4.4 because it used Hadoop 1.x which didn't require the native winutils.exe binary.

Based on the foregoing, I don't think this is going to work on Windows unless you install winutils.exe and set HADOOP_HOME to point the runtime at its installed path or rollback to an earlier Hadoop version that doesn't require winutils.exe (assuming that exists). Same reasoning applies to #833 since it contains an apparently identical winutils.exe exception.