HBase test cases DON'T hang with Hadoop is running.

[okram]

We were trying to compile Titan/HBase and the first test case hung again like before. I theoreized that maybe Hadoop needs to be running as I typically always have Hadoop running. So I do that and wala! -- HBase tests go through and pass successfully.

Later, I realize that Titan/HBase tests wrote data to HDFS.

2013-03-29 01:21:10.795 java[42951:1703] Unable to load realm info from SCDynamicStore Found 4 items -rw-r--r-- 1 marko supergroup 630 2013-03-27 19:28 /user/marko/FathersName.groovy -rw-r--r-- 1 marko supergroup 2028 2013-03-27 19:26 /user/marko/graph-of-the-gods.json drwxr-xr-x - marko supergroup 0 2013-03-27 19:30 /user/marko/output drwxr-xr-x - marko supergroup 0 2013-03-28 15:30 /user/marko/src ~$ hadoop fs -ls src 2013-03-29 01:21:22.253 java[42971:1703] Unable to load realm info from SCDynamicStore Found 1 items drwxr-xr-x - marko supergroup 0 2013-03-28 15:30 /user/marko/src/test ~$ hadoop fs -ls src/test 2013-03-29 01:21:25.013 java[42990:1703] Unable to load realm info from SCDynamicStore Found 1 items drwxr-xr-x - marko supergroup 0 2013-03-28 15:30 /user/marko/src/test/titan-hbase-test-data ~$ hadoop fs -ls src/test/titan-hbase-test-data 2013-03-29 01:21:31.276 java[43009:1703] Unable to load realm info from SCDynamicStore Found 8 items drwxr-xr-x - marko supergroup 0 2013-03-28 15:30 /user/marko/src/test/titan-hbase-test-data/-ROOT- drwxr-xr-x - marko supergroup 0 2013-03-28 15:30 /user/marko/src/test/titan-hbase-test-data/.META. drwxr-xr-x - marko supergroup 0 2013-03-29 00:57 /user/marko/src/test/titan-hbase-test-data/.archive drwxr-xr-x - marko supergroup 0 2013-03-29 00:57 /user/marko/src/test/titan-hbase-test-data/.logs drwxr-xr-x - marko supergroup 0 2013-03-29 00:57 /user/marko/src/test/titan-hbase-test-data/.oldlogs -rw-r--r-- 1 marko supergroup 38 2013-03-27 19:51 /user/marko/src/test/titan-hbase-test-data/hbase.id -rw-r--r-- 1 marko supergroup 3 2013-03-27 19:51 /user/marko/src/test/titan-hbase-test-data/hbase.version drwxr-xr-x - marko supergroup 0 2013-03-28 15:30 /user/marko/src/test/titan-hbase-test-data/titan

[xedin]

Interesting, that means that hadoop setup should be interfering with the way we run HBase for tests (probably can't start). Can you please elaborate on how do you have things set up on your local machine? Because in isolated environments (e.g. Travis) that couldn't happen and those tests pass successfully.

[okram]

On titan03 branch, I do:

$ cd titan-hbase
$ mvn clean install
...
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running com.thinkaurelius.titan.blueprints.HBaseBlueprintsTest
Deleteing old test directories (if any).
2013-04-25 10:59:38.964 java[54810:1703] Unable to load realm info from SCDynamicStore

It hangs there. If I start Hadoop, it works. ?!

$ start-all.sh
starting namenode, logging to /usr/local/hadoop-1.1.2/libexec/../logs/hadoop-marko-namenode-markos-macbook.out
2013-04-25 11:01:27.943 java[54951:1703] Unable to load realm info from SCDynamicStore
localhost: starting datanode, logging to /usr/local/hadoop-1.1.2/libexec/../logs/hadoop-marko-datanode-markos-macbook.out
localhost: 2013-04-25 11:01:28.939 java[55039:1b03] Unable to load realm info from SCDynamicStore
localhost: starting secondarynamenode, logging to /usr/local/hadoop-1.1.2/libexec/../logs/hadoop-marko-secondarynamenode-markos-macbook.out
starting jobtracker, logging to /usr/local/hadoop-1.1.2/libexec/../logs/hadoop-marko-jobtracker-markos-macbook.out
localhost: starting tasktracker, logging to /usr/local/hadoop-1.1.2/libexec/../logs/hadoop-marko-tasktracker-markos-macbook.out
localhost: 2013-04-25 11:01:32.564 java[55278:1b03] Unable to load realm info from SCDynamicStore
$ mvn clean install
...
Running com.thinkaurelius.titan.blueprints.HBaseBlueprintsTest
Deleteing old test directories (if any).
2013-04-25 11:05:24.620 java[56320:1703] Unable to load realm info from SCDynamicStore
Testing testVertexEqualityForSuppliedIdsAndHashCode...
Testing testGetVertexWithNull...
Testing testRemoveVertex...
Testing testRemoveVertexWithEdges...
Testing testGetNonExistantVertices...
Testing testRemoveVertexNullId...

My local machine is a MacOSX with Hadoop 1.1.2. When I run the HBase tests that hang, its with "ps aux | grep hadoop" reporting no Hadoop threads. Start Hadoop, it works............. :/

[mbroecheler]

For me the HBase tests work without having to start Hadoop. Anybody else experiencing this? @okram: Has anything changed on this?

[okram]

Nothing has changed. I tried it again last week and still the same problem. ……… :/

On May 2, 2013, at 12:24 PM, Matthias Broecheler notifications@github.com wrote:

For me the HBase tests work without having to start Hadoop. Anybody else experiencing this? @okram: Has anything changed on this?

— Reply to this email directly or view it on GitHub.

[mbroecheler]

@xedin : Any ideas for what may be going on here? For me, the tests continue to work but I am also not running Hadoop.

[okram]

Perhaps it realizes there is a $HADOOP_PREFIX environmental variable and tries to use Hadoop to do the data storage. One thing that is interesting is that if I run the tests with Hadoop running, the HBase data is in HDFS! Therefore, I think HDFS wants to use Hadoop if possible. ??..

On May 10, 2013, at 2:40 PM, Matthias Broecheler notifications@github.com wrote:

@xedin : Any ideas for what may be going on here? For me, the tests continue to work but I am also not running Hadoop.

— Reply to this email directly or view it on GitHub.

[xedin]

global HADOOP_PREFIX could be a problem, also other Hadoop/HBase related variables set in your .bashrc (.zshrc), can you try to remove those and run the tests? Also would be nice if you could remove all of the files from titan-hbase/logs directory, run the tests and attach those logs.