Giraffa and WebUI in Standalone not working

[milandesai]

There are a couple of problems here:

1. giraffa/lib directory was renamed to giraffa/libs, but the scripts still look for the former. We should keep it lib to be consistent with standard conventions.
2. We're now using the Apache Curator library in the NamespaceProcessor, which needs to be copied to the region server class path. Perhaps it's time to make a module specifically for the NamespaceProcessor so we can collect its dependencies and separate it from the client?
3. Web UI doesn't work in standalone. We get the following error:

java.lang.IllegalStateException: No Java compiler available at org.apache.jasper.JspCompilationContext.createCompiler(JspCompilationContext.java:225) at org.apache.jasper.JspCompilationContext.compile(JspCompilationContext.java:560) at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:299) at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:315) at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:265) at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)

[pjeli]

1. Agreed.
2. I am in favor of separating client and server modules for Giraffa.
3. Did you set JAVA_HOME? Also you should check the classpath of the Standalone Giraffa servers to make sure they have jasper-compiler.jar and jasper-compiler-jdt.jar in them.

[pjeli]

I was able to make slight progress by dropping in the "ecj-4.3.1.jar" file into hbase-1.0.1/lib.

Now I see:

org.apache.jasper.JasperException: /webapps/giraffa/index.jsp(65,0) No tag "genericpage" defined in tag library imported with prefix "t"
    at org.apache.jasper.compiler.DefaultErrorHandler.jspError(DefaultErrorHandler.java:40)
    at org.apache.jasper.compiler.ErrorDispatcher.dispatch(ErrorDispatcher.java:407)
....

[shvachko]

How do we use Curator library, Why?

[pjeli]

Curator dependency came from Issue #35 where we needed a way to deal with a "Distributed Long" value in order to generate BlockId segments. This was mostly in place to set up for Issues #34 and #209 where we would need to use a "Distributed Long" to generate proper non-colliding and sequential INodeID(s).

From my experiments so far I have not seen Curator be an issue to standalone cluster.

[pjeli]

From the email with @kpelykh:

Problems:

1) During runtime jetty webapp container incorrectly unpacks webapp/giraffa resources from jar file to tmp location (for details read giraffa/giraffa-core/src/test/resources/webapps/readme.txt) , as a result webapp context can't find WEB-INF/web.xml file which contain servlet mappings (note: you shouldn't really need to have JSP compiler during run-time since all JSP pages are precompiled during build time and available under giraffa-core-0.3-SNAPSHOT.jar/org/apache/giraffa/web/*_jsp.class) In order to fix this problem, giraffa web resources should be added as a separate folder to giraffa-standalone-0.3-SNAPSHOT.tgz:

During packaging phase copy giraffa/giraffa-core/src/main/resources/webapps/giraffa to giraffa-webapps/webapps/giraffa, so in giraffa-standalone-0.3-SNAPSHOT.tgz you should end up with 4 folders: bin, conf, giraffa-webapps, libs

next step is to place giraffa-webapps on the classpath when you start giraffa:

HBASE_CLASSPATH_PREFIX=/path/to/giraffa/giraffa-webapps:/path/to/giraffa/libs/webjars-servlet-2.x-1.1.jar:/home/vagrant/giraffa/libs/ ./bin/start-giraffa.sh

you also need to put webjars-servlet-2.x-1.1.jar on the classpath, since it is used in web.xml:

WebjarsServlet org.webjars.servlet.WebjarsServlet 2

2) The second issue is related to JSP dependencies: gradle compiles JSP files using org.apache.tomcat:jasper:6.0.45 (specified in libraries.gradle), however hbase doesn't have such dependency and instead it's using org.mortbay.jetty:jsp-2.1:6.1.14. As a result when hbase tries to load Jsp files it throws the following error: javax.servlet.jsp.JspFactory.getJspApplicationContext(Ljavax/servlet/ServletContext;)Ljavax/servlet/jsp/JspApplicationContext; this is expected, since jsp file was compiled with a different version of the library (note both libraries have javax.servlet.jsp.JspFactory class) .

To fix this issue you need to use either tomcat:jasper:6.0.45 or jetty:jsp-2.1:6.1.14 for jsp compilation

Hope it helps. let me know if you have any additional questions

[pjeli]

Attaching a patch that I used to test your suggestions, @kpelykh. issue_220.patch.txt

Unfortunately even with this I am still getting the same error. :(

• I can confirm that giraffa-webapps is making its way to the classpath after my patch.
• I used jasper:6.0.45 in-place of jsp-2.1:6.1.14 in my HBase lib. I never saw the issue you talked about; I keep getting the genericpage.tag not found or defined error instead.

Perhaps I am missing something though:

• Should I remove the webapp from the giraffa-core jar?
• I saw in HBASE_CLASSPATH_PREFIX that you added everything from giraffa-standalone/giraffa/libs; was this intentional? If so, then there is no need to explicitly add the WebjarsServlet jar.

Thanks again for your help Kostya; much appreciated. Sorry for having to bug you again.

[pjeli]

Additional notes:

Sometimes when I refresh I also see this: org.apache.jasper.JasperException: Unable to load class for JSP at org.apache.jasper.JspCompilationContext.load(JspCompilationContext.java:630) at org.apache.jasper.servlet.JspServletWrapper.getServlet(JspServletWrapper.java:149) at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:340) at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:313) at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:260) at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) .... Caused by: java.lang.ClassNotFoundException: org.apache.jsp.webapps.giraffa.index_jsp at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at org.apache.jasper.servlet.JasperLoader.loadClass(JasperLoader.java:134) at org.apache.jasper.servlet.JasperLoader.loadClass(JasperLoader.java:66) at org.apache.jasper.JspCompilationContext.load(JspCompilationContext.java:628)

Which is very odd because index.jsp does not exist in org.apache.jsp package so I do not know why it would look for it there.

[shvachko]

During startup I see that webapp/giraffa is copied into /tmp/Jetty_0_0_0_0_40010_giraffa____evh34o/webapps/webapp/giraffa with webapps\ twice. Then the webUI is looking into this directory, but it is messed up, imho. HBase and HDFS use the same HttpServer2, but their webapps are not in '/tmp'. Is it helpful?

[shvachko]

Made some progress, I hope. I added https://repo1.maven.org/maven2/tomcat/jasper-compiler-jdt/5.5.23/jasper-compiler-jdt-5.5.23.jar into hbase-1.0.1.1/lib. Now I do not see java.lang.IllegalStateException: No Java compiler available. I have a different exception instead, saying org.apache.jasper.JasperException: /giraffa/index.jsp(65,0) No tag "genericpage" defined in tag library imported with prefix "t" This seems to be giraffa exception, which we can fix. So the compiler is missing indeed. I think that the web UI uses HBase libraries with jasper 5.5.23. But it never looks into giraffa/lib, which would have used jasper 6.0.45. So there are some classpath issues along with packaging. Anybody looked into how it worked before, when we used maven? Can you guys pick up from here?

[pjeli]

Hey Konst,

Thanks for taking a look unfortunately you've only just caught up to where I am stuck at now. I looked into it before but there have been a number of other changes since then and one of the biggest is that we now use HttpServer2. I had found a way to circumvent the issues that came with that but now I cannot resolve the issue you are seeing; which is where I am stuck at too. Let me know if you find anything.

[pjeli]

Spoke to Konst on the phone today to try to knock this out. He recommended I try to downgrade the jasper dependency to '5.5.23'. I did, and added the set the following dependencies:

"javax.el:javax.el-api:3.0.0",
"tomcat:jasper-compiler:5.5.23",
"tomcat:jasper-compiler-jdt:5.5.23"

The result is that I am unable to compile due to an NPE:

[ant:jasper] java.lang.NullPointerException
[ant:jasper]    at org.apache.taglibs.standard.tlv.JstlBaseTLV.validate(JstlBaseTLV.java:140)
[ant:jasper]    at org.apache.taglibs.standard.tlv.JstlFmtTLV.validate(JstlFmtTLV.java:96)
[ant:jasper]    at org.apache.jasper.compiler.TagLibraryInfoImpl.validate(TagLibraryInfoImpl.java:913)
[ant:jasper]    at org.apache.jasper.compiler.Validator.validateXmlView(Validator.java:1891)
[ant:jasper]    at org.apache.jasper.compiler.Validator.validate(Validator.java:1859)
[ant:jasper]    at org.apache.jasper.compiler.Compiler.generateJava(Compiler.java:210)
[ant:jasper]    at org.apache.jasper.compiler.Compiler.compile(Compiler.java:435)
[ant:jasper]    at org.apache.jasper.JspCompilationContext.compile(JspCompilationContext.java:608)

[herrlich]

I tried to downgrade the jasper dependency of Giraffa to 5.5.23, it compiles successfully. But when I deployed the new files to the cluster, I still got this exception: /webapps/giraffa/index.jsp(65,0) No tag "genericpage" defined in tag library imported with prefix "t"

[pjeli]

I propose we re-write the WebUI using SparkJava. It is a very simple way to write a web interface. I have been using it recently and it is really quick to start and go with. It uses Jetty underneath and can handle JSP though I recommend we move to pure HTML and Javascript + AJAX.

[herrlich]

In fact, I also support re-writing a new WebUI.....

[shvachko]

Finally some real progress here. I made giraffa WebUI work with the following 2 methods. The problem turned out to be with the classpass. Whatever we set CLASSPATH to in giraffa-config.sh it is not populated down to HBase. So

Method 1 (Classpath)

1. Add the following line at the end of giraffa-config.sh HADOOP_CLASSPATH=${GIRAFFA_HOME}:$CLASSPATH where ${GIRAFFA_HOME} specifies the location of webapps for giraffa.
2. Copy 9 jars from giraffa/lib/*6.0.45.jar to hbase/lib. Note DO NOT copy giraffa-core.jar
3. Run bin/start-giraffa.sh and check the UI.

Method 2 (WEB-INF/lib)

1. Add the following line at the end of giraffa-config.sh `HADOOP_CLASSPATH=${GIRAFFA_HOME}:${GIRAFFA_HOME}/lib/giraffa-core-0.3-SNAPSHOT.jar, here we only add the location ofwebappsandgiraffa-core.jar``, but no other dependencies.
2. Copy all jars from giraffa/lib but excluding hadoop*.jar and hbase*.jar into giraffa/webapps/giraffa/WEB_INF/lib
3. Run bin/start-giraffa.sh and check the UI.

Method 1 requires mixing giraffa dependencies - the 6.0.45 jars - with hbase libraries. Which we can fix by downgrading tomcat from 6.0.45 to 5.5.23. I believe Chris did this on an internal branch. Method 2 needs to pack some libraries inside WEB_INF/lib. Which should be specified in gradle script.

LMK what you think

[pjeli]

1. Did you try Method 1 with downgrading the tomcat jars? I believe I tested it and got a failure.
2. If Method 1 works with downgrading the tomcat jars then I believe it's just a simple config change and the easiest fix. If not then we should do Method 2.

I'll test these myself right now but do let me know what you did exactly as it sounds like Method 1 is more a suggestion than an actual fix.

[shvachko]

No I did not try Method 1 with downgraded tomcat. You probably gotten a failure because giraffa webapps was not in the classpath, as in my description. If you have a patch for downgraded tomcat I can verify if method 1 works with it. Why would it not if all jars are already on HBase classpath?

[herrlich]

Yeah I downgraded tomcat from 6.0.45 to 5.5.23 in the last week of my internship (without modifying the classpath), and at that time I still could not make the web UI work. I will try to work on method 1 later this week to see if I can make that work. (The system of my computer crashed yesterday:(, so I need some time to reinstall that and try to deploy a single node Giraffa later.)

[shvachko]

Chris, could you please share your downgrade changes with me. Any form would work: a patch or a pull request. Want to try it too.

[herrlich]

Hi Konstantin, I checked my branch just now, and it was so sad to find that I forgot to synchronize my changes when I left... But I read the code again, I think I just changed the 55th line "org.apache.tomcat:jasper:6.0.45" to be the version we want in libraries/libraries.gradle, before I built Giraffa. And maybe I also added the dependency of "jasper-compiler-jdt:5.5.23" at that time, but I'm not very sure about this.

[shvachko]

OK. I downgraded jasper by modifying libraries.gradle with this

libraries.jasper = [
    "tomcat:jasper-compiler-jdt:5.5.23",
    "org.mortbay.jetty:jsp-api-2.1:6.1.14",
    "taglibs:standard:1.1.2"
]

Then I changed giraffa-config.sh as in Method 2:

HADOOP_CLASSPATH=${GIRAFFA_HOME}:${GIRAFFA_HOME}/lib/giraffa-core-0.3-SNAPSHOT.jar

Without any changes to HBASE_HOME/lib the WebUI still does not work. Had to make 2 changes.

1. I removed jasper-runtime-5.5.23.jar because it conflicts with jsp-2.1-6.1.14. The former is first on the HBase class path and PageContextImpl is used from there rather than from the latter causing NoSuchMethodException.
2. I also had to add bootstrap-2.3.1.jar to HBASE_HOME/lib. Then giraffa UI worked.

So I think we should go with method 2, that is place all necessary jars into WEB_INF/lib in order to make us independent of hbase and hadoop dependencies.

The alternative to adding and deleting jars is to add the following at the of giraffa-config.sh

HBASE_CLASSPATH=${GIRAFFA_HOME}:${GIRAFFA_HOME}/conf
HBASE_CLASSPATH=${HBASE_CLASSPATH}:${GIRAFFA_HOME}/lib/giraffa-core-0.3-SNAPSHOT.jar
HBASE_CLASSPATH=${HBASE_CLASSPATH}:${GIRAFFA_HOME}/lib/curator-client-2.8.0.jar
HBASE_CLASSPATH=${HBASE_CLASSPATH}:${GIRAFFA_HOME}/lib/curator-framework-2.8.0.jar
HBASE_CLASSPATH=${HBASE_CLASSPATH}:${GIRAFFA_HOME}/lib/curator-recipes-2.8.0.jar
HBASE_CLASSPATH=${HBASE_CLASSPATH}:${GIRAFFA_HOME}/lib/commons-fileupload-1.2.2.jar
HBASE_CLASSPATH=${HBASE_CLASSPATH}:${GIRAFFA_HOME}/lib/bootstrap-2.3.1.jar
HBASE_CLASSPATH=${HBASE_CLASSPATH}:${GIRAFFA_HOME}/lib/jasny-bootstrap-2.3.0-j5.jar
HBASE_CLASSPATH=${HBASE_CLASSPATH}:${GIRAFFA_HOME}/lib/jquery-1.9.0.jar
HBASE_CLASSPATH=${HBASE_CLASSPATH}:${GIRAFFA_HOME}/lib/jquery-form-3.28.0-2013.02.06.jar
HBASE_CLASSPATH=${HBASE_CLASSPATH}:${GIRAFFA_HOME}/lib/standard-1.1.2.jar
HBASE_CLASSPATH=${HBASE_CLASSPATH}:${GIRAFFA_HOME}/lib/datatables-1.9.4.jar
export HBASE_CLASSPATH

HBASE_CLASSPATH_PREFIX=${GIRAFFA_HOME}/lib/jsp-2.1-6.1.14.jar
export HBASE_CLASSPATH_PREFIX

Just for note keeping if we decide to downgrade.

[shvachko]

Could you guys check this. It worked for me.

[pjeli]

So good work so far @shvachko but more is needed. We uncovered that the Curator jars need to be on HBase classpath.

That alone won't be enough. We also need to fix some bugs.

1. GiraffaHbaseServlet's keyFactory variable is never set. On line 77 we need to set it there. This breaks the HTable browser.
2. I see the following coming back when trying to use the Filesystem Browser:

(tried to access field org.apache.hadoop.hdfs.DFSClient.namenode from class org.apache.hadoop.hdfs.GiraffaClient)

[pjeli]

Dived a little deeper this morning. Even if we set the keyFactory variable on line 77 we end up with an Exception saying FullPathRowKeyFactory cannot be cast to RowKeyFactory.

It took me a short while to understand our issue is that we are dealing with multiple class-loaders. I'm not sure where the other class-loaders are coming from but I'm guessing from the WEBUI.

This would also explain the "tried to access field" issue we are running into.

[pjeli]

Trying to load all the giraffa/lib/*.jar files into WEB-INF is not the right approach. You will get into all sorts of weird issues due to the multiple class-loaders. I see now that all Coprocessors get their own Classloader via HBase code.

We actually need to minimalize what goes into WEB-INF.

[pjeli]

I tried to remove the giraffa.jar from WEB-INF lib in hopes that it would use the HBASE_CLASSPATH and ClassLoader instead but now I get:

java.lang.NoClassDefFoundError: org/apache/jasper/runtime/AnnotationHelper
    at org.apache.giraffa.web.htable_jsp._jspx_meth_t_005fgenericpage_005f0(htable_jsp.java:121)
    at org.apache.giraffa.web.htable_jsp._jspService(htable_jsp.java:100)
    at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:98)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
    at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)

I am stuck again.

[shvachko]

Yes class loading is messed up. I guess introducing WEB-INF/lib is not a good idea. We should fix the CLASSPATH for starting hbase. It may mean having our own start-hbase script.

[shvachko]

I updated the branch with new changes. Please verify. I essentially fixed the classpath setting for the web UI. The following jars should be on the path

HBASE_CLASSPATH=${GIRAFFA_HOME}:${GIRAFFA_HOME}/conf
HBASE_CLASSPATH=${HBASE_CLASSPATH}:${GIRAFFA_HOME}/lib/giraffa-core-0.3-SNAPSHOT.jar
HBASE_CLASSPATH=${HBASE_CLASSPATH}:${GIRAFFA_HOME}/lib/curator-client-2.8.0.jar
HBASE_CLASSPATH=${HBASE_CLASSPATH}:${GIRAFFA_HOME}/lib/curator-framework-2.8.0.jar
HBASE_CLASSPATH=${HBASE_CLASSPATH}:${GIRAFFA_HOME}/lib/curator-recipes-2.8.0.jar

HBASE_CLASSPATH=${HBASE_CLASSPATH}:${GIRAFFA_HOME}/lib/annotations-api-6.0.45.jar
HBASE_CLASSPATH=${HBASE_CLASSPATH}:${GIRAFFA_HOME}/lib/bootstrap-2.3.1.jar
HBASE_CLASSPATH=${HBASE_CLASSPATH}:${GIRAFFA_HOME}/lib/catalina-6.0.45.jar
HBASE_CLASSPATH=${HBASE_CLASSPATH}:${GIRAFFA_HOME}/lib/commons-fileupload-1.2.2.jar
HBASE_CLASSPATH=${HBASE_CLASSPATH}:${GIRAFFA_HOME}/lib/coyote-6.0.45.jar
HBASE_CLASSPATH=${HBASE_CLASSPATH}:${GIRAFFA_HOME}/lib/datatables-1.9.4.jar
HBASE_CLASSPATH=${HBASE_CLASSPATH}:${GIRAFFA_HOME}/lib/ecj-4.3.1.jar
HBASE_CLASSPATH=${HBASE_CLASSPATH}:${GIRAFFA_HOME}/lib/el-api-6.0.45.jar
HBASE_CLASSPATH=${HBASE_CLASSPATH}:${GIRAFFA_HOME}/lib/jasny-bootstrap-2.3.0-j5.jar
HBASE_CLASSPATH=${HBASE_CLASSPATH}:${GIRAFFA_HOME}/lib/jquery-1.9.0.jar
HBASE_CLASSPATH=${HBASE_CLASSPATH}:${GIRAFFA_HOME}/lib/jquery-form-3.28.0-2013.02.06.jar
HBASE_CLASSPATH=${HBASE_CLASSPATH}:${GIRAFFA_HOME}/lib/jsp-api-6.0.45.jar
HBASE_CLASSPATH=${HBASE_CLASSPATH}:${GIRAFFA_HOME}/lib/jstl-1.1.2.jar
HBASE_CLASSPATH=${HBASE_CLASSPATH}:${GIRAFFA_HOME}/lib/juli-6.0.45.jar
HBASE_CLASSPATH=${HBASE_CLASSPATH}:${GIRAFFA_HOME}/lib/servlet-api-6.0.45.jar
HBASE_CLASSPATH=${HBASE_CLASSPATH}:${GIRAFFA_HOME}/lib/standard-1.1.2.jar
HBASE_CLASSPATH=${HBASE_CLASSPATH}:${GIRAFFA_HOME}/lib/webjars-servlet-2.x-1.1.jar
export HBASE_CLASSPATH

HBASE_CLASSPATH_PREFIX=${GIRAFFA_HOME}/lib/jasper-6.0.45.jar
HBASE_CLASSPATH_PREFIX=${HBASE_CLASSPATH_PREFIX}:${GIRAFFA_HOME}/lib/jasper-el-6.0.45.jar
export HBASE_CLASSPATH_PREFIX
export HBASE_DISABLE_HADOOP_CLASSPATH_LOOKUP=true

The script does it in a more generic way. Both HTable and File Browsers work now. Upload and mkdir don't work on File Browser, but I don't think it is related to classpath, we can fix it later.

[pjeli]

I get the following in my HBase master and no WebUI shows up:

2016-10-03 11:17:06,259 ERROR [lm-sjn-00877839:16020.activeMasterManager] master.HMaster: Coprocessor postStartMaster() hook failed
java.io.IOException: Unable to initialize WebAppContext
        at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:849)
        at org.apache.giraffa.web.GiraffaWebServer.start(GiraffaWebServer.java:55)
        at org.apache.giraffa.web.GiraffaWebObserver.postStartMaster(GiraffaWebObserver.java:72)
        at org.apache.hadoop.hbase.master.MasterCoprocessorHost$62.call(MasterCoprocessorHost.java:755)
        at org.apache.hadoop.hbase.master.MasterCoprocessorHost.execOperation(MasterCoprocessorHost.java:970)
        at org.apache.hadoop.hbase.master.MasterCoprocessorHost.postStartMaster(MasterCoprocessorHost.java:751)
        at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:744)
        at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:165)
        at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1428)
        at java.lang.Thread.run(Thread.java:745)
Caused by: org.mortbay.util.MultiException[java.lang.NullPointerException, java.lang.NullPointerException, java.lang.NullPointerException, java.lang.NullPointerException, java.lang.NullPointerException, java.lang.NullPointerException]
        at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:707)
        at org.mortbay.jetty.servlet.Context.startContext(Context.java:140)
        at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282)
        at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:518)
        at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499)
        at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
        at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152)
        at org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156)
        at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
        at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
        at org.mortbay.jetty.Server.doStart(Server.java:224)
        at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
        at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:827)
        ... 9 more

[pjeli]

So it seems I was getting the NPEs I posted above because I had an additional change on top of @shvachko's where I modified line 77 of GiraffaHbaseServlet to have:

keyFactory = RowKeyFactoryProvider.createFactory(conf, null);

I thought this was necessary because otherwise the keyFactory variable in that class is unused. Seems that setting that variable is the cause of the NPEs -- though I cannot explain why.

Anyway; I can confirm now I can see the HBase browser working. The File browser is failing with:

Failed to load resource: the server responded with a status of 503 (java.lang.ClassCastException: org.apache.hadoop.fs.LocalFileSystem cannot be cast to org.apache.giraffa.GiraffaFileSystem)

Which I think @shvachko already acknowledged would fix in a different issue.

So I think it's almost there just need to decide if maybe we should remove that unused variable then?

[milandesai]

That is odd - the variable is used, but isn't assigned, so in theory not modifying line 77 should produce a NPE. If setting it is causing NPEs, we should investigate because something's not right. In any case, I am getting an HTTP Error 500 (java.lang.IllegalStateException: No Java compiler available)

[milandesai]

I see from earlier comments that the compiler is fixed by adding the jasper compiler jar from the maven repo to the hbase lib directory. Can we have a summary comment that outlines the additional steps needed to get this to work? Right now they seem to be spread out in the comments above. Unless the jasper jar is all that is needed.

[pjeli]

@milandesai, start from a fresh installation. You need to rebuild the tar and deploy it and probably clean out your HBase installation.

I ran into other oddities as well until I started from scratch.

[milandesai]

Yep it was from a fresh installation. Are there any additional steps I needed to take besides just launching Giraffa as usual? I.e. copying certain jars over from giraffa/lib to hbase/lib? Right now I'm copying giraffa-core.jar and curator-*.jar. I think I read somewhere above I'm not supposed to copy giraffa-core.jar anymore?

[pjeli]

Oh you are not supposed to copy over any jars anymore. The intent is to set HBASE_CLASSPATH instead. I think Konst talked about it in a post here.

~Plamen

On Oct 3, 2016, at 7:47 PM, Milan Desai notifications@github.com wrote:

Yep it was from a fresh installation. Are there any additional steps I needed to take besides just launching Giraffa as usual? I.e. copying certain jars over from giraffa/lib to hbase/lib? Right now I'm copying giraffa-core.jar and curator-*.jar. I think I read somewhere above I'm not supposed to copy giraffa-core.jar anymore?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[milandesai]

Tried again without copying any jars, Jsp compiler error is gone, getting: /webapps/giraffa/index.jsp(65,0) No tag "genericpage" defined in tag library imported with prefix "t"

[shvachko]

Milan, yeh, you should not add any additional jars anywhere. Plamen, the `ClassCastException`` is a configuration problem. Here are the steps

1. Build giraffa distro from my branch: ./gradlew clean build tar -x test. Untar giraffa-standalone-0.3-SNAPSHOT.tgz
2. Untar fresh hbase-1.0.1.1
3. Configure hbase-1.0.1.1/conf dir by
   1. Add giraffa-default.xml
   2. Add giraffa-site.xml specifying fs.defaultFS = grfs:/// fs.grfa.impl = org.apache.giraffa.GiraffaFileSystem fs.AbstractFileSystem.grfa.impl = org.apache.giraffa.GiraffaFs
   3. Set up hbase-site.xml hbase.rootdir = hdfs://localhost:8020/hbase hbase.tmp.dirl = /your/local/dir hbase.coprocessor.master.classes = org.apache.giraffa.web.GiraffaWebObserver hbase.cluster.distributed = false hbase.zookeeper.quorum = localhost
4. Set up hadoop-2.5.1 as usually
5. Run bin/start-giraffa.sh. Check http://localhost:40010/

Could you try this, guys. I'll update Wiki correspondingly once this is committed.

[milandesai]

Web UI is working for me with the above instructions! About putting a giraffa-site.xml and giraffa-default.xml into hbase/conf. What if we created a webconf directory in giraffa and added that to the GIRAFFA_WEB_CLASSPATH? Then we can avoid copying anything to hbase.

Secondly, before we commit, we should still probably fix the potential NullPointerException in GiraffaHbaseServlet.

[shvachko]

I would prefer to commit classpath changes here and work on optimizing configuration as a next step. Which potential NPE?

[milandesai]

Sounds good. The field "keyFactory" is never set in GiraffaHBaseServlet but it is used in the methods (though Plamen mentioned setting it seems to cause problems). I just realized it's not a part of your change though and can be fixed elsewhere. +1 on the current changes.

[pjeli]

@shvachko I can confirm that adding the giraffa-site.xml and giraffa-default.xml to the HBase conf dir fixed the UI entirely. I can access all the browsers and utilize them.

-Last question -- has this been tested with the MiniCluster demo?- Answered my own question by running the demo -- yes it works. +1 Konst. Thank you for the work here and un-stuck'ing us.

[shvachko]

I just committed this. Good to have the gang back.