search

epicureanism / inspire-foss

Automatically exported from code.google.com/p/inspire-foss
GNU General Public License v3.0
0 stars 0 forks source link

deegree FSLoader error (deegree-tools-base 3.1pre11) #10

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
I am trying to get the FSLoader working again but it fails (see below). I 
remember from Bolsena that there were some changes from deegree 3.0 to 3.1 and 
that we changed stuff in the workspace. I have checked-in everything. Also 
going back to 3.0.4 does not work for me, so I am stuck. Hopefully it is a 
simple issue like changing stuff in the workspace....

Steps

1. install deegree-tools (cd tools/loader/deegree-tools: .get-deegree-tools.sh; 
install-deegree-tools.sh) 
2. try to load data in the DB e.g. using 
/etl/NL.Kadaster/AdministrativeUnits/test/load.sh (uses config 
/webapps/deegree3/src/main/webapp/WEB-INF/workspace/jdbc/inspire_blob.xml)
3. observe errors

What is the expected output? What do you see instead?
/Users/just/project/customers/kadaster/svn/project/inspire-foss/trunk/tools/load
er/../../webapps/deegree3/src/main/webapp/WEB-INF/workspace/datasources/feature/
inspire-postgis.xml 
/Users/just/project/customers/kadaster/svn/project/inspire-foss/trunk/tools/load
er/../../webapps/deegree3/src/main/webapp/WEB-INF/workspace/jdbc/inspire_blob.xm
l GML_32 USE_EXISTING gemeente-au.xml

Exception in thread "main" java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.deegree.commons.tools.ToolBox$ToolInfo.invoke(ToolBox.java:412)
    at org.deegree.commons.tools.ToolBox.main(ToolBox.java:201)
Caused by: java.lang.NullPointerException: Name is null
    at java.lang.Enum.valueOf(Enum.java:195)
    at org.deegree.tools.feature.persistence.FeatureStoreLoader$Action.valueOf(FeatureStoreLoader.java:94)
    at org.deegree.tools.feature.persistence.FeatureStoreLoader.main(FeatureStoreLoader.java:180)
    ... 6 more

Original issue reported on code.google.com by jus...@gmail.com on 18 Aug 2011 at 1:21

GoogleCodeExporter commented 9 years ago 
Could you try the web console based Loader?

Original comment by mar...@beefcafe.de on 18 Aug 2011 at 2:11

GoogleCodeExporter commented 9 years ago 
I cannot get the feature store initialized, also after upgrading pom.xml to 
deegree  3.1-pre2 (checked in): 

[17:08:33]  INFO: [FeatureStoreManager] 
--------------------------------------------------------------------------------
[17:08:33]  INFO: [FeatureStoreManager] Setting up feature stores.
[17:08:33]  INFO: [FeatureStoreManager] 
--------------------------------------------------------------------------------
[17:08:33]  INFO: [FeatureStoreManager] Setting up feature store 
'inspire-postgis' from file 'inspire-postgis.xml'...
[17:08:33] ERROR: [FeatureStoreManager] No feature store provider for namespace 
'http://www.deegree.org/datasource/feature/sql' (file: 
'file:/Users/just/project/customers/kadaster/svn/project/inspire-foss/trunk/weba
pps/deegree3/src/main/webapp/WEB-INF/workspace/datasources/feature/inspire-postg
is.xml') registered. Skipping it.
[17:08:33] ERROR: [FeatureStoreManager] Error creating feature store: No 
feature store provider for namespace 
'http://www.deegree.org/datasource/feature/sql' (file: 
'file:/Users/just/project/customers/kadaster/svn/project/inspire-foss/trunk/weba
pps/deegree3/src/main/webapp/WEB-INF/workspace/datasources/feature/inspire-postg
is.xml') registered. Skipping it.
org.deegree.feature.persistence.FeatureStoreException: No feature store 
provider for namespace 'http://www.deegree.org/datasource/feature/sql' (file: 
'file:/Users/just/project/customers/kadaster/svn/project/inspire-foss/trunk/weba
pps/deegree3/src/main/webapp/WEB-INF/workspace/datasources/feature/inspire-postg
is.xml') registered. Skipping it.
    at org.deegree.feature.persistence.FeatureStoreManager.create(FeatureStoreManager.java:188)
    at org.deegree.feature.persistence.FeatureStoreManager.init(FeatureStoreManager.java:130)
    at org.deegree.commons.config.DeegreeWorkspace.initAll(DeegreeWorkspace.java:230)
    at org.deegree.services.controller.OGCFrontController.initWorkspace(OGCFrontController.java:905)
    at org.deegree.services.controller.OGCFrontController.reload(OGCFrontController.java:943)
    at org.deegree.console.ConfigManager.applyChanges(ConfigManager.java:211)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.el.parser.AstValue.invoke(AstValue.java:191)
    at org.apache.el.MethodExpressionImpl.invoke(MethodExpressionImpl.java:276)
    at com.sun.faces.facelets.el.TagMethodExpression.invoke(TagMethodExpression.java:98)
    at javax.faces.component.MethodBindingMethodExpressionAdapter.invoke(MethodBindingMethodExpressionAdapter.java:88)
    at com.sun.faces.application.ActionListenerImpl.processAction(ActionListenerImpl.java:102)
    at javax.faces.component.UICommand.broadcast(UICommand.java:315)
    at javax.faces.component.UIViewRoot.broadcastEvents(UIViewRoot.java:775)
    at javax.faces.component.UIViewRoot.processApplication(UIViewRoot.java:1267)
    at com.sun.faces.lifecycle.InvokeApplicationPhase.execute(InvokeApplicationPhase.java:82)
    at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:101)
    at com.sun.faces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:118)
    at javax.faces.webapp.FacesServlet.service(FacesServlet.java:312)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
    at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
    at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
    at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
    at java.lang.Thread.run(Thread.java:680)
[17:08:33]  INFO: [FeatureStoreManager]

Original comment by jus...@gmail.com on 18 Aug 2011 at 3:10

GoogleCodeExporter commented 9 years ago 
What do you mean by 'upgrading pom.xml to deegree 3.1-pre2'? pre2 is rather old 
-- it does not have the generic SQLFeatureStore (hence the error message). 
Please use pre11, if you want to upgrade.

Original comment by mar...@beefcafe.de on 18 Aug 2011 at 3:37

GoogleCodeExporter commented 9 years ago 
Ok. You started with pre11, so this version should be fine.

Original comment by mar...@beefcafe.de on 18 Aug 2011 at 3:38

GoogleCodeExporter commented 9 years ago 
Does the console loader work?

Original comment by mar...@beefcafe.de on 18 Aug 2011 at 3:40

GoogleCodeExporter commented 9 years ago 
@comment 3: I mean setting dependencies in /webapps/deegree3/pom.xml for 
deegree-services and deegree-services-console. I tried 3.1-pre11 but then maven 
failed to download e.g. 
http://artefacts.deegree.org/libs-releases-local/org/deegree/deegree-services/3.
1-pre11, so I found 
http://artefacts.deegree.org/libs-releases-local/org/deegree/deegree-services/3.
1-pre2. Maybe /webapps/deegree3/pom.xml should be fixed.
I do use 3.1-pre11 for deegree-tools-base though, i.e. 
http://artefacts.deegree.org/libs-releases-local/org/deegree/deegree-tools-base/
3.1-pre11/deegree-tools-base-3.1-pre11.zip

Original comment by jus...@gmail.com on 18 Aug 2011 at 4:05

GoogleCodeExporter commented 9 years ago 
@comment 5: there is no link to a loader on the console under "feature stores", 
probably due to the error during init (see comment 2).

Original comment by jus...@gmail.com on 18 Aug 2011 at 4:09

GoogleCodeExporter commented 9 years ago 
The problems that you're experiencing with the webapp (and it's POM) is due to 
some significant changes in the build process / module organization in the last 
months.

Several months ago, we changed our approach for building deegree webservices 
and how we deal with deegree configurations (the deegree workspace).

Basically, we have a single WAR download [1] for the services now that should 
be suited for (almost) everything. It does not include any configuration. It 
has a relatively complex build (also compared to the POM in inspire-foss it 
does not include everything that's required and module names changed). The good 
thing is that it's relativaly good tested and that there's usually no need to 
built any other deegree webapp artifact manually.

The big idea why we only require a single download nowadays, is that we 
separated the workspace (config) artefacts from the service build. To support 
that, the web console got management stuff (even a REST-interface) for dealing 
with workspaces.

Best practice would be:

1. Put the inspire-foss workspace (everything below WEB-INF/workspace) into 
~/.deegree/inspire-foss/ (this is where deegree looks for workspaces by 
default). The only caveat is that we have  to merge WEB-INF/data into the 
workspace as well (and change references in config files accordingly).
2. Fire up the WAR [1], choose workspace -> "inspire-foss" -> "start".

The GML loader should be usable from the console then (feature stores -> 
loader).

IMHO, to avoid redundancy (and to allow for drop-in replacements of new deegree 
web service releases in the future), we should perform a little bit of 
restructuring in inspire-foss.

Refactor the deegree 3 webapp into a Maven project with artifact type 
"deegree-workspace". We use this for all provided workspace projects in 
deegree-trunk (and at lat/lon). It's really simple, see [2] for an example.

This way, the inspire-foss deegree 3 config could even be integrated in our 
build system and offered from the console as a downloadable workspace to every 
deegree 3 user.

I will be on a weekend trip from tomorow on, but next week, I should be able to 
help with the changes.

[1] 
http://artefacts.deegree.org/libs-releases-local/org/deegree/deegree-webservices
/3.1-pre11/deegree-webservices-3.1-pre11.war
[2] 
http://svn.wald.intevation.org/svn/deegree/deegree3/trunk/deegree-workspaces/dee
gree-workspace-inspire/

Original comment by mar...@beefcafe.de on 18 Aug 2011 at 4:23

GoogleCodeExporter commented 9 years ago 
Ok, thanks for the help Markus.

Original comment by jus...@gmail.com on 18 Aug 2011 at 4:34

GoogleCodeExporter commented 9 years ago 
Hi,

I've migrated the workspace to a deegree-workspace maven project, and at
least the console importer is working fine. You can check it out from
bitbucket:

https://bitbucket.org/hwbllmnn/inspire-foss-workspace

Feel free to move it to the svn repository if you like.

Best regards, Andreas

Original comment by stran...@hwbllmnn.de on 19 Aug 2011 at 10:13

GoogleCodeExporter commented 9 years ago 
Thanks Andreas. I got much further: cloned the WS, downloaded the .war 
http://artefacts.deegree.org/libs-releases-local/org/deegree/deegree-webservices
/3.1-pre11/deegree-webservices-3.1-pre11.war, put a symlink to the WS in 
~/.deegree, could run the war in Tomcat, saw the WS as active and load INSPIRE 
GML OK via the console. The remaining issues I have are:

- I still cannot load via the deegree-tools FSLoader with the new WS, same 
exception as before:
sunda:trunk just$ cd etl/NL.Kadaster/AdministrativeUnits/test/
sunda:test just$ ./load.sh 
/Users/just/project/customers/kadaster/svn/project/inspire-foss/trunk/tools/load
er/../../webapps/deegree/inspire-foss-workspace/src/main/workspace/datasources/f
eature/inspire-postgis.xml 
/Users/just/project/customers/kadaster/svn/project/inspire-foss/trunk/tools/load
er/../../webapps/deegree/inspire-foss-workspace/src/main/workspace/jdbc/inspire_
blob.xml GML_32 USE_EXISTING gemeente-au.xml
Exception in thread "main" java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.deegree.commons.tools.ToolBox$ToolInfo.invoke(ToolBox.java:412)
    at org.deegree.commons.tools.ToolBox.main(ToolBox.java:201)
Caused by: java.lang.NullPointerException: Name is null
    at java.lang.Enum.valueOf(Enum.java:195)
    at org.deegree.tools.feature.persistence.FeatureStoreLoader$Action.valueOf(FeatureStoreLoader.java:94)
    at org.deegree.tools.feature.persistence.FeatureStoreLoader.main(FeatureStoreLoader.java:180)
    ... 6 more
- is there a way to have a Maven project in which I can automatically download 
and run the .war ? (I am not very versed in Maven)

I have checked in all relevant stuff and the WS in SVN:
http://code.google.com/p/inspire-foss/source/browse/#svn%2Ftrunk%2Fwebapps%2Fdee
gree
Under this directory I would like to have the same situation as before (under 
webapps/deegree3): that I can run Tomcat with the deegree generic .war possibly 
with the WS preconfigured. This is also for users who checkout the project, cd 
to webapps/deegree and after some minimal setup run via Maven. Possibly even 
with zero setup, like the old inspire-demo with memory FS. Then I will remove 
the current /webapps/deegree3 dir.

The other good news is that via the console loader and demo requests OSM data 
transformed to INSPIRE is working ! Dreaming on: from now all NMCA's should 
just upload their data to OSM and then we just need a single transformation 
OSM->INSPIRE....

Original comment by jus...@gmail.com on 19 Aug 2011 at 12:45

GoogleCodeExporter commented 9 years ago 
btw Feel free to make changes in SVN. You have commit rights as 
stranger@hwbllmnn.de
The FSLoader tools are under /tools/loader/bin/fsloader.sh. Via 
/tools/loader/deegree-tools/*.sh the deegree-tools (now 3.1-pre11) can be 
downloaded and installed locally. 
The command I use to test FSLoad is 
/etl/NL.Kadaster/AdministrativeUnits/test/load.sh 
This attempts to load the GML gemeente-au.xml

Or should I maybe migrate all FSLoading to WFS-T ?

Original comment by jus...@gmail.com on 19 Aug 2011 at 12:50

GoogleCodeExporter commented 9 years ago 
Alright, I've fixed the fsloader app, switched the tool handling to 
pre12-SNAPSHOT and it's working now. Once pre12 is released, I suggest we 
switch to pre12 (should be released next week).

Original comment by stran...@hwbllmnn.de on 19 Aug 2011 at 1:44

GoogleCodeExporter commented 9 years ago 
Great ! FSLoad works again now. Closing the issue now. Thanks.

Original comment by jus...@gmail.com on 20 Aug 2011 at 10:41