Closed KoalaGeo closed 4 years ago
archaeogeek
commented
4 years ago Hi,
I implement it myself in docker (3.10.x) by including it in a war file and then deploying in a manner similar to the notes on deploying with a custom war here: https://github.com/geonetwork/docker-geonetwork/tree/master/4.0.0-alpha.1. The benefit of doing it that way is that I can test the build with jetty before building the war file and deploying it. I haven't tried any other approaches though.
All the best
Jo
KoalaGeo
commented
4 years ago Cheers for that, I'll have a go....:-)
archaeogeek
commented
4 years ago Great- I'll close this then!
Jo
KoalaGeo
commented
4 years ago Hi Jo,
Sorry please could you checkout my gist
The files are copying in to the container but I still can't see any changes to Metadata & Templates > Standards....
Wanted to avoid building the war file route.
Cheers
archaeogeek
commented
4 years ago I don't know that much yet about implementing any geonetwork customisations in docker other than the approach I'm using, but when I look at your gist there seems to be a discrepancy between the data_dir you're setting and the one you're copying the files into- I would check when the system is up and running where it's actually getting the schema from.
KoalaGeo
commented
4 years ago Hi,
Got there in the end! https://gist.github.com/KoalaGeo/b31666235b3784086bc7f8fb8643d129
Cheers :-)
KoalaGeo
commented
4 years ago The templates are all in there however when try to use to create a record theres "Create metadata error"
archaeogeek
commented
4 years ago There are a number of reasons why that occurs- you need to find the log files to figure out which problem it is!
KoalaGeo
commented
4 years ago I'm getting the below, no prob the creating metadata using the other templates. It also won't let me duplicate the Gemini templates to make new records:
Impl.java:139) at org.hibernate.internal.AbstractSessionImpl$NonContextualJdbcConnectionAccess.obtainConnection(AbstractSessionImpl.java:380) at org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.obtainConnection(LogicalConnectionImpl.java:228) ... 40 more Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:607) at org.postgresql.core.PGStream.<init>(PGStream.java:64) at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:147) ... 54 more
archaeogeek
commented
4 years ago I've seen this before but with two different causes. I'll come back to you with details, but it will take a while, sorry!
archaeogeek
commented
4 years ago @KoalaGeo OK it's to do with the jar file- the one in the repository is slightly outdated. I'll have to rebuild it. I'll let you know when it's done.
archaeogeek
commented
4 years ago @KoalaGeo I've committed a new jar file target/schema-iso19139.gemini23-3.7.jar on the 310newjar branch. Could you try deploying that one and let me know how you get on please?
KoalaGeo
commented
4 years ago Still not working at expected. Can import templates but not create records.
Last 100 lines of logs below:
at net.sf.saxon.expr.Expression.evaluateItem(Expression.java:352) at net.sf.saxon.expr.ExpressionTool.evaluate(ExpressionTool.java:296) at net.sf.saxon.expr.LetExpression.eval(LetExpression.java:339) at net.sf.saxon.expr.LetExpression.processLeavingTail(LetExpression.java:548) at net.sf.saxon.instruct.Choose.processLeavingTail(Choose.java:686) at net.sf.saxon.instruct.Instruction.process(Instruction.java:93) at net.sf.saxon.expr.LetExpression.process(LetExpression.java:378) at net.sf.saxon.instruct.ForEach.processLeavingTail(ForEach.java:321) at net.sf.saxon.instruct.Block.processLeavingTail(Block.java:556) at net.sf.saxon.instruct.Template.applyLeavingTail(Template.java:203) at net.sf.saxon.instruct.ApplyTemplates.applyTemplates(ApplyTemplates.java:345) at net.sf.saxon.instruct.ApplyTemplates.apply(ApplyTemplates.java:210) at net.sf.saxon.instruct.ApplyTemplates.processLeavingTail(ApplyTemplates.java:174) at net.sf.saxon.instruct.Block.processLeavingTail(Block.java:556) at net.sf.saxon.instruct.Block.processLeavingTail(Block.java:556) at net.sf.saxon.instruct.Instruction.process(Instruction.java:93) at net.sf.saxon.instruct.ForEach.processLeavingTail(ForEach.java:321) at net.sf.saxon.expr.LetExpression.processLeavingTail(LetExpression.java:557) at net.sf.saxon.instruct.Block.processLeavingTail(Block.java:556) at net.sf.saxon.instruct.Template.applyLeavingTail(Template.java:203) at net.sf.saxon.instruct.ApplyTemplates.applyTemplates(ApplyTemplates.java:345) at net.sf.saxon.instruct.ApplyTemplates.apply(ApplyTemplates.java:210) at net.sf.saxon.instruct.ApplyTemplates.processLeavingTail(ApplyTemplates.java:174) at net.sf.saxon.instruct.Block.processLeavingTail(Block.java:556) at net.sf.saxon.instruct.Block.processLeavingTail(Block.java:556) at net.sf.saxon.expr.LetExpression.processLeavingTail(LetExpression.java:557) at net.sf.saxon.instruct.Block.processLeavingTail(Block.java:556) at net.sf.saxon.instruct.Instruction.process(Instruction.java:93) at net.sf.saxon.instruct.ElementCreator.processLeavingTail(ElementCreator.java:296) at net.sf.saxon.instruct.Template.applyLeavingTail(Template.java:203) at net.sf.saxon.instruct.ApplyTemplates.applyTemplates(ApplyTemplates.java:345) at net.sf.saxon.Controller.transformDocument(Controller.java:1807) at net.sf.saxon.Controller.transform(Controller.java:1621) at org.fao.geonet.utils.Xml.transform(Xml.java:515) at org.fao.geonet.utils.Xml.transform(Xml.java:388) at org.fao.geonet.kernel.search.SearchManager.getIndexFields(SearchManager.java:1038) at org.fao.geonet.kernel.search.SearchManager.buildIndexDocument(SearchManager.java:693) at org.fao.geonet.kernel.search.SearchManager.index(SearchManager.java:634) at org.fao.geonet.kernel.datamanager.base.BaseMetadataIndexer.indexMetadata(BaseMetadataIndexer.java:608) at org.fao.geonet.kernel.DataManager.indexMetadata(DataManager.java:175) at org.fao.geonet.kernel.IndexMetadataTask.run(IndexMetadataTask.java:114) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2020-10-01 14:39:12,419 ERROR [geonetwork] - Failed fetching url: http://ogcdev.bgs.ac.uk/geonetwork/srv/en/csw?SERVICE=CSW&REQUEST=GetRecordById&ID=9df8df52-d788-37a8-e044-0003ba9b0d98&elementSetName=full&OutputSchema=http://www.isotc211.org/2005/gmd& java.io.FileNotFoundException: http://ogcdev.bgs.ac.uk/geonetwork/srv/en/csw?SERVICE=CSW&REQUEST=GetRecordById&ID=9df8df52-d788-37a8-e044-0003ba9b0d98&elementSetName=full&OutputSchema=http://www.isotc211.org/2005/gmd& at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1896) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498) at org.fao.geonet.util.XslUtil.getUrlContent(XslUtil.java:878) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at net.sf.saxon.functions.ExtensionFunctionCall.invokeMethod(ExtensionFunctionCall.java:690) at net.sf.saxon.functions.ExtensionFunctionCall.call(ExtensionFunctionCall.java:343) at net.sf.saxon.functions.ExtensionFunctionCall.iterate(ExtensionFunctionCall.java:224) at net.sf.saxon.expr.Expression.evaluateItem(Expression.java:352) at net.sf.saxon.expr.ExpressionTool.evaluate(ExpressionTool.java:296) at net.sf.saxon.expr.LetExpression.eval(LetExpression.java:339) at net.sf.saxon.expr.LetExpression.processLeavingTail(LetExpression.java:548) at net.sf.saxon.instruct.Choose.processLeavingTail(Choose.java:686) at net.sf.saxon.instruct.Instruction.process(Instruction.java:93) at net.sf.saxon.expr.LetExpression.process(LetExpression.java:378) at net.sf.saxon.instruct.ForEach.processLeavingTail(ForEach.java:321) at net.sf.saxon.instruct.Block.processLeavingTail(Block.java:556) at net.sf.saxon.instruct.Template.applyLeavingTail(Template.java:203) at net.sf.saxon.instruct.ApplyTemplates.applyTemplates(ApplyTemplates.java:345) at net.sf.saxon.instruct.ApplyTemplates.apply(ApplyTemplates.java:210) at net.sf.saxon.instruct.ApplyTemplates.processLeavingTail(ApplyTemplates.java:174) at net.sf.saxon.instruct.Block.processLeavingTail(Block.java:556) at net.sf.saxon.instruct.Block.processLeavingTail(Block.java:556) at net.sf.saxon.instruct.Instruction.process(Instruction.java:93) at net.sf.saxon.instruct.ForEach.processLeavingTail(ForEach.java:321) at net.sf.saxon.expr.LetExpression.processLeavingTail(LetExpression.java:557) at net.sf.saxon.instruct.Block.processLeavingTail(Block.java:556) at net.sf.saxon.instruct.Template.applyLeavingTail(Template.java:203) at net.sf.saxon.instruct.ApplyTemplates.applyTemplates(ApplyTemplates.java:345) at net.sf.saxon.instruct.ApplyTemplates.apply(ApplyTemplates.java:210) at net.sf.saxon.instruct.ApplyTemplates.processLeavingTail(ApplyTemplates.java:174) at net.sf.saxon.instruct.Block.processLeavingTail(Block.java:556) at net.sf.saxon.instruct.Block.processLeavingTail(Block.java:556) at net.sf.saxon.expr.LetExpression.processLeavingTail(LetExpression.java:557) at net.sf.saxon.instruct.Block.processLeavingTail(Block.java:556) at net.sf.saxon.instruct.Instruction.process(Instruction.java:93) at net.sf.saxon.instruct.ElementCreator.processLeavingTail(ElementCreator.java:296) at net.sf.saxon.instruct.Template.applyLeavingTail(Template.java:203) at net.sf.saxon.instruct.ApplyTemplates.applyTemplates(ApplyTemplates.java:345) at net.sf.saxon.Controller.transformDocument(Controller.java:1807) at net.sf.saxon.Controller.transform(Controller.java:1621) at org.fao.geonet.utils.Xml.transform(Xml.java:515) at org.fao.geonet.utils.Xml.transform(Xml.java:388) at org.fao.geonet.kernel.search.SearchManager.getIndexFields(SearchManager.java:1038) at org.fao.geonet.kernel.search.SearchManager.buildIndexDocument(SearchManager.java:693) at org.fao.geonet.kernel.search.SearchManager.index(SearchManager.java:634) at org.fao.geonet.kernel.datamanager.base.BaseMetadataIndexer.indexMetadata(BaseMetadataIndexer.java:608) at org.fao.geonet.kernel.DataManager.indexMetadata(DataManager.java:175) at org.fao.geonet.kernel.IndexMetadataTask.run(IndexMetadataTask.java:114) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
archaeogeek
commented
4 years ago You might need to see what errors you get in chrome dev console when you explicitly trigger that error- I think that will be more useful than the java logs in this instance.
Or you might need to go further back in the log files, from memory there should be some xsl errors cropping up but the java logs are ridiculously verbose...
KoalaGeo
commented
4 years ago So there's two 404's on loadng the create record page, and a 400 error when actually click create.
Thanks for your patience on this!
archaeogeek
commented
4 years ago You can ignore the 404s, but we could do with knowing the response to the 400 error. I think I know what it's going to say already but the new jar file in the new branch should have sorted it :thinking:
KoalaGeo
commented
4 years ago This bit?
archaeogeek
commented
4 years ago I think it's the bit in the actual response tab. However, from memory I think it's also in the java logs, but you need to go quite a long way back for it...
KoalaGeo
commented
4 years ago 
archaeogeek
commented
4 years ago Dammit, I think what we need to see is the java log generated if you open the url that triggers that response in a new tab (eg the blah/records/duplicate?blah one from your image showing the request headers. However can you also check the size of the jar file that you've got, just in case it's not the right one still?
KoalaGeo
commented
4 years ago Added that output in comment - https://gist.github.com/KoalaGeo/b31666235b3784086bc7f8fb8643d129
I've downloaded & copied in again - just redeploying
archaeogeek
commented
4 years ago OK, it's definitely the same issue that I had, and that I think you should have the fix for. The key is that the more recent jar file is 12.1kb whereas the incorrect one is only 11.5kb. For information, I might not be able to help much tomorrow, then I'm on leave all next week, so if we can't fix this today you might want to put it to one side for a bit!
KoalaGeo
commented
4 years ago From the newjar branch the one I've got is 11.8kb....
KoalaGeo
commented
4 years ago Still getting same error on after deploying. Maybe I need to learn the build from source war route...
archaeogeek
commented
4 years ago That's the right one- 12,095 is what mine reports in bytes. So... when you redeploy are you rebuilding the image from scratch- is it possible that the file in your image isn't getting updated?
KoalaGeo
commented
4 years ago I'm doing: `FROM geonetwork:3.10.3-postgres
ENV POSTGRES_DB_HOST=postgres ENV POSTGRES_DB_PORT=5432 ENV POSTGRES_DB_USERNAME=postgres ENV POSTGRES_DB_PASSWORD=postgres ENV POSTGRES_DB_NAME=geonetwork ENV DATA_DIR=$CATALINA_HOME/webapps/geonetwork/WEB-INF/data
RUN mkdir $DATA_DIR/config/schema_plugins/iso19139.gemini23/ COPY geonetwork_customisation/iso19139.gemini23-3.10.x/src/main/plugin/iso19139.gemini23 $DATA_DIR/config/schema_plugins/iso19139.gemini23/ COPY geonetwork_customisation/iso19139.gemini23-3.10.x/target/schema-iso19139.gemini23-3.7.jar $CATALINA_HOME/webapps/geonetwork/WEB_INF/lib/
CMD ["catalina.sh", "run"]`
KoalaGeo
commented
4 years ago Will cont. in a couple of weeks, thanks again for your help.
archaeogeek
commented
4 years ago OK sorry we couldn't get it sorted now. My gut feeling, by the way, is that the actual docker image isn't getting updated with the new file, but I'm quite new to docker myself so I was just looking it up to see if I could prove my hunch either way...
archaeogeek
commented
4 years ago Did you get this working? afaict it should be fine, so the only thing I can think of doing is replicating your build process exactly to see what happens...
KoalaGeo
commented
4 years ago I'm going to have a go later using https://github.com/geonetwork/docker-geonetwork/blob/master/3.10.4/Dockerfile but adding the below before the CMD to start.
RUN mkdir $DATA_DIR/config/schema_plugins/iso19139.gemini23/ COPY geonetwork_customisation/iso19139.gemini23-3.10.x/src/main/plugin/iso19139.gemini23 $DATA_DIR/config/schema_plugins/iso19139.gemini23/ COPY geonetwork_customisation/iso19139.gemini23-3.10.x/target/schema-iso19139.gemini23-3.7.jar $CATALINA_HOME/webapps/geonetwork/WEB_INF/lib/
KoalaGeo
commented
4 years ago That didn't work. :-(
archaeogeek
commented
4 years ago OK, all very weird, since my approach worked from clean! I will try to replicate your approach and see how I get on- I might be able to do this tomorrow but otherwise it will be next week.
archaeogeek
commented
4 years ago Hi @KoalaGeo I've sorted it! Your dockerfile has a minor typo in it- https://github.com/BritishGeologicalSurvey/gc-geonetwork/blob/main/Dockerfile.bgsgeonetwork#L13 should read WEB-INF rather than WEB_INF
Sorry it took me so long to spot. The way I actually found it was by opening a bash prompt inside the running container using docker exec -t -i bgs-geonetwork /bin/bash and then checking the contents of the WEB-INF directory.
KoalaGeo
commented
4 years ago 🤦 Sorry! Well that's good news :-)
Thank you for taking the time to look into it.
Hi,
Have you any examples of adding this plugin to a deployment on Docker?
Can you run the COPY arguments in the dockerfile? Or is it better to run https://github.com/geonetwork/core-geonetwork/blob/3.10.x/add-schema.sh in the dockerfile? Or another way?
Many thanks,
Edd