keeps / dbptk-enterprise

DBPTK Enterprise deployment using docker
https://database-preservation.com
GNU Lesser General Public License v3.0
4 stars 2 forks source link

empty argument error - while preparing database #18

Closed Laurira closed 9 months ago

Laurira commented 3 years ago

This is the most critical issue at the moment. I can not browse database bigger than Sakila. I cleaned the machine, I have plenty of space and RAM but DBPTK Enterprise is not working.

image

solr_1 | Caused by: java.lang.OutOfMemoryError: Java heap space solr_1 | dbptke_1 | 2021-08-24 12:17:26.937 WARN 1 --- [nio-8080-exec-4] c.d.c.s.index.DatabaseRowsSolrManager : Could not insert a document batch in collectiondbv-database-b49692d4-5b2f-4834-ae68-bb5e435c145c. Last response (if any): null dbptke_1 | dbptke_1 | org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://172.18.0.2:8983/solr/dbv-database-b49692d4-5b2f-4834-ae68-bb5e435c145c: Server error writing document id 24981989 to the index dbptke_1 | at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:665) dbptke_1 | at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:265) dbptke_1 | at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248) dbptke_1 | at org.apache.solr.client.solrj.impl.LBSolrClient.doRequest(LBSolrClient.java:368) dbptke_1 | at org.apache.solr.client.solrj.impl.LBSolrClient.request(LBSolrClient.java:296) dbptke_1 | at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.sendRequest(BaseCloudSolrClient.java:1143) dbptke_1 | at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.requestWithRetryOnStaleState(BaseCloudSolrClient.java:906) dbptke_1 | at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.request(BaseCloudSolrClient.java:838) dbptke_1 | at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211) dbptke_1 | at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:177) dbptke_1 | at com.databasepreservation.common.server.index.DatabaseRowsSolrManager.insertDocument(DatabaseRowsSolrManager.java:387) dbptke_1 | at com.databasepreservation.common.server.index.DatabaseRowsSolrManager.addRow(DatabaseRowsSolrManager.java:160) dbptke_1 | at com.databasepreservation.modules.viewer.DbvtkExportModule.handleDataRow(DbvtkExportModule.java:134) dbptke_1 | at com.databasepreservation.model.modules.filters.IdentityFilter.handleDataRow(IdentityFilter.java:88) dbptke_1 | at com.databasepreservation.model.modules.filters.ObservableFilter.handleDataRow(ObservableFilter.java:123) dbptke_1 | at com.databasepreservation.modules.siard.in.content.SIARD2ContentImportStrategy.endElement(SIARD2ContentImportStrategy.java:374) dbptke_1 | at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown Source) dbptke_1 | at org.apache.xerces.impl.xs.XMLSchemaValidator.endElement(Unknown Source) dbptke_1 | at org.apache.xerces.impl.XMLNSDocumentScannerImpl.scanEndElement(Unknown Source) dbptke_1 | at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source) dbptke_1 | at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source) dbptke_1 | at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source) dbptke_1 | at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source) dbptke_1 | at org.apache.xerces.parsers.XMLParser.parse(Unknown Source) dbptke_1 | at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source) dbptke_1 | at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source) dbptke_1 | at com.databasepreservation.modules.siard.in.content.SIARD2ContentImportStrategy.importContent(SIARD2ContentImportStrategy.java:184) dbptke_1 | at com.databasepreservation.modules.siard.in.input.SIARDImportDefault.migrateDatabaseTo(SIARDImportDefault.java:64) dbptke_1 | at com.databasepreservation.DatabaseMigration.migrate(DatabaseMigration.java:123) dbptke_1 | at com.databasepreservation.common.server.controller.SIARDController.convertSIARDtoSolr(SIARDController.java:706) dbptke_1 | at com.databasepreservation.common.server.controller.SIARDController.loadFromLocal(SIARDController.java:669) dbptke_1 | at com.databasepreservation.common.api.v1.CollectionResource.createCollection(CollectionResource.java:196) dbptke_1 | at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) dbptke_1 | at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) dbptke_1 | at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) dbptke_1 | at java.lang.reflect.Method.invoke(Method.java:498) dbptke_1 | at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) dbptke_1 | at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) dbptke_1 | at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) dbptke_1 | at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219) dbptke_1 | at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) dbptke_1 | at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:469) dbptke_1 | at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:391) dbptke_1 | at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:80) dbptke_1 | at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:253) dbptke_1 | at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) dbptke_1 | at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) dbptke_1 | at org.glassfish.jersey.internal.Errors.process(Errors.java:292) dbptke_1 | at org.glassfish.jersey.internal.Errors.process(Errors.java:274) dbptke_1 | at org.glassfish.jersey.internal.Errors.process(Errors.java:244) dbptke_1 | at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) dbptke_1 | at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:232) dbptke_1 | at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) dbptke_1 | at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:392) dbptke_1 | at org.glassfish.jersey.servlet.ServletContainer.serviceImpl(ServletContainer.java:385) dbptke_1 | at org.glassfish.jersey.servlet.ServletContainer.doFilter(ServletContainer.java:560) dbptke_1 | at org.glassfish.jersey.servlet.ServletContainer.doFilter(ServletContainer.java:501) dbptke_1 | at org.glassfish.jersey.servlet.ServletContainer.doFilter(ServletContainer.java:438) dbptke_1 | at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) dbptke_1 | at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) dbptke_1 | at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) dbptke_1 | at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) dbptke_1 | at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) dbptke_1 | at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) dbptke_1 | at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.filterAndRecordMetrics(WebMvcMetricsFilter.java:114) dbptke_1 | at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:104) dbptke_1 | at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) dbptke_1 | at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) dbptke_1 | at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) dbptke_1 | at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) dbptke_1 | at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) dbptke_1 | at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)

2021-08-24 12:17:25,931 [http-nio-8080-exec-4] ERROR o.a.s.c.s.impl.BaseCloudSolrClient - Request to collection [dbv-database-b49692d4-5b2f-4834-ae68-bb5e435c145c] failed due to (500) org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://172.18.0.2:8983/solr/dbv-database-b49692d4-5b2f-4834-ae68-bb5e435c145c: Server error writing document id 24981989 to the index, retry=0 commError=false errorCode=500

Laurira commented 3 years ago

I discovered that the main error is: java.io.FileNotFoundException: teeregisterMntRa_2_1.siard_lobseg_1/content/schema1/table1/lob7/record7.bin (No such file or directory)

But I have copied those files manually aside to the siard file itself into the folder /dbvtk/SIARD-files/ image

Why is dbptk not able to see those?

Laurira commented 3 years ago

I remembered that I did not compare the users of file and dbvtk. So yes these are different. The automatically copied siard file itself has user and group root:root but the lobseq files were all with user 1000:1000 So I restarted everything and did chwon root:root -R for all folders and but the error is still the same.

hmiguim commented 3 years ago

Caused by: java.lang.OutOfMemoryError: Java heap space

Please follow this https://github.com/keeps/dbptk-enterprise/issues/19#issuecomment-910106952, this seams to be the same problem.

luis100 commented 9 months ago

Solved on #19