Closed apapadak closed 1 year ago
jorgediaz-lr
commented
1 year ago Hi @apapadak
You are saying:
I am trying to run a 'display orphan index entries' with Entity = 'Web Content Article' and on a specific site and I am getting the following exception in the logs
Is the error produced with other entities different from "Web Content Article" or if you don't enable the "display orphan index entries" option?
Can you annex the complete log file?
Thank you
apapadak
commented
1 year ago Hi @jorgediaz-lr,
Apparently, it's thrown in many other entities as I can see in the exceptions log (attached). These exceptions are thrown the moment that I enter the index-checker UI screen and the 'java.lang.ClassNotFoundException: com.liferay.journal.model.impl.JournalArticleModelImpl' its thrown by itself when i try to run a search as described in the initial post.
Also, index-checker runs with no such issues in our QA env whereas the exceptions above show up in the QA2 env. The strange thing is that both installations are, more or less, similar. Same Liferay version, 3 live-node cluster, 3 ElasticSearch cluster nodes.
Seems like a classloader issue but I am not sure where to start to fix it.
Kind regards,
Anthony
jorgediaz-lr
commented
1 year ago Hi @apapadak
Which "Index Checker" plugin version are you using? Please use the latest version if you aren't using it.
If you see the ServiceUtil.getJavaClass code of the latest version, the call to jorgediazest.util.service.ServiceUtil.getClassModelFromPortal can throw a ClassNotFoundException but it is ignored, the "catch" section is empty, see:
https://github.com/jorgediaz-lr/index-checker/blob/ed325d07c34c44e3cb30ea72ba62b55aaf4579c2/modules/servicebuilder-reflection/src/main/java/jorgediazest/util/service/ServiceUtil.java#L134-L138
The idea of that ServiceUtil code, as far as I remember was to:
jorgediazest.util.service.ServiceUtil.getClassModelFromPortalCan you try enabling the DEBUG traces of jorgediazest.util.service.ServiceUtil category? This log trace may give more information: https://github.com/jorgediaz-lr/index-checker/blob/ed325d07c34c44e3cb30ea72ba62b55aaf4579c2/modules/servicebuilder-reflection/src/main/java/jorgediazest/util/service/ServiceUtil.java#L146
To enable the DEBUG traces, you have to go to Liferay Configuration => Server Administration => Log traces
apapadak
commented
1 year ago Hi @jorgediaz-lr ,
Yes, you are correct.
Initially, I had installed the latest version (index-checker) from GitHub and tried to run it on our QA2 env as described in the first post (ticked 'display orphan index entries' with Entity = 'Web Content Article' and on a specific site). After some time I got an Internal Server Error on the browser and no exception in the logs. I thought that this must be an edge case in the code somewhere and it was suppressing the exceptions (maybe because it wasn't expecting to get them for this edge case).
So I went and added a printStackTrace() in every try .. catch block that I could find in the hopes that I will get more information. But as you explained, it is meant to ignore these kind of exceptions so these are not the root cause of the issue.
We have reason to believe that we may have orphaned entries in this site in the test of thousands so I've also tried to increase the following parameter on the portlet.properties #
#
index.search.limit=120000 but it didn't make any change and I still see the Internal Server Error.
After timing the duration of the request (from the moment I press the button until I see the Internal Server Error), it seems to be consistently about 60 seconds every time I do it. So I think it's hitting some timeout in the QA2 Liferay/Tomcat/Apache configuration which doesn't happen in our QA environment because we have much less orphaned indices. A few seconds also after the Internal Server Error I see some index-checker related logs which seem to be normal output which also strengthens the hypothesis that it's some kind of timeout that severs the connection in the browser/Liferay server, but the index-checker is still running in the back.
2023-11-11 19:04:22.286 INFO [ajp-nio-10.90.1.200-8009-exec-13][Comparison:108] com.liferay.journal.model.JournalArticle {uid=com.liferay.journal.model.JournalArticle_PORTLET_3726197, groupId=2435000, permissionsRoleId=10145, classNameId=0, version=1.0, folderId=37215986, companyId=10136, classPK=0, modifiedDate=1677227406, AssetCategory.title={es_ES=ict,sv_SE=ict, bg_BG=ict, hr_HR=ict, lv_LV=ict, nl_NL=ict, mt_MT=ict, hu_HU=ict, de_DE=ict, el_GR=ict, cs_CZ=ict, da_DK=ict, et_EE=ict, pl_PL=ict, fi_FI=ict, sk_SK=ict, it_IT=ict, lt_T=ict, sl_SI=ict, pt_PT=ict, fr_FR=ict, ro_RO=ict, en_GB=ict (2)}, AssetEntries_AssetCategories.categoryId=37216195, treePath=[37215986, 37215984], resourcePrimKey=37216198, creatDate=1677227406, status=0, id=37216197} 2023-11-11 19:04:22.286 INFO [ajp-nio-10.90.1.200-8009-exec-13][Comparison:108] com.liferay.journal.model.JournalArticle {uid=com.liferay.journal.model.JournalArticle_PORTLET_3726214, groupId=2435000, permissionsRoleId=10145, classNameId=0, version=1.0, folderId=37215986, companyId=10136, classPK=0, modifiedDate=1677227406, AssetCategory.title={es_ES=healh, sv_SE=health, bg_BG=health, hr_HR=health, lv_LV=health, nl_NL=health, mt_MT=health, hu_HU=health, de_DE=health, el_GR=health, cs_CZ=health, da_DK=health, et_EE=health, pl_PL=helth, fi_FI=health, sk_SK=health, it_IT=health, lt_LT=health, sl_SI=health, pt_PT=health, fr_FR=health, ro_RO=health, en_GB=health (2)}, AssetEntries_AssetCategories.categoryId=3725982, treePath=[37215986, 37215984], resourcePrimKey=37216215, createDate=1677227406, status=0, id=37216214} 2023-11-11 19:04:22.287 INFO [ajp-nio-10.90.1.200-8009-exec-13][Comparison:108] com.liferay.journal.model.JournalArticle {uid=com.liferay.journal.model.JournalArticle_PORTLET_3726231, groupId=2435000, permissionsRoleId=10145, classNameId=0, version=1.0, folderId=37215986, companyId=10136, classPK=0, modifiedDate=1677227406, AssetCategory.title={es_ES=healh, sv_SE=health, bg_BG=health, hr_HR=health, lv_LV=health, nl_NL=health, mt_MT=health, hu_HU=health, de_DE=health, el_GR=health, cs_CZ=health, da_DK=health, et_EE=health, pl_PL=helth, fi_FI=health, sk_SK=health, it_IT=health, lt_LT=health, sl_SI=health, pt_PT=health, fr_FR=health, ro_RO=health, en_GB=health (2)}, AssetEntries_AssetCategories.categoryId=3725982, treePath=[37215986, 37215984], resourcePrimKey=37216232, createDate=1677227406, status=0, id=37216231} 2023-11-11 19:04:22.287 INFO [ajp-nio-10.90.1.200-8009-exec-13][Comparison:108] com.liferay.journal.model.JournalArticle {uid=com.liferay.journal.model.JournalArticle_PORTLET_3726250, groupId=2435000, permissionsRoleId=10145, classNameId=0, version=1.0, folderId=37215986, companyId=10136, classPK=0, modifiedDate=1677227406, AssetCategory.title={es_ES=enery, sv_SE=energy, bg_BG=energy, hr_HR=energy, lv_LV=energy, nl_NL=energy, mt_MT=energy, hu_HU=energy, de_DE=energy, el_GR=energy, cs_CZ=energy, da_DK=energy, et_EE=energy, pl_PL=enrgy, fi_FI=energy, sk_SK=energy, it_IT=energy, lt_LT=energy, sl_SI=energy, pt_PT=energy, fr_FR=energy, ro_RO=energy, en_GB=energy (2)}, AssetEntries_AssetCategories.categoryId=3726248, treePath=[37215986, 37215984], resourcePrimKey=37216251, createDate=1677227406, status=0, id=37216250}
I will take a look at it Monday onwards and let you know.
Many thanks!
jorgediaz-lr
commented
1 year ago Hi @apapadak
Yes, if you are having a "Internal Server Error" thrown by your load balancer or web server that is in front of the Liferay server, I think the root cause is just a timeout caused by a long execution of Index Checker, as you said here:
So I think it's hitting some timeout in the QA2 Liferay/Tomcat/Apache configuration which doesn't happen in our QA environment because we have much less orphaned indices. A few seconds also after the Internal Server Error I see some index-checker related logs which seem to be normal output which also strengthens the hypothesis that it's some kind of timeout that severs the connection in the browser/Liferay server, but the index-checker is still running in the back.
Perhaps you can try filtering by site or modified date, but I think you should try connecting directly to the Liferay tomcat server bypassing anything that can throw the timeout error or change the configuration to avoid throwing that error.
Let me know if you need additional help or I can close this issue.
apapadak
commented
1 year ago Hi @jorgediaz-lr ,
We updated the Apache TimeOut parameter to 600 seconds on the /etc/httpd/conf.d/httpd.conf file and we no longer had the Internal Server Error issue.
Many thanks for your help, the issue can be closed now!!
Hi Jorge,
I am using this plugin with Liferay liferay.workspace.target.platform.version = 7.2.10.fp14.
I am trying to run a 'display orphan index entries' with Entity = 'Web Content Article' and on a specific site and I am getting the following exception in the logs:
java.lang.ClassNotFoundException: com.liferay.journal.model.impl.JournalArticleModelImpl at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1365) at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1188) at jorgediazest.util.service.ServiceUtil.getClassModelFromPortal(ServiceUtil.java:122) at jorgediazest.util.service.ServiceUtil.getJavaClass(ServiceUtil.java:137) at jorgediazest.util.service.ServiceUtil.getLiferayModelImplClass(ServiceUtil.java:53) at jorgediazest.util.service.ServiceImpl.getLiferayModelImplClass(ServiceImpl.java:85) at jorgediazest.util.service.ServiceImpl.getTableInfo(ServiceImpl.java:126) at jorgediazest.util.service.ServiceImpl.getTableInfo(ServiceImpl.java:122) at jorgediazest.util.model.ModelImpl.getTableInfo(ModelImpl.java:412) at jorgediazest.util.model.ModelImpl.getAttributeNames(ModelImpl.java:186) at jorgediazest.util.model.ModelFactory.getModelObject(ModelFactory.java:80) at jorgediazest.util.model.ModelFactory.getModelObject(ModelFactory.java:58) at jorgediazest.indexchecker.model.IndexCheckerModelFactory.getModelObject(IndexCheckerModelFactory.java:50) at jorgediazest.indexchecker.portlet.IndexCheckerPortlet.getModelList(IndexCheckerPortlet.java:339) at jorgediazest.indexchecker.portlet.IndexCheckerPortlet.executeCheck(IndexCheckerPortlet.java:222) at jorgediazest.indexchecker.portlet.IndexCheckerPortlet.executeCheck(IndexCheckerPortlet.java:575) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.liferay.portal.kernel.portlet.LiferayPortlet.callActionMethod(LiferayPortlet.java:178) at com.liferay.portal.kernel.portlet.bridges.mvc.MVCPortlet.callActionMethod(MVCPortlet.java:415) at com.liferay.portal.kernel.portlet.LiferayPortlet.processAction(LiferayPortlet.java:88) .. .. .. at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:750)The request runs for a few seconds and I get an internal server error.
Any ideas?
Many thanks,
Anthony