search

IHTSDO / classification-service

Standalone Classification Service providing OWL Reasoning against SNOMED CT RF2 files.
Other
13 stars 1 forks source link

java.io.FileNotFoundException: store/releases/SnomedCT_InternationalRF2_PRODUCTION_20220131T120000Z.zip #5

Closed waydes closed 1 year ago

waydes commented 2 years ago

I am learning what I can about the classification-service. I am using Ubuntu 20.04 and on my laptop, so using localhost rather than a domain. Using just the classification service and not Snowstorm. I tried to classify the latest release on the International Version and our Veterinary Extension relase from 10/01/2021. previousPackage is SnomedCT_InternationalRF2_PRODUCTION_20220131T120000Z.zip in store/releases, no dependency package and the rf2Delta is our release file. I get the java.io.FileNotFoundException for SnomedCT_InternationalRF2_PRODUCTION_20220131T120000Z.zip which is present. The user and group are my login and group and the classification-service runs from my home directory. Permissions are read and write. I have googled for solutions, but am stumped. Any suggestions? Thanks

2022-02-22 11:46:35.826 ERROR 12418 --- [enerContainer-1] o.s.o.r.s.s.ClassificationJobManager : Classification failed store/releases/SnomedCT_InternationalRF2_PRODUCTION_20220131T120000Z.zip, branch 20955e62-c947-4232-95ef-dbc543f15b28.

java.io.FileNotFoundException: store/releases/SnomedCT_InternationalRF2_PRODUCTION_20220131T120000Z.zip at org.springframework.core.io.FileSystemResource.getInputStream(FileSystemResource.java:179) at org.ihtsdo.otf.resourcemanager.ResourceManager.readResourceStream(ResourceManager.java:47) at org.snomed.otf.reasoner.server.service.ClassificationJobManager.getInputStreams(ClassificationJobManager.java:187) at org.snomed.otf.reasoner.server.service.ClassificationJobManager.classify(ClassificationJobManager.java:148) at org.snomed.otf.reasoner.server.service.ClassificationJobManager.consumeClassificationJob(ClassificationJobManager.java:116) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:170) at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:120) at org.springframework.jms.listener.adapter.MessagingMessageListenerAdapter.invokeHandler(MessagingMessageListenerAdapter.java:114) at org.springframework.jms.listener.adapter.MessagingMessageListenerAdapter.onMessage(MessagingMessageListenerAdapter.java:77) at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:736) at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:696) at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:674) at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:318) at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:257) at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1189) at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1179) at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1076) at java.base/java.lang.Thread.run(Thread.java:829)

kaicode commented 2 years ago

Hi @waydes,

The path of the store directory is relative to the directory in which you start the Classification Service application.

On our Ubuntu servers application directory is /opt/classification-service, that contains the jar file. The store is in /opt/classification-service/store. The java process is started from within the application directory. This way the default configuration can be used which has a relative path of store/releases for the releases directory, see default application.properties.

If you do not cd to the application directory before starting the application the store will be expected to be within the current working directory.

I hope that helps. You could set the snomed.release.storage.local.path property to an absolute path rather than relative but that may result in a more complicated setup. Up to you.

Let me know how it goes. Kai

waydes commented 2 years ago

That seems to have been the problem. The classifier is working on my ubuntu laptop. Since our Coldfusion servers are on windows, I am testing Snowstorm and the classification-service there. Still not quite working.

I do have a question about starting Snowstorm using sudo on the Ubuntu server. When I did the developer training the instructions started Snowstorm as root: sudo java -Xms4g -Xmx4g -jar snowstorm*.jar | sudo tee -a snowstorm.log I consistently get an error that won't allow me to run java as root when I put snowstorm in /opt/snowstorm. I also thought it is a security issue to start java as root. My questions are:

  1. Is it really necessary to run Snowstorm as root?
  2. What was changed in order for java to be started as root? Thanks.

On Wed, Feb 23, 2022 at 5:01 AM Kai Kewley @.***> wrote:

Hi @waydes https://github.com/waydes,

The path of the store directory is relative to the directory in which you start the Classification Service application.

On our Ubuntu servers application directory is /opt/classification-service, that contains the jar file. The store is in /opt/classification-service/store. The java process is started from within the application directory. This way the default configuration can be used which has a relative path of store/releases for the releases directory, see default application.properties https://github.com/IHTSDO/classification-service/blob/4.2.0/src/main/resources/application.properties#L18 .

If you do not cd to the application directory before starting the application the store will be expected to be within the current working directory.

I hope that helps. You could set the snomed.release.storage.local.path property to an absolute path rather than relative but that may result in a more complicated setup. Up to you.

Let me know how it goes. Kai

— Reply to this email directly, view it on GitHub https://github.com/IHTSDO/classification-service/issues/5#issuecomment-1048665949, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABSUF37A6WSMRY7QX5QUBXLU4S5A7ANCNFSM5PCEOCPQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you were mentioned.Message ID: @.***>

kaicode commented 2 years ago

  1. Snowstorm does not need to be run as root and should not be in production environments. The developer training does it that way because it's easy, not because it's recommended. In production Snowstorm should be run under a non-root user and probably wrapped within some sort of process handler. We don't provide guidance for this because it's a very generic problem and there is a different solution for every environment in which java can be run.

  2. That was the standard setup in the Ubuntu AWS image we used. I don't recommend it.

I have never used a Windows server but a quick search found this project which provides a service wrapper for a java application in a windows environment: Windows Service Wrapper (on GitHub). The sample configuration starts a java process with a jenkins.war file. The Snowstorm jar file could be started in the same way.

Here is a first guess at the winsw configuration for Snowstorm:

<service>
  <id>snowstorm</id>
  <name>Snowstorm</name>
  <description>This service runs Snowstorm terminology server.</description>
  <executable>java</executable>
  <arguments>-Xms2g -Xmx4g -jar "%BASE%\snowstorm.jar" --server.port=8080 --elasticsearch.urls=http://localhost:9200</arguments>
  <log mode="roll"></log>
</service>

I don't have access to a windows machine so I'm not able to test that but it should get you started.

waydes commented 2 years ago

Thanks so much for the reply. I am running Snowstorm under Ubuntu and Windows successfully and making calls the classifier successfully. These are test machines, so non-root user but not bother with a process handler. When we get to that point, I will follow up on your suggestion. Thanks again.

On Tue, Mar 1, 2022 at 5:54 AM Kai Kewley @.***> wrote:

1.

Snowstorm does not need to be run as root and should not be in production environments. The developer training does it that way because it's easy, not because it's recommended. In production Snowstorm should be run under a non-root user and probably wrapped within some sort of process handler. We don't provide guidance for this because it's a very generic problem and there is a different solution for every environment in which java can be run. 2.

That was the standard setup in the Ubuntu AWS image we used. I don't recommend it.

I have never used a Windows server but a quick search found this project which provides a service wrapper for a java application in a windows environment: Windows Service Wrapper (on GitHub) https://github.com/winsw/winsw. The sample configuration starts a java process with a jenkins.war file. The Snowstorm jar file could be started in the same way.

Here is a first guess at the winsw configuration for Snowstorm:

snowstorm Snowstorm This service runs Snowstorm terminology server. java -Xms2g -Xmx4g -jar "%BASE%\snowstorm.jar" --server.port=8080 --elasticsearch.urls=http://localhost:9200

I don't have access to a windows machine so I'm not able to test that but it should get you started.

— Reply to this email directly, view it on GitHub https://github.com/IHTSDO/classification-service/issues/5#issuecomment-1055349693, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABSUF34YQEJI6KPNEYNZJ7DU5YANPANCNFSM5PCEOCPQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you were mentioned.Message ID: @.***>