search

DeepPhe / DeepPhe-Release

Public release of the DeepPhe analytic software
Other
30 stars 10 forks source link

Error in running DeepPhe on Ubuntu #37

Closed ghostk closed 2 years ago

ghostk commented 2 years ago

I followed the instructions at https://github.com/DeepPhe/DeepPhe-Release/wiki/Linux-Installation-Instructions to install DeepPhe on my system. Everything seems to be fine until I ran the command ./run.sh ~/docs/dphe-examples/reports

Examining the error log cTAKES.error.log shows this:

..... Caused by: org.neo4j.driver.exceptions.ClientException: There is no procedure with the name deepphe.addPatientSummary registered for this database instance. Please ensure you've spelled the procedure name correctly and that the procedure is properly deployed. .....

I am using Ubuntu 20.04.4 LTS, I have confirmed the deepphe-neo4j-plugin-0.4.0.jar has been copied to neo4j-community-3.5.12/plugins/ directory. Any idea about this error and how to resolve it? Many thanks!

Kun

JohnLevander commented 2 years ago

Hello, thanks for the report.

Are you sure that neo4j wasn't running when you copied the .jar into the plugins folder? Let's rule that possibility out by restarting neo4j?

Thanks.

ghostk commented 2 years ago

Thanks for your quick response! I went to the neo4j-community-3.5.12/bin directory and run ./neo4j restart It restarted fine, but when I reran the deepphe using sudo ./run.sh ~/docs/dphe-examples/reports The same error occurs.

Below is the log file of neo4j restart:

2022-07-20 20:20:51.830+0000 WARN dbms.active_database is deprecated. 2022-07-20 20:20:51.845+0000 INFO ======== Neo4j 3.5.12 ======== 2022-07-20 20:20:51.853+0000 INFO Starting... 2022-07-20 20:20:54.399+0000 INFO Bolt enabled on 127.0.0.1:7687. 2022-07-20 20:20:55.499+0000 INFO Started. 2022-07-20 20:20:56.110+0000 INFO Remote interface available at http://localhost:7474/

Any thoughts on other possibilities?

JohnLevander commented 2 years ago

Do you need to run as sudo?

ghostk commented 2 years ago

Yes, if not, I got Permission denied error that prevents the program to initialize neo4j connection for the ontology.db. Here is the output without sudo:

.20 Jul 2022 15:31:37 WARN ServiceFactory - Could not immediately initialize neo4j connection for: resources/graph/neo4j/ontology.db 20 Jul 2022 15:31:37 WARN ServiceFactory - java.io.FileNotFoundException: /home/kun/DeepPhe-Release-xn0.5.3/dphe-cli/target/deepphe-0.4.0-bin/deepphe-0.4.0/resources/graph/neo4j/logs/debug.log (Permission denied) 20 Jul 2022 15:31:37 WARN ServiceFactory - Deleting old logs for a second attempt ... 20 Jul 2022 15:31:37 ERROR ServiceFactory - Could not initialize neo4j connection for: resources/graph/neo4j/ontology.db 20 Jul 2022 15:31:37 ERROR ServiceFactory - java.io.FileNotFoundException: /home/kun/DeepPhe-Release-xn0.5.3/dphe-cli/target/deepphe-0.4.0-bin/deepphe-0.4.0/resources/graph/neo4j/logs/debug.log (Permission denied) Error: [255] DeepPhe failed. The script will now exit.

I really appreciate your assistance!

JohnLevander commented 2 years ago

I think it would be good to fix the permissions...and try to get it to run without needing sudo. I'm not sure what issues that might be introducing. Do you need help with the chmod commands?

I would give your user read/write access to every file/directory under /home/kun/DeepPhe-Release-xn0.5.3.

ghostk commented 2 years ago

I changed the ownership of all files/sub-directories within neo4j and DeepPhe directories to my user and file permission to 766. Now I can run without sudo as: ./run.sh ~/docs/dphe-examples/reports

But the same problem persisted as below. Any other measures to try?

Caused by: org.neo4j.driver.exceptions.ClientException: There is no procedure with the name deepphe.addPatientSummary registered for this database instance. Please ensure you've spelled the procedure name correctly and that the proced

JohnLevander commented 2 years ago

What version of java are you using?

You can execute java --version at the command line. It needs to be Java 1.8.

ghostk commented 2 years ago

I believe that is the one in use. I installed the openjdk 8 following the instruction: sudo apt-get update sudo apt-get install openjdk-8-jdk

Then, in the installer.conf file, I set JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64

When I run java -version, here is the output: openjdk version "1.8.0_312" OpenJDK Runtime Environment (build 1.8.0_312-8u312-b07-0ubuntu1~20.04-b07) OpenJDK 64-Bit Server VM (build 25.312-b07, mixed mode)

So, I believe that complies with the program requirement.

JohnLevander commented 2 years ago

It really sounds like the jar is not in the correct spot. Can you go to the neo4j plugins directory and type pwd.

plugins johnlevander$ pwd
/dev//neo4j-community-3.5.12/plugins

And then in there, ls -la

plugins johnlevander$ ls -la
total 12304
drwxr-xr-x   4 me  admin      128 Jul 15 23:51 .
drwxr-xr-x  17 me  admin      544 Jul 16 17:36 ..
-rw-r--r--@  1 me  admin     6148 Jul 15 22:55 .DS_Store
-rw-r--r--   1 me  admin  5362561 Jul 21 12:08 deepphe-neo4j-plugin-0.4.0.jar

and paste the output?

ghostk commented 2 years ago

Sure, here it is: (base) kun@kun-Precision-5520:~/neo4j-community-3.5.12/plugins$ pwd /home/kun/neo4j-community-3.5.12/plugins (base) kun@kun-Precision-5520:~/neo4j-community-3.5.12/plugins$ ls -la total 5256 drwxr-xr-x 2 kun kun 4096 Jul 20 14:18 . drwxr-xr-x 11 kun kun 4096 Jul 20 14:19 .. -rwxr-xr-x 1 kun kun 5367283 Jul 20 14:57 deepphe-neo4j-plugin-0.4.0.jar -rwxr-xr-x 1 kun kun 2217 Oct 15 2019 README.txt

Anything looks wrong?

JohnLevander commented 2 years ago

It looks right.

What does your installer.conf file look like? What is NEO4J_HOME set to in there? Is it possible you have another installation of neo4j on your machine?

Another possibility is that the neo4j process doesn't have permission to read the .jar file. You are running ./neo4j console as user kun as well?

ghostk commented 2 years ago

Here is what's in installer.conf: NEO4J_HOME=/home/kun/neo4j-community-3.5.12 JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64

Yes, I can run ./neo4j as user kun, here is what the permission looks like in ~/neo4j-community-3.5.12/bin: drwxr-xr-x 3 kun kun 4096 Oct 15 2019 . drwxr-xr-x 11 kun kun 4096 Jul 20 14:19 .. -rwxr-xr-x 1 kun kun 2082 Oct 11 2019 cypher-shell -rwxr-xr-x 1 kun kun 16161 Oct 15 2019 neo4j -rwxr-xr-x 1 kun kun 8865 Oct 15 2019 neo4j-admin -rwxr-xr-x 1 kun kun 8460 Oct 15 2019 neo4j-import drwxr-xr-x 2 kun kun 4096 Oct 15 2019 tools

I do not recall having installed other instance of neo4j. Actually, I do not have neo4j command in the PATH (whichis and whereis commands find none).

And I actually was able to get an output directory within DeepPhe unix directory. So, it seems the program runs, but abruptively stopped due to that error. (base) kun@kun-Precision-5520:~/DeepPhe-Release-xn0.5.3/scripts/unix/output$ ls -l total 12 drwxr-xr-x 3 kun kun 4096 Jul 20 14:21 HTML drwxr-xr-x 3 kun kun 4096 Jul 20 14:21 JSON drwxr-xr-x 2 kun kun 4096 Jul 20 15:30 summary

Any other things to check?

seanfinan commented 2 years ago

Hi Kun,

You could open the neo4j graphical browser in a web browser: http://localhost:7474/browser/

Then list the available procedures with the command: CALL dbms.procedures()

You should see a list of many procedures that start with "db." and "dbms." At the end of the list you should see a collection of the deepphe. procedures that are injected by the plugin:

"deepphe.addCancerInfo""deepphe.addCancerInfo(patientId :: STRING?, cancerJson :: STRING?) :: (text :: STRING?)""Appends Cancer Summary information to a Patient node.""WRITE"

"deepphe.addMentionInfo""deepphe.addMentionInfo(noteId :: STRING?, mentionJson :: STRING?) :: (text :: STRING?)""Creates or appends to a Mention node.""WRITE"

"deepphe.addNoteInfo""deepphe.addNoteInfo(patientId :: STRING?, noteJson :: STRING?) :: (text :: STRING?)""Creates or appends to a Note node.""WRITE"

"deepphe.addPatientInfo""deepphe.addPatientInfo(patientJson :: STRING?) :: (text :: STRING?)""Creates or appends to a Patient node.""WRITE"

"deepphe.addPatientSummary""deepphe.addPatientSummary(patientSummaryJson :: STRING?) :: (text :: STRING?)""Creates or appends to a Patient node and cancer summary.""WRITE"

"deepphe.clearPatientSummary""deepphe.clearPatientSummary(patientId :: STRING?) :: (text :: STRING?)""Clears a Patient node and cancer summary.""WRITE"

"deepphe.initializeDphe""deepphe.initializeDphe() :: (text :: STRING?)""Creates root nodes for Patient, Document, Stage.""WRITE"

"deepphe.shutdownServer""deepphe.shutdownServer() :: (text :: STRING?)""Shuts down the Neo4j Service.""DBMS"

Sean

From: @. @.> Sent: Thursday, July 21, 2022 6:23 PM To: DeepPhe/DeepPhe-Release @.> Cc: Subscribed @.> Subject: Re: [DeepPhe/DeepPhe-Release] Error in running DeepPhe on Ubuntu (Issue #37) [EXTERNAL]

Here is what's in installer.conf: NEO4J_HOME=/home/kun/neo4j-community-3.5.12 JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64

Yes, I can run ./neo4j as user kun, here is what the permission looks like in ~/neo4j-community-3.5.12/bin: drwxr-xr-x 3 kun kun 4096 Oct 15 2019 . drwxr-xr-x 11 kun kun 4096 Jul 20 14:19 .. -rwxr-xr-x 1 kun kun 2082 Oct 11 2019 cypher-shell -rwxr-xr-x 1 kun kun 16161 Oct 15 2019 neo4j -rwxr-xr-x 1 kun kun 8865 Oct 15 2019 neo4j-admin -rwxr-xr-x 1 kun kun 8460 Oct 15 2019 neo4j-import drwxr-xr-x 2 kun kun 4096 Oct 15 2019 tools

I do not recall having installed other instance of neo4j. Actually, I do not have neo4j command in the PATH (whichis and whereis commands find none).

And I actually was able to get an output directory within DeepPhe unix directory. So, it seems the program runs, but abruptively stopped due to that error. (base) @.***:~/DeepPhe-Release-xn0.5.3/scripts/unix/output$ ls -l total 12 drwxr-xr-x 3 kun kun 4096 Jul 20 14:21 HTML drwxr-xr-x 3 kun kun 4096 Jul 20 14:21 JSON drwxr-xr-x 2 kun kun 4096 Jul 20 15:30 summary

Any other things to check?

— Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https://github.com/DeepPhe/DeepPhe-Release/issues/37*issuecomment-1191991327__;Iw!!NZvER7FxgEiBAiR_!qk0VsHbdNPtpzeHo9T7Yq1Gr9_f84pfpQsk7D6h9hIo4Uq7wK2GeHIRmq0CipmRYWp_jcFDhMyhpuY8mQQ6A6c9bLrqYnMs$, or unsubscribehttps://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AB4J2NF3O6XPPNTCK26XVADVVHEUJANCNFSM54E6EB5A__;!!NZvER7FxgEiBAiR_!qk0VsHbdNPtpzeHo9T7Yq1Gr9_f84pfpQsk7D6h9hIo4Uq7wK2GeHIRmq0CipmRYWp_jcFDhMyhpuY8mQQ6A6c9b2RlfCUc$. You are receiving this because you are subscribed to this thread.Message ID: @.***>

ghostk commented 2 years ago

Hi Sean,

Good point! For the deepphe ones, I have

"deepphe.addMentionInfo" "deepphe.addMentionInfo(noteId :: STRING?, mentionJson :: STRING?) :: (text :: STRING?)" "Creates or appends to a Mention node." "WRITE"
"deepphe.addNoteInfo" "deepphe.addNoteInfo(patientId :: STRING?, noteJson :: STRING?) :: (text :: STRING?)" "Creates or appends to a Note node." "WRITE"
"deepphe.addPatientInfo" "deepphe.addPatientInfo(patientJson :: STRING?) :: (text :: STRING?)" "Creates or appends to a Patient node." "WRITE"
"deepphe.initializeDphe" "deepphe.initializeDphe() :: (text :: STRING?)" "Creates root nodes for Patient, Document, Stage." "WRITE"
"deepphe.shutdownServer" "deepphe.shutdownServer() :: (text :: STRING?)" "Shuts down the Neo4j Service." "DBMS"

I missed some of the procedures you have (certainly the one that causes the error addPatientSummary. Could it be due to the jar file I have is incomplete? Is there an alternative I could download or regenerate?

seanfinan commented 2 years ago

I don't know how it happened, but you must have a spoiled build. The strange thing is that all of those deepphe.add* procedures are declared in the same java class. I would think that you should have all or none of them.

John probably told you that we are currently working on releasing v5. of the software. The current candidate was built yesterday and is available from deepphe.github.io I have also changed the instructions for the linux build to reference the current version v0.5.7 My advice is that you remove your current installation and start over.

Sean

From: @. @.> Sent: Thursday, July 21, 2022 7:18 PM To: DeepPhe/DeepPhe-Release @.> Cc: Finan, Sean @.>; Comment @.***> Subject: Re: [DeepPhe/DeepPhe-Release] Error in running DeepPhe on Ubuntu (Issue #37) [EXTERNAL]

Hi Sean,

Good point! For the deepphe ones, I have

"deepphe.addMentionInfo" "deepphe.addMentionInfo(noteId :: STRING?, mentionJson :: STRING?) :: (text :: STRING?)" "Creates or appends to a Mention node." "WRITE" "deepphe.addNoteInfo" "deepphe.addNoteInfo(patientId :: STRING?, noteJson :: STRING?) :: (text :: STRING?)" "Creates or appends to a Note node." "WRITE" "deepphe.addPatientInfo" "deepphe.addPatientInfo(patientJson :: STRING?) :: (text :: STRING?)" "Creates or appends to a Patient node." "WRITE" "deepphe.initializeDphe" "deepphe.initializeDphe() :: (text :: STRING?)" "Creates root nodes for Patient, Document, Stage." "WRITE" "deepphe.shutdownServer" "deepphe.shutdownServer() :: (text :: STRING?)" "Shuts down the Neo4j Service." "DBMS"

I missed some of the procedures you have (certainly the one that causes the error addPatientSummary. Could it be due to the jar file I have is incomplete? Is there an alternative I could download or regenerate?

— Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https://github.com/DeepPhe/DeepPhe-Release/issues/37*issuecomment-1192021321__;Iw!!NZvER7FxgEiBAiR_!sLCghepySFV_X4nprGQVoGy8xqlYthXCS2j9IrcxjGiA316OUgzXtDcvBK1xkGAXvp80nN8F4v1P4leWjX3Hkvcvfcx7xWQ$, or unsubscribehttps://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AB4J2NHO2VGL2HTWJDYLJRLVVHLFJANCNFSM54E6EB5A__;!!NZvER7FxgEiBAiR_!sLCghepySFV_X4nprGQVoGy8xqlYthXCS2j9IrcxjGiA316OUgzXtDcvBK1xkGAXvp80nN8F4v1P4leWjX3HkvcveiibImU$. You are receiving this because you commented.Message ID: @.***>

ghostk commented 2 years ago

I tried to start over using the v0.5.7 version, but still only the five procedures in neo4j database procedures. I did not receive any error messages during building the program.

JohnLevander commented 2 years ago

I appreciate your patience with this issue. I think we should arrange a zoom meeting, I think I can get you up and running much quicker that way. Are you open to that?

ghostk commented 2 years ago

Hi John and Sean,

I restarted my computer, installed a clean neo4j and tried reset-db.sh again this morning. It worked! I suspected neo4j somehow did not reset and take in the new jar file. After I restarted my computer this morning, I checked the clean neo4j db procedures, and then stopped neo4j before running reset-db.sh. And when I restarted neo4j after that, all the procedures are there.

"deepphe.addCancerInfo" "deepphe.addCancerInfo(patientId :: STRING?, cancerJson :: STRING?) :: (text :: STRING?)" "Appends Cancer Summary information to a Patient node." "WRITE" "deepphe.addMentionInfo" "deepphe.addMentionInfo(noteId :: STRING?, mentionJson :: STRING?) :: (text :: STRING?)" "Creates or appends to a Mention node." "WRITE" "deepphe.addNoteInfo" "deepphe.addNoteInfo(patientId :: STRING?, noteJson :: STRING?) :: (text :: STRING?)" "Creates or appends to a Note node." "WRITE" "deepphe.addPatientInfo" "deepphe.addPatientInfo(patientJson :: STRING?) :: (text :: STRING?)" "Creates or appends to a Patient node." "WRITE" "deepphe.addPatientSummary" "deepphe.addPatientSummary(patientSummaryJson :: STRING?) :: (text :: STRING?)" "Creates or appends to a Patient node and cancer summary." "WRITE" "deepphe.clearPatientSummary" "deepphe.clearPatientSummary(patientId :: STRING?) :: (text :: STRING?)" "Clears a Patient node and cancer summary." "WRITE" "deepphe.initializeDphe" "deepphe.initializeDphe() :: (text :: STRING?)" "Creates root nodes for Patient, Document, Stage." "WRITE" "deepphe.shutdownServer" "deepphe.shutdownServer() :: (text :: STRING?)" "Shuts down the Neo4j Service." "DBMS"

And this is on the v0.5.7. Thanks very much for your assistance. Hope this may help future users of deepphe.

JohnLevander commented 2 years ago

Excellent! I'm going to mark this as closed. Feel free to reopen or file another issue if you run into more trouble.

seanfinan commented 2 years ago

I don't understand. I can see it here https://github.com/DeepPhe/DeepPhe-Release/blob/Translational-v0.5.0/dphe-neo4j-plugin/src/main/java/org/healthnlp/deepphe/neo4j/plugin/WriteFunctions.java on line #89.

Do you see it in the source that you downloaded?

From: @. @.> Sent: Friday, July 22, 2022 4:05 PM To: DeepPhe/DeepPhe-Release @.> Cc: Finan, Sean @.>; Comment @.***> Subject: Re: [DeepPhe/DeepPhe-Release] Error in running DeepPhe on Ubuntu (Issue #37) [EXTERNAL]

I tried to start over using the v0.5.7 version, but still only the five procedures in neo4j database procedures. I did not receive any error messages during building the program.

— Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https://github.com/DeepPhe/DeepPhe-Release/issues/37*issuecomment-1192897422__;Iw!!NZvER7FxgEiBAiR_!vzvNg0xBe0RIlkNkQa7IjhoGRoLwjbkPySNG0xiCcQcPx5bR44sWF76uEqSyAxMjofXVI1YZ753J3zZX20iQKNmT7dNW-nA$, or unsubscribehttps://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AB4J2NESH6EHIJHEOEQQZJDVVL5IHANCNFSM54E6EB5A__;!!NZvER7FxgEiBAiR_!vzvNg0xBe0RIlkNkQa7IjhoGRoLwjbkPySNG0xiCcQcPx5bR44sWF76uEqSyAxMjofXVI1YZ753J3zZX20iQKNmTfiV94qk$. You are receiving this because you commented.Message ID: @.***>