search

usc-isi-i2 / Web-Karma

Information Integration Tool
http://www.isi.edu/integration/karma/
Apache License 2.0
586 stars 196 forks source link

empty response from RDF generation web service #347

Closed turbomam closed 6 years ago

turbomam commented 7 years ago

I just pulled the latest dev branch:

user@server:~/karmagit/Web-Karma$ git status
On branch development
Your branch is up-to-date with 'origin/development'.
Untracked files:
  (use "git add <file>..." to include in what will be committed)

        karma-web/nohup.out

nothing added to commit but untracked files present (use "git add" to track)

and compiled

user@server:~/karmagit/Web-Karma$ mvn clean install
[INFO] Scanning for projects...

... snip ...

[INFO] Reactor Summary:
[INFO]
[INFO] ISI Karma Maven Webapp ............................. SUCCESS [  0.900 s]
[INFO] karma-util ......................................... SUCCESS [  6.024 s]
[INFO] karma-typer ........................................ SUCCESS [  1.005 s]
[INFO] karma-common ....................................... SUCCESS [ 10.094 s]
[INFO] karma-commands ..................................... SUCCESS [  0.010 s]
[INFO] commands-common .................................... SUCCESS [  0.687 s]
[INFO] commands-update .................................... SUCCESS [  0.719 s]
[INFO] commands-worksheet ................................. SUCCESS [  1.781 s]
[INFO] commands-alignment ................................. SUCCESS [  1.540 s]
[INFO] commands-python .................................... SUCCESS [  1.840 s]
[INFO] commands-alignment-openrdf ......................... SUCCESS [  0.662 s]
[INFO] commands-publish ................................... SUCCESS [  0.771 s]
[INFO] karma-jdbc ......................................... SUCCESS [  0.466 s]
[INFO] commands-publish-jdbc .............................. SUCCESS [  0.102 s]
[INFO] commands-publish-alignment-openrdf ................. SUCCESS [  1.081 s]
[INFO] commands-update-jdbc ............................... SUCCESS [  0.271 s]
[INFO] commands-import .................................... SUCCESS [  0.015 s]
[INFO] import-common ...................................... SUCCESS [  0.303 s]
[INFO] import-csv ......................................... SUCCESS [  0.490 s]
[INFO] import-database .................................... SUCCESS [  0.395 s]
[INFO] import-excel ....................................... SUCCESS [  0.242 s]
[INFO] import-file ........................................ SUCCESS [  0.115 s]
[INFO] import-ontology .................................... SUCCESS [  0.267 s]
[INFO] import-json ........................................ SUCCESS [  0.311 s]
[INFO] import-service ..................................... SUCCESS [  0.285 s]
[INFO] import-spatial ..................................... SUCCESS [  0.633 s]
[INFO] import-xml ......................................... SUCCESS [  0.437 s]
[INFO] import-avro ........................................ SUCCESS [  0.312 s]
[INFO] commands-include ................................... SUCCESS [  0.014 s]
[INFO] karma-web .......................................... SUCCESS [ 16.364 s]
[INFO] karma-offline ...................................... SUCCESS [  1.952 s]
[INFO] commands-bloom ..................................... SUCCESS [  1.124 s]
[INFO] karma-jsonld ....................................... SUCCESS [  4.111 s]
[INFO] karma-mr ........................................... SUCCESS [  6.684 s]
[INFO] karma-storm ........................................ SUCCESS [  1.774 s]
[INFO] karma-web-services ................................. SUCCESS [  0.014 s]
[INFO] web-services-rdf ................................... SUCCESS [ 17.522 s]
[INFO] web-services-publish-es ............................ SUCCESS [ 24.028 s]
[INFO] karma-spark ........................................ SUCCESS [  3.525 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:50 min
[INFO] Finished at: 2017-08-14T12:38:14+00:00
[INFO] Final Memory: 159M/799M
[INFO] ------------------------------------------------------------------------

what ports are already in use?

user@server:~/karmagit/Web-Karma/karma-web-services/web-services-rdf$ sudo netstat -tupln | sed -r 's/[.:0-9]+://g' | sed -r 's/ +/ /g' | cut -f4,7 -d' ' | grep -v '^[^0-9]' | sort -n | uniq
22 1059/sshd
68
80 15026/apache2
111
111 14973/rpcbind
733
3306 20526/mysqld
3838 21867/shiny-server
5353
5820 21712/java
8005 21530/java
8009 21530/java
8080 21530/java
8787 10111/rserver
9999 21834/java
27017 21618/mongod
35843
57576

start the Karma RDF generation web service on avaialble port 9090

user@server:~/karmagit/Web-Karma$ cd karma-web-services/web-services-rdf
user@server:~/karmagit/Web-Karma/karma-web-services/web-services-rdf$ mvn -Djetty.port=9090 jetty:run
[INFO] Scanning for projects...

... snip ...

[INFO] Started ServerConnector@19ae0eaa{HTTP/1.1}{0.0.0.0:9090}
[INFO] Started Jetty Server

startup seems to take much longer than the interactive web karma server

at this point I started a new terminal session on the same server

some new java process is running on port 9090!

user@server:~$ sudo netstat -tupln | sed -r 's/[.:0-9]+://g' | sed -r 's/ +/ /g' | cut -f4,7 -d' ' | grep -v '^[^0-9]' | sort -n | uniq
22 1059/sshd
68
80 15026/apache2
111
111 14973/rpcbind
733
3306 20526/mysqld
3838 21867/shiny-server
5353
5820 21712/java
8005 21530/java
8009 21530/java
8080 21530/java
8787 10111/rserver
9090 23253/java
9999 21834/java
27017 21618/mongod
35843
57576

karma rdf service responds to OPTIONS request

user@server:~$ curl -vX OPTIONS localhost:9090
* Rebuilt URL to: localhost:9090/
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 9090 (#0)
> OPTIONS / HTTP/1.1
> Host: localhost:9090
> User-Agent: curl/7.47.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Allow: GET,HEAD,POST,OPTIONS
< Content-Length: 0
< Server: Jetty(9.0.5.v20130815)
<
* Connection #0 to host localhost left intact

model file is available:

user@server:~$ curl http://localhost/encounter_no_part4file-model.ttl | head
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 61602  100 61602    0     0  8551k      0 --:--:-- --:--:-- --:--:--  9.7M
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix pato: <http://purl.obolibrary.org/obo/pato#> .
@prefix protege: <http://protege.stanford.edu/plugins/owl/protege#> .
@prefix nci: <http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix dc: <http://purl.org/dc/elements/1.1/> .
@prefix obo: <http://purl.obolibrary.org/obo/> .
@prefix owl: <http://www.w3.org/2002/07/owl#> .
@prefix oboInOwl: <http://www.geneontology.org/formats/oboInOwl#> .
@prefix turbo: <http://turbo.org/> .

data file is available:

user@server:~$ curl http://localhost/encounter_data.csv | head
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   172  100   172    0     0  21869      0 --:--:-- --:--:-- --:--:-- 24571
EncID,PSC,EncDate,EncCode,CodeType
1,123,11/22/2016,401,ICD9
2,123,11/22/2016,401,ICD9
3,123,11/22/2016,458,ICD9
4,123,1/15/2017,823,ICD9
5,555,2/13/2017,B35.9,ICD10

so try submitting a request to the RDF generation service

note the empty result

user@server:~/karmagit/Web-Karma$ curl -v --request POST --data 'R2rmlURI=http://localhost/encounter_no_part4file-model.ttl&ContentType=CSV&DataURL=http://localhost/encounter_data.csv' http://localhost:9090/rdf/r2rml/rdf     
Note: Unnecessary use of -X or --request, POST is already inferred.
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 9090 (#0)
> POST /rdf/r2rml/rdf HTTP/1.1
> Host: localhost:9090
> User-Agent: curl/7.47.0
> Accept: */*
> Content-Length: 118
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 118 out of 118 bytes
< HTTP/1.1 200 OK
< Content-Type: text/plain
< Transfer-Encoding: chunked
< Server: Jetty(9.0.5.v20130815)
<

* Connection #0 to host localhost left intact
dkapoor commented 7 years ago

Hi Mark, Can you also share the R2RML model file that you generate? Are you running the Karma UI and service on the same machine? Are they both running at the same time?

Thanks.

turbomam commented 7 years ago

When I did the exercise above, no other Karma process was running on the same server. I do have other copies of the Web-Karma repo in other folders, and I have run the web interface in the past.

The R2RML model is 61KB. Do you want me to paste the whole thing in here?

dkapoor commented 7 years ago

You can just drag and drop the file in the text area and it would attach the file.

dkapoor commented 7 years ago

Can you attach the R2RML model: encounter_no_part4file-model.ttl. The file that you uploaded is a different one.

turbomam commented 7 years ago

Whoa! That's wasn't even an R2RML file at all! Sorry.

encounter_no_part4file-model.ttl.txt

dkapoor commented 7 years ago

So, the CSV file is missing two fields: PK and DiagCode that are being used in the model. Can you attach some lines from the original CSV file that you used for modeling and also attach as a zip file your karma home folder. Thanks.

turbomam commented 7 years ago

I created a new and extremely simple ontology, CSV data file and R2RML mapping file. The web service does return RDF triples from the about this new simple data now. You can close the case if you want, or I can send the files you requested on the 21st.

Thanks for your feedback. It helped me think through things I might have been doing wrong.

dkapoor commented 7 years ago

We will mark this as closed in that case and you can reopen it if you see any issues again.