Activity types/units API calls don’t retrieve data in 2.2

[jakhag]

The following API calls are working in 2.1 but not in 2.2:

Activity Types: List 
In 2.2: 404 Not Found  http://alpha.openphacts.org:3002/pharmacology/filters/activities?app_id=f91c5b2b&app_key=18a5d823d0e4933ac5fe22a3d52974c1&activity_unit=centimeter In 2.1: https://beta.openphacts.org/2.1/pharmacology/filters/activities?app_id=f91c5b2b&app_key=18a5d823d0e4933ac5fe22a3d52974c1&activity_unit=centimeter

Activity Types: Count  In 2.2: 404 Not Found  http://alpha.openphacts.org:3002/pharmacology/filters/activities/count?app_id=f91c5b2b&app_key=18a5d823d0e4933ac5fe22a3d52974c1&activity_unit=centimeter In 2.1: https://beta.openphacts.org/2.1/pharmacology/filters/activities/count?app_id=f91c5b2b&app_key=18a5d823d0e4933ac5fe22a3d52974c1&activity_unit=centimeter

Activity Units: List 
In 2.2: 404 Not Found  http://alpha.openphacts.org:3002/pharmacology/filters/units/pages?app_id=f91c5b2b&app_key=18a5d823d0e4933ac5fe22a3d52974c1 In 2.1: https://beta.openphacts.org/2.1/pharmacology/filters/units/pages?app_id=f91c5b2b&app_key=18a5d823d0e4933ac5fe22a3d52974c1

[stain]

Look for "Created activity types" in https://github.com/openphacts/ops-platform-setup/tree/2.0.0/data-sources - you have to create some derived data.

See also https://github.com/openphacts/ops-chembl-rdf/blob/data-plugin/data/insert_queries.sparql

[egonw]

There are two bugs here:

1. when the SPARQL does not return something, the LDA layer fails causing a 404 instead of a 200. That's a software bug.
2. the other issue is that the ChEMBL-RDF changed and the LDA specification was not updated

I'll make a patch for the second issue; that should solve the problem, but I really think the software need to be fixed too (bug 1 above).

[egonw]

Here' the patch for counting the act types again:

https://github.com/openphacts/OPS_LinkedDataApi/pull/23

Please confirm this fixes the second 404.

[randykerber]

The second query (Activity Types: Count) now returns a result.

[randykerber]

First query works for v2.1 Virtuoso fails with v2.2 Virtuoso.

Third query fails on all combos of v2.1 and v2.2 IMS and Virtuoso.

[egonw]

Cool, that seems to confirm that my SPARQL update was correct after all... I will make a PR to fix the related queries today then!

[egonw]

@randykerber I cannot confirm the second query to be fixed in ops4j yet :(

Also, I checked and all other queries actually were doing the right thing, and already used 'a' instead of 'subclass of'... so, that won't help the other two, I guess... but I'll explore.

[randykerber]

@egonw: I'm running and debugging the LDA only locally on my laptop right now. Latest changes have not been pushed to alpha.openphacts.org yet, so I'm guessing that's the reason we get different answers with second query. I'll let you know when change is applied to alpha.

[egonw]

Ah, of course... OK, can you give me a full list of current fails with your local instance? (some exploration of the two remaining fails of this original issue, I have yet to figure out what goes wrong...)

[randykerber]

@egonw -- Check the following file: https://github.com/openphacts/GLOBAL/blob/master/issues/problem-calls-encoded.csv

This is current list of GitHub issues I've been testing. First field is issue number, second is which I think are fixed (closed, at least run from my local LDA), and last field is the LDA command (minus host and app_id and app_key).

I've created a shell script to run through this csv file and test combos of IMS and SPARQL endpoints. It's a "work in progress". https://github.com/openphacts/GLOBAL/tree/master/issues

[randykerber]

The latest LDA version is in the "endpoints-in-request" branch if you want to take a shot at running it locally. Ian and I are working on it to make it more useful for debugging. Check with us for updates. https://github.com/openphacts/OPS_LinkedDataApi/tree/endpoints-in-request

[randykerber]

@egonw -- there's now an updated version of LDA running at http://alpha.openphacts.org:3002/.

So should be able to run ops4j again.

[ianwdunlop]

QUDT labels are missing ?item rdfs:label ?qudt_label returns nothing in the /pharmacology/filters/units/pages API call

PREFIX chembl: <http://rdf.ebi.ac.uk/terms/chembl#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX chembl: <http://rdf.ebi.ac.uk/terms/chembl#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT DISTINCT ?item  WHERE {
  {
    SELECT DISTINCT ?item ?qudt_label (COUNT(?activity) AS ?count) WHERE {
    GRAPH <http://www.ebi.ac.uk/chembl>  {
    ?activity chembl:hasQUDT ?item .
    ?item rdfs:label ?qudt_label
  }
  } GROUP BY ?item ?qudt_label ORDER BY DESC( ?count )
  } 
  }   LIMIT 10 OFFSET 0

The OpenPHACTS units have been added but probably need to add the QUDT units from the QUDT Catalog

[randykerber]

2 files on Units are loaded into Virtuoso in the graph http://rdf.openphacts.org/graph/qudt:

• http://qudt.org/1.1/vocab/OVG_units-qudt-(v1.1).ttl
• https://github.com/openphacts/jqudt/blob/master/src/main/resources/onto/ops.ttl

[randykerber]

All test commands now appear to be working.

[jakhag]

Checked them with my testing workflow, all three commands are working.

[egonw]

We can get stuff including, as QUDT is getting new energy. I met Stuart a few weeks ago, and he's a very open science chemist: https://units.unf.edu/

[Chris-Evelo]

I think I remember that we included QUDT because ChEMBL was using different units for the same thing. But then ChEMBL harmonised it so QUDT was no longer needed. Maybe we disabled some things after that?