jmandel commented 12 years ago

API Design Sketched out

Allow GET parameters on the existing "fetch all" calls starting with:

/records/xxx/lab_results/
/records/xxx/vital_signs/.

Each parameter decomposes to a rule that can be expressed in SPARQL.

Specific rules for each clinical statement type

Rules for LabResults

`loinc=x|y|z` →

sp:labName/sp:code ?l. 
FILTER(
   ?l = uri(<http://purl.bioontology.org/ontology/LNC/x>) ||
   ?l = uri(<http://purl.bioontology.org/ontology/LNC/y>) ||
   ?l = uri(<http://purl.bioontology.org/ontology/LNC/z>)
)

`date_from=x` →

sp:specimenCollected/sp:startDate ?d. 
FILTER(?d >= x).
)

Rules for VitalSigns

? Determine whether to support queries across clinical statements (vitals → encounter)

`encounter_type=x`→

?v sp:encounter/sp:encounterType/sp:code ?c. 
FILTER(c=uri(<http://smartplatforms.org/terms/codes/EncounterType#x>’)

Each API response includes a Summary

Each API response comes with a spapi:ResponseSummary attached. For example

[] a spapi:ResponseSummary; 
smartapi:processingTimeMs: "120";
smartapi:nextPageURL: "http://sandbox-api.smartplatforms.org/records/123/medications/1235?limit=10&offset=20";
smartapi:resultsReturned: 7;
smartapi:totalResultCount: 200;
smartapi:resultOrder (<http://sandbox-api.smartplatforms.org/records/123/medications/1235>,
<http://sandbox-api.smartplatforms.org/records/123/medications/963>,
<http://sandbox-api.smartplatforms.org/records/123/medications/8254>,
<http://sandbox-api.smartplatforms.org/records/123/medications/9732>,
<http://sandbox-api.smartplatforms.org/records/123/medications/235>,
<http://sandbox-api.smartplatforms.org/records/123/medications/87342>,
<http://sandbox-api.smartplatforms.org/records/123/medications/2336>);

jmandel commented 12 years ago

To-date

Sketched API Design Implemented Prototype for Filtering and Pagination

Next Steps

Review query generation mechanism in smart_server and identify hook points for filters Refactor code and update clients (with the new ontology calls model and filters parameters)

nschwertner commented 12 years ago

To Do:

Support for sorting
"Response summary" object
Code refactoring
API Playground (checkout the ontology updates)
Apps (BPC - time range)

nschwertner commented 12 years ago

First pass on refactoring the code: https://github.com/chb/smart_server/commit/8ccf8542a234aa320c8525f035dedeeb8a64195e

jmandel commented 12 years ago

You might let each class manage its own paginator + register a set of filtwr handlers in a dictionary that can be discovered at runtime.

Rather than writing procedural code that explicitly names each handler... On Aug 2, 2012 11:48 AM, "nschwertner" < reply@reply.github.com> wrote:

First pass on refactoring the code:

https://github.com/chb/smart_server/commit/8ccf8542a234aa320c8525f035dedeeb8a64195e

Reply to this email directly or view it on GitHub:

https://github.com/chb/smart_project_management/issues/2#issuecomment-7459232

nschwertner commented 12 years ago

You mean that each API call handler should look up its filters+paginator (from a table?) and then provide them in a dictionary to the tripplestore record builder? I.e. push approach (vs. the current pull approach for discovery)?

nschwertner commented 12 years ago

Complete implementation (with sorting, response summary, etc): https://github.com/chb/smart_server/commit/2e2270a555bb6a471ae4c5d265b3c555430a9f00

jmandel commented 12 years ago

Updated the ontology to include filters + default sorts directly as properties on API calls.
Merged the finger-grained-api-calls branch into https://github.com/chb/smart_server/tree/adding-0.5-models
Simplified the code (more declarative now) by using values from the ontology

Previously the code was searching the entire triples store (not just an individual record!) when applying filters and paginating; this may work on small data sets but it will crash and burn on larger sets. These changes try to keep all search operations scoped as narrowly as possible (first to a given record; then to matching results within the record).

@nschwertner: can you merge these changes back into your branch and keep working from there?

Also quick note: result order isn't working (RDF/XML triples aren't being generated correctly)

nschwertner commented 12 years ago

Impressive. The code looks crisper with the ontology values. Also quite educational in terms of python patterns.

I did some cleanup on the code, tested it, and cleaned up a couple bugs. Also added the date_from filter to the ontology. See https://github.com/chb/smart_server/tree/finer_grained_api

@jmandel The result order seems all right to me. Not sure what the problem that you are referring to is. Please clarify.

Moving on to the clients...

jmandel commented 12 years ago

@jmandel https://github.com/jmandel The result order seems all right to me. Not sure what the problem that you are referring to is. Please clarify.

In the RDF/XML generated, I wasn't seeing rdf:first and rdf:rest predicates creating a list; instead I was seeing HTML-escaped angle brackets...

The thing I expect is like:

Go to http://any23.org
Paste in

@prefix rdf: .
@prefix s:   .


    s:students (
        
        
        
    ).

Choose "rdfxml" or "ntriples" output and Convert.

nschwertner commented 12 years ago

Quick question... I wasn't able to get results for the encounters and the vital signs on the current finer_grained_api has the adding-0.5-models merged in. I traced the encounters problem to a typo in the ontology, which is fixed here (together with adding the default sort predicates for a few calls): https://github.com/chb/smart_common/commit/39c0e0e63111053f30afcb6f47bf4007cd8e3c23

However, I still cannot get results from the vital signs call. I noticed that it is now http://smart-vm:7000/records/.../vital_sign_sets/ and tried it in place of the usual call. It works, but returns an empty RDF. What is the story for the vital signs changes? Should I rebuild my local database/triplestore?

nschwertner commented 12 years ago

The turtle fragment that you suggested didn't work out, but the following did work (from http://www.w3.org/TR/rdf-primer/#collections ):

<?xml version="1.0"?>
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
         xmlns:s="http://example.org/students/vocab#">

   <rdf:Description rdf:about="http://example.org/courses/6.001">
      <s:students rdf:parseType="Collection">
            <rdf:Description rdf:about="http://example.org/students/Amy"/>
            <rdf:Description rdf:about="http://example.org/students/Mohamed"/>
            <rdf:Description rdf:about="http://example.org/students/Johann"/>
      </s:students>
   </rdf:Description>
</rdf:RDF>

The output looks like this:

<http://example.org/courses/6.001> <http://example.org/students/vocab#students> _:node174fmimjnx9342 .
_:node174fmimjnx9342 <http://www.w3.org/1999/02/22-rdf-syntax-ns#first> <http://example.org/students/Amy> .
_:node174fmimjnx9342 <http://www.w3.org/1999/02/22-rdf-syntax-ns#rest> _:node174fmimjnx9343 .
_:node174fmimjnx9343 <http://www.w3.org/1999/02/22-rdf-syntax-ns#first> <http://example.org/students/Mohamed> .
_:node174fmimjnx9343 <http://www.w3.org/1999/02/22-rdf-syntax-ns#rest> _:node174fmimjnx9344 .
_:node174fmimjnx9344 <http://www.w3.org/1999/02/22-rdf-syntax-ns#first> <http://example.org/students/Johann> .
_:node174fmimjnx9344 <http://www.w3.org/1999/02/22-rdf-syntax-ns#rest> <http://www.w3.org/1999/02/22-rdf-syntax-ns#nil> .

A sort of linked list structure implemented as triples, it seems. The only questions is what is the best way to build this with rdflib. I am trying to find out...

jmandel commented 12 years ago

Trouble with the way GitHub was applying markdown to my e-mailed response. The turtle representation of a list should look like like:


@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>.
@prefix s:   <http://example.org/students/vocab#>.

<http://example.org/courses/6.001>
    s:students (
        <http://example.org/students/Amy>
        <http://example.org/students/Mohamed>
        <http://example.org/students/Johann>
    ).

As for Vitals, I've renamed the class to "VitalSignSet" to better indicate that one statements consists of a group of readings together. The patient generator will need to follow suit (I haven't updated it yet.)

jmandel commented 12 years ago

On today's call we decided to:

disable paging by default (i.e. default page size == infinity)
allow paging when specified by the client (in terms of limit and/or offset)

nschwertner commented 12 years ago

Lift the default limits on the pagination https://github.com/chb/smart_server/commit/0615b324fdd33f617c0dde0d2d4a58cc74b13ba0

Adding RDF Collections output to the ResponseSummary https://github.com/chb/smart_server/commit/a5919436010b9ee044b4e65cfde8fd97c865a5c0

Implement date_to and date_from filters for the vital sign sets https://github.com/chb/smart_common/commit/d675714064d1e66e766bca47028faa2e2cd935bd

smart-classic / smart-issues

Fine-grained API Calls #2

API Design Sketched out

Specific rules for each clinical statement type

Rules for LabResults

`loinc=x|y|z` →

`date_from=x` →

Rules for VitalSigns

`encounter_type=x`→

Each API response includes a Summary

To-date

Next Steps

smart-classic / smart-issues

Fine-grained API Calls #2

API Design Sketched out

Specific rules for each clinical statement type

Rules for LabResults

loinc=x|y|z →

date_from=x →

Rules for VitalSigns

encounter_type=x→

Each API response includes a Summary

To-date

Next Steps

`loinc=x|y|z` →

`date_from=x` →

`encounter_type=x`→