Engage CLAW

[birkland]

Islandora CLAW PHP/Camel microservices share some similar patterns to API-X.  Take a look at CLAW microservices & middleware to see where similarities and differences in API-X lie. Be the point person for communication between API-X or Islandora team members. A face-to-face may be appropriate

[ruebot]

We're starting to break everything apart into their own repos, so this one might be more helpful https://github.com/Islandora-CLAW/alpaca

The PHP Microservices are here: https://github.com/islandora-claw/crayfish

[birkland]

Great, thanks for pointing that out!

[DiegoPino]

@birkland, can you point me to a brief specs/WIP of the needed stuff(framework definition) to make a fedora 4 external service talk to/expose via API-X? I want to look at the "common denominator" of what we are doing at CLAW v/s what is needed to weight effort/understand also divergence from our model.

[birkland]

Hi @DiegoPino The purpose of the in-progress design activities is precisely to develop such a specification, based largely on assembling our past activities into a cohesive whole.

So as far as finding a common denominator, at least from the standpoint on understanding API-X it may be best to trace its history. In particular:

• The initial white paper
• The high-level requirements wiki page
• Recent presentations from DC Fedora users group and LDCX
• The doc section of the code, as well as amherst-apix (and associated wiki page) and aic-validation proof-of-concepts.

For additional context, the DC Fedora users group presentation hints at some historic roots in the fedora 3 content model architecture. Some important characteristics of the fedora 3 CMA as far as we are concerned are:

• An object explicitly declares its participation in a model via a relationship to a content model (cModel) object
• Each cModel object may declare a number of 'services' that can be bound to objects of this model
  • These services are modeled as methods with parameters
  • Each service method for a given object maps to a unique URI. Clients invoke methods by performing a GET on such URI, with the appropriate parameters
• When a service is invoked by a client, the repository then invokes an implementing service. How it does so is defined by a WSDL document. Fedora may pass along additional parameters to this implementing service, such as the URI of a datastream. Fedora then forwards the response back to the client.

API-X is still interested in the general concept of binding services to repository objects, but differs significantly in implementation and intent. For example

• We want services to bind to objects based on describing objects that match (e.g. inferred membership in a class), rather than having objects explicitly declare relationships to special objects
• The repository is not the mediator between objects and services. API-X fulfills that role
• There are use cases that involve API-X being able to alter the representation of resources in the repository (e.g. add additional link headers, provide additional representations such as compacted json). Scenarios like these involve an API-X extension using a service to achieve some end, rather than exposing the service directly to the client)
• While extensions can expose services bound to objects at URIs akin to Fedora 3, API-X doesn't necessarily impose a specific interaction model, or conceptualize such bindings as 'methods'.

[DiegoPino]

@birkland extremely grateful for this extensive explanation. I will give this some thinking, particularly because fedora 3 CMA concept differs radically to what we would need to do based on our current RDF scenario where multiple rdf:type in a single Resource can for example be "bound" to different services(e.g of use case: ordering of execution can play a role) or if wan't to target whole subgraphs based on a local root resource definition (again, starting by rdf:type or by predicate selection?), like in the case of an aggregation. With inferred "membership to a class", you mean real infered based reasoning?

Thanks again for this.

[birkland]

@DiegoPino Yes, the history of Fedora 3 CMA is important because it provides an opportunity to reflect on what it did right, and what it did "wrong". Increasingly, I'm thinking that some degree of inference may be necessary for solving the binding problem. One of the defining characteristics of SSWAP is that does rely on real inferred reasoning, particularly in the discovery process. From Gessler et al. (2009)

the discovery server dereferences terms up to three levels of indirection in an attempt to broaden its knowledge of concepts (ontology terms) used by the resource description graph (RDG). We refer to the resulting set of RDF statements as a three degree closure. We then validate the graph for SSWAP consistency using the OWL reasoner Pellet. We use Pellet to make explicit any and all implicit statements. The reasoner performs the following four operations: i) classification: computing all subclass relations between named classes, ii) realization: assigning individuals to their most specific subclass, iii) consistency checking: complete check for logical contradictions, and iv) satisfiability checking: complete check for classes empty by necessity.

This addresses the "which data can this service operate on" mapping problem. The problem facing API-X is slightly different: "which data do we want this service to operate on". For API-X, it could possibly be more appropriate to bind based on something like SHACL, which can express the intent for an exact match, or specify an entailment that must be used. In either case, API-X is faced with an engineering challenge. SSWAP's reasoning is done at transaction-time, and is intended for asynchronous workflows. API-X needs to support synchronous scenarios, so might have motivation to pre-compute service bindings if inference is ultimately used, depending on the desired performance characteristics.

[DiegoPino]

Ok, i hope we got some free time to talk in person about this during OR2016. Pre-compute service sounds logic to me, even when this could be implemented via temporary caching system maybe?, since "which data do we want this service to operate on" could also be a user/config choice, which in turn could resolve via "which data can this service operate on" using one-time, or less-times calls. Will definitively have to dig deeper into SHACL, since it's the second time someone refers to it today! Is in Repo stored resource hinting or the use of annotations inside Resources also viable? I mean, in scenarios where same set of exposed services could work for different needs/ per class definition under a multi repo scenario?

[ajs6f]

I am really wishing I had the time right now to participate in this thread, but for lack thereof, I will offer the following suggestion: if we don't make some time to "talk in person about this during OR2016" it isn't likely to happen. @birkland , what do you say to trying to put together some kind of informal meetup for API-X?

[ruebot]

I'm game for a meet-up, and/or dinner and drinks :smile:

[ajs6f]

I'm game for @ruebot to buy me a drink.

[birkland]

That sounds great! Here's a "lunch & dinner time slots" doodle poll: http://doodle.com/poll/tfgpp5cmeqcrrrfb

If that doesn't work out, we can look at session times.

[ruebot]

@birkland can you put a time zone on that Doodle poll?

[whikloj]

@ruebot aren't you all going to be in the same time zone for this meet-up?

[ajs6f]

@whikloj He's thinking of jet lag. We will all physically be there, but some of us may be mentally thousands of miles away.

[acoburn]

Or in my case, I really will be thousands of miles away :-(

[jwestgard]

... me too, but there in spirit!

On May 27, 2016, at 15:43 PM, Aaron Coburn notifications@github.com wrote:

Or in my case, I really will be thousands of miles away :-(

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[whikloj]

Does Doodle have a setting for mental time? Cause mine is using Central and I could really use an extra 7 to 7000 minute delay.

[birkland]

Hm, apparently you can't add time zone support after the fact. I think the default behaviour is "no time zone", i.e. it has the same literal values for everyone. So I edited the description to specify Dublin's time zone (UTC +1).

[ajs6f]

@birkland Do you want to advertise this to the fedora-* lists?

[DiegoPino]

@birkland, just in case i was not explicit enough, please count me in for that OR2016-API-X meeting. Thanks.

[birkland]

Let's meet by the registration desk during lunch break to decide where to go

[birkland]

We've taken a table in the dining hall by the main (south) entrance

[birkland]

Here are my notes from the meeting. Please jump in to add or correct anything I may have missed

Nick Ruest (York University), Diego Pino Navarro, A. Soroka (UVA libraries), Andrew Woods (Duraspace), Dan Davis (Smithsonian), and myself met to discuss API-X.

Nick Ruest has graciously offered to be an API-X stakeholder on behalf of the Islandora/CLAW community. The current API-X meeting time works very well for all CLAW developers to participate.

There is a general notion that API-X will capture the consensus of the various members of the Fedora community who are interested in the general idea of binding services to the repository. Ideally, API-X will produce a declarative standard and/or set of best practices for representing the notion of service binding in repository objects such that multiple API-X implementation could exist.

It was noted that CLAW and API-X share several technical and philosophical similarities (e.g. both involve using (micro) services to expose new functionality on repository resources, but there are a few technical differences that may or may not be significant. For example, at present CLAW is bound to workflows involving Drupal, so is not entirely general purpose at the moment.

So once we’ve developed the initial version of API -X, we’ll compare implementations, refine the design docs based upon consensus among the API-X stakeholders Ontologies

Diego and Ruth are have similar views related to ontologies - i.e. that they must be in the repository because one cannot rely on resolving externally hosted ontologies.
At present, there are no known best practices for storing Ontologies in Fedora. Diego believes that classes and properties can be represented as individual Fedora objects, whilst Ruth’s opinion is that ontologies should be document-oriented.

The current API-X draft design proposes using ontologies (direct or inferred class membership) as a mechanism of service binding. Islandora is developing services to generate/populate UIs and enact policy based on ontology. So now that there are use cases that involve leveraging ontologies to provide functionality among at least two Fedora-related projects, API-X ought to become involved in the process of discovering best practice in the community.

[ruebot]

@birkland awesome! Tiny change: Nick Ruest (University of York) -> Nick Ruest (York University)

[ajs6f]

That's because Canadians, unlike Brits, put the surname after the given name.

On Jun 21, 2016, at 10:02 PM, Nick Ruest notifications@github.com wrote:

@birkland awesome! Tiny change: Nick Ruest (University of York) -> Nick Ruest (York University)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[ruebot]

...or people will think I'm at University of York, and British... again! On Jun 21, 2016 17:27, "A. Soroka" notifications@github.com wrote:

That's because Canadians, unlike Brits, put the surname after the given name.

On Jun 21, 2016, at 10:02 PM, Nick Ruest notifications@github.com wrote:

@birkland awesome! Tiny change: Nick Ruest (University of York) -> Nick Ruest (York University)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/fcrepo4-labs/fcrepo-api-x/issues/18#issuecomment-227577607, or mute the thread https://github.com/notifications/unsubscribe/AANVwV_FysbJwk3AfbXfBtvQJz2KqVGkks5qOFc5gaJpZM4IeOsj .

[dannylamb]

I'd say as of Islandora-CLAW/CLAW#504, we're officially engaged. Feel free to close this one @birkland.

[birkland]

@dannylamb Perfect!

[ajs6f]

Agreed about the ring having been, in fact, put upon it. On to https://github.com/Islandora-CLAW/CLAW/issues/308 and beyond!