Help with SDI - how to call a capability

vangelisv commented 9 years ago

Hi, we've created in Fispace EE, a test capability (id : 46) of an existing capability type (id:9).

Querying SDI (http://37.131.251.117:8080/sdi/api/capabilities/) we get correct XML: `...

46

<description>createFarm</description>
<uri>http://ifarma.agrostis.gr</uri>
<cte_id>9</cte_id>

...`

We try now to call this capability (POST to it from a client). In SDI documentation (FIspace-SDi-Tech.pdf, ed. 27-02-2015, page 15), it reads:

POST /api/capabilities/{request_message}/(capabilityId) i.e. request_message is “provide_weather_conditions”. This would be an example of a typical call of >a client that uses a specific capability of type “provide weather conditions”. The capabilityId is used to >define the specific implementation (i.e. the specific weather conditions provider) that should be used.

We query SDI http://37.131.251.117:8080/sdi/api/capability-types/9 (GET) to get

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<capabilityType xmlns="http://www.limetri.eu/schemas/ygg">
  <id>9</id>
  <name>provide advice</name>
  <requestMessageType>ProvideAdviceRequestMessage</requestMessageType>
  <responseMessageType>ProvideAdviceResponseMessage</responseMessageType>
  <schemaLocation>classpath:/schema/domain/ag/AGMessages.xsd</schemaLocation>
  <contextPath>eu.fispace.api.ag</contextPath>
</capabilityType>

thus we try to call the capability by POSTting to http://37.131.251.117:8080/sdi/api/capabilities/ProvideAdviceRequestMessage/46 but we get

<html>
<head>
   <title>Status page</title>
</head>
<body style="font-family: sans-serif;">
<p style="font-size: 1.2em;font-weight: bold;margin: 1em 0px;">Not Found</p>
<p>The server has not found anything matching the request URI</p>
<p>You can get technical details <a href="http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.5">here</a>.<br>
Please continue your visit at our <a href="/">home page</a>.
</p>
</body>
</html>

According to https://bitbucket.org/fispace/phase3support/issues/21/test-sdi-issue-95-bitbucket-apps the capability should be called with POST to http://37.131.251.117:8080/sdi/api/capabilities/{capabiliy description in lower case}/use Still, POSTing to http://37.131.251.117:8080/sdi/api/capabilities/createfarm/use we get again

<html>
<head>
   <title>Status page</title>
</head>
<body style="font-family: sans-serif;">
<p style="font-size: 1.2em;font-weight: bold;margin: 1em 0px;">Not Found</p>
<p>The server has not found anything matching the request URI</p>
<p>You can get technical details <a href="http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.5">here</a>.<br>
Please continue your visit at our <a href="/">home page</a>.
</p>
</body>
</html>

Can you please HELP? Also, I have is it possible to just call a capability (with a Request message) and get back the response fro the capability implementation (REST API) without involving a business process and the BCM?

perezdf commented 9 years ago

are you implemting in java?

Using eu.limetri.ygg.client.YggClientSimple is as simple as


    public Response run(@DefaultValue("") @QueryParam("token") String token)
            throws AuthenticationException, IOException, HttpFailure {

        // retrieve token
        client.setTokenString(token);

        ProvideAdviceRequestMessage requestAdvice = new ProvideAdviceRequestMessage();
        requestAdvice.setBusinessProcessId(yourBP.getId());

        client.use(provideAdviceCP, requestAdvice);

        return Response.ok("ok").build();
    }

vangelisv commented 9 years ago

No, our client is in javascript (Wirecloud widget).

perezdf commented 9 years ago

I have been investigating the issue by my own application. I am wondering there are a bug in the documentation, rather than

POST /api/capabilities/{request_message}/(capabilityId)

try with

POST /api/capabilities/{capability type name}/use

Note: if the capability type names contains spaces, you should replace them by '_' underlines

I can not be 100% sure because my app is based on java, but taking a look logs and debugging YggClientSimple, I am 95% sure.

I hope, it could help you

vangelisv commented 9 years ago

I've created a new capability (id 6), (btw all existing capabilities have been deleted?), with capability type id 11

    <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
    <capabilityType xmlns="http://www.limetri.eu/schemas/ygg">
      <id>11</id>
      <name>cropfield details</name>
      <requestMessageType>RequestMessage</requestMessageType>
      <responseMessageType>CropFieldDetailMessage</responseMessageType>
      <schemaLocation>classpath:/schema/domain/ag/AGMessages.xsd</schemaLocation>
      <contextPath>eu.fispace.api.ag</contextPath>
    </capabilityType>

As you suggest then I should POST /api/capabilities/cropfield_details/use. I tried it and it gives:

java.lang.NullPointerException
    at java.io.StringReader.<init>(StringReader.java:50)
    at eu.limetri.ygg.server.camel.UnMarshallProcessor.unmarshallXml(UnMarshallProcessor.java:105)
    at eu.limetri.ygg.server.camel.UnMarshallProcessor.process(UnMarshallProcessor.java:87)
    at org.apache.camel.processor.DelegateSyncProcessor.process(DelegateSyncProcessor.java:63)
... and more follows

A. How can I find an example of a valid RequestMessage for this call?

B. I'm somehow puzzled here: since I POST to the capability type, how can SDI redirect the call to my capability (REST service) and not to any other capability of this type?

perezdf commented 9 years ago

I can set up a sniffer on my app, I dont have such kind of problem and communication is carry on

perezdf commented 9 years ago

btw just a quick overview, is RequestMessage a message in AG? an CropFieldDetailMessage and element, not a type?

I think RequestMessage belongs to ygg and CropFieldDetailMessage doesnt look a element, but it is just a quick perception

vangelisv commented 9 years ago

I have no idea what a RequestMessage is, nor CropFieldDetailMessage. These are referenced in the capability type 11, already in FiSpace EE. This is what I'm asking, how can I find these definitions, is there any URL or documentation on the messages in AG?

Secondly, I did not understand your comment about the sniffer in your app. What do you mean?

perezdf commented 9 years ago

well, who knows the accuracy of this capability...

regarding of definition and structure of messages

http://central.maven.org/maven2/eu/fispace/api/0.14.3/api-0.14.3-sources.jar

even is a .jar extention you can unzip as usually

sniifer is to record the http communication

vangelisv commented 9 years ago

AGMessages.xsd (in the jar you've sent me), references (imports) a number of other xsd's (e.g. )

Where can I find those?

perezdf commented 9 years ago

https://bitbucket.org/fispace/core/downloads in this specific case,

https://bitbucket.org/fispace/core/get/0.14.3.zip

vangelisv commented 9 years ago

Can't find them there...

perezdf commented 9 years ago

https://bitbucket.org/fispace/core/src/e3f9aed2923877404df421e7da7a7167c24d4b6d/api/src/main/resources/schema/domain/?at=0.14.3

vangelisv commented 9 years ago

These I've seen, allright. I'm missing the xsds that are imported from AGMessages.xsd, For example I cannot find the

<xsd:import namespace="http://www.limetri.eu/schemas/ygg"/>

vangelisv commented 9 years ago

The reason I'm asking is to be able to generate a sample request message (XML) in order to test the POST to the capability (type)

perezdf commented 9 years ago

Feel free to ask what you would need, I will try as much I can ;-), this is a voluntary activity I am carrying

https://bitbucket.org/limetri/ygg/src/cd48af4eada2ec604b9e7fa35b293cb70b20a182/api/src/main/resources/eu/limetri/ygg/schema/?at=default

perezdf commented 9 years ago

it will be also interested this documentation https://bitbucket.org/limetri/ygg/

vangelisv commented 9 years ago

Good! Some progress. I'm POSTing to http://37.131.251.117:8080/sdi/api/capabilities/cropfield_details/use

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<CropFieldAdviceRequestMessage xmlns:ns1="http://www.limetri.eu/schemas/ygg" xmlns="http://www.fispace.eu/domain/ag">
</CropFieldAdviceRequestMessage>

Response from SDI is : Not Found, specified resource does not exist

What does this mean? What is the "specified resource"?

perezdf commented 9 years ago

well, the xsd defines some attributes right? are you forced to use javascript? I think with a strong complex type language as java may guide you better because the data structure are statically check by the compiler.

By the way, I feel (based on the aleatoriability of the capability you are selected) you are starting to attempt to use the scenario for the code directly.

Let me ask, if you have already defined a business flow in any format (diagram, paper, doc, etc...) I strongly recommend you a top bottom approach.

vangelisv commented 9 years ago

We develop a set of widgets to provide farm financial analysis services, based on our backend FMIS, ifarma. ifarma is a full featured FMIS, based on an elaborate mySQL database and PHP. We've developed services (REST API), in PHP to provide farm financial analyses. These API services need now to be exposed / called as SDI/Fispace capabilities. The widgets have been developed in Wirecloud GE (hence the Javascript) and will call the SDI capabilities, and these, in turn will call REST APIs. Not only the business flow is defined but the development is almost complete and we are now at unit and system testing. We will have the prototype ready in a couple of weeks. What we are missing is the integration with SDI.

perezdf commented 9 years ago

I see, if I can help you, do not hesitate. Good luck

vangelisv commented 9 years ago

Thanks for all the help. What would greatly help is a simple but end-to-end working example: A client calling a capability, implemented by a REST API (backend). Code for client, code for backend, capability and type definitions, messages (request and response) definitions and how you put all these together in FiSpace. I know that a new version of the Fispace (and the documentation) is on the way. It was supposed to be delivered in June. Any idea what is the latest status?

perezdf commented 9 years ago

https://bitbucket.org/fispace/archetypes/src/763e6044ad86/app-decoupled/?at=default

sadly is based on cycle 15, some adjustment should be doned

perezdf commented 9 years ago

https://bitbucket.org/perezdf/open-owm-adapter

Also based on cycle 15

vangelisv commented 9 years ago

I've seen previous versions of this code. Is there an example of a client USING a capability in a language other than java? Also in the decoupled project, the backend is not really defining a capability. It is using a capability i.e. it is also a client. We have a REST API, exposed at a URI, say http://agrostis.gr/someservice.php We need to define a capabiity for this REST API and then USE this capability from a javascript client, POSTing a request message and receiving a Response. Is there an example of this?

perezdf commented 9 years ago

hi,

trying to respond your question,

I've seen previous versions of this code. Is there an example of a client USING a capability in a language other than java?

there are some example of connection to FIspace using javascript and PHP, also I hear there are people using .NET. What I can do it is to extend the open-own-adapter with a javascript client.

Also in the decoupled project, the backend is not really defining a capability. It is using a capability i.e. it is also a client.

It is not mandatory to create the capabilites from the app, you can configure using frontend and let the app just for the usage. However, open-owm-adapter implements all necessary steps to configure a business process. I have just released version 0.3.0 which currently is deployed in:

http://195.87.138.166:8088/wc-provider/ http://195.87.138.166:8089/app/

Also working on the documentation:

http://perezdf.bitbucket.org

We have a REST API, exposed at a URI, say http://agrostis.gr/someservice.php We need to define a capabiity for this REST API and then USE this capability from a javascript client, POSTing a request message and receiving a Response.

I am working on an step by step tutorial. First step is to have a draft of the business flow. Second, to reuse existing messages of core or create new ones. Third, create capability types and business process template. Fourth, capabilities and business processes

vangelisv commented 9 years ago

Ii seems to me that, maybe, the solution to my problem in understanding lies somewhere around your Notice at the end of Step 1 of your tutorial:

Sequential diagram depicts two alternatives when weather provider is called. Either the WC provider is able to response inmediatly, so, it can generate the weather resource and return a 303 (redirection) to where the resource can be obtained. Or, WC provider returns a 204 and prepare the report until a Receve Resource Availability Cap is called.

Specific capability/Receive Resource Available Notification is the standard pattern design in Fispace. An app is able to generated any value added item for a user in a specific cap. The user calls this specific cap and in order to retrieve the item, it needs to implement a Receive Resource Available Notification cap.

Could you please try to explain what you mean there in simpler terms?

perezdf commented 9 years ago

"Understanding lies"?

perezdf commented 9 years ago

standardfispacebusinessflow - new page

Maybe to depict the simplest case is easier.

Basically, consumer calls to a capability. Fispace connects to the producer which implements it. When the producer is able to respond directly at once (no other process are involved, the execution time is less than time out, it is a simple process), the producer informs Fispace about where the resource is available. In order to inform about it, http 303 is used, so Fispace embeeds this URI in a Resource Available Resource Request and sends to the capability consumer (which should implement this capability). Consumer receives the URI to obtain the resource and retrieve it. The resource doesnt go across Fispace but Fispace considers complete the business process.

vangelisv commented 9 years ago

Yes, this is much more clear now. I still though a couple of questions: Every consumer has to implement the RECEIVE_RESOURCE_AVAILABLE_NOTIFICATION capability. But if the consumer calls/posts to different capabilities or even make multiple posts to the same capability how does FISpace or the consumer itself distinguish, for which initial post to the capability, the RECEIVE_RESOURCE_AVAILABLE_NOTIFICATION is called for? Example. I have consumer, say consumer1, makes 3 posts: 1) Post to Capability1 (provider 1) 2) Post to Capability2 (provider 2), message 2 3) Post to Capability2 (provider 2), message 3 FISpace posts to providers 1 and 2, suppose all results are ready, 303 URI are returned by all 3 and they are posted through RECEIVE_RESOURCE_AVAILABLE_NOTIFICATION to the consumer1.

Questions: 1/ How can the consumer tell which of the received URIs correspond to which of the original posts 2/ Where/How, in SDI configuration the RECEIVE_RESOURCE_AVAILABLE_NOTIFICATION implementation for a consumer is declared?

perezdf commented 9 years ago

1/ How can the consumer tell which of the received URIs correspond to which of the original posts

(in a simple scenario) because the key is one step backwards, the business process.

Supported by real runnable example (http://195.87.138.166:8080/sdi/admin-api/business-processes)

<ns2:businessProcess>
<ns2:id>6</ns2:id>
<ns2:name>WC_BP_9627a3ff-1c38-4be4-b54b-37b78488e41c</ns2:name>
<ns2:bpt_id>3</ns2:bpt_id>
<ns2:capabilities>
<ns2:id>9</ns2:id>
<ns2:description>Capability To provide weather condition</ns2:description>
<ns2:uri>
http://195.87.138.166:8088/wc-provider/backend/HanlderBPResource/PROVIDE_WEATHER_CONDITIONS
</ns2:uri>
<ns2:cte_id>40</ns2:cte_id>
</ns2:capabilities>
<ns2:capabilities>
<ns2:id>15</ns2:id>
<ns2:description>Received Resource Av. Notific Cap</ns2:description>
<ns2:uri>
http://195.87.138.166:8089/app/backend/OWMAdapterResource/RECEIVE_RESOURCE_AVAILABLE_NOTIFICATION
</ns2:uri>
<ns2:cte_id>4</ns2:cte_id>
</ns2:capabilities>
</ns2:businessProcess>

I am 195.87.138.166:8089, I want to consume PROVIDE_WEATHER_CONDITIONS provided by 195.87.138.166:8088. I register my capability to receive the resource. I make a contract by the business process.

8089 calls, 8088 receives a request from Fispace, and responds, *Fispace to the one who is interested to this resource, this is the URL", Fispace detects the 303 and search for the one interested to one resource (8089).

80XX wants also to consume PROVIDE_WEATHER_CONDITIONS, he needs a business process.

_2/ Where/How, in SDI configuration the RECEIVE_RESOURCE_AVAILABLENOTIFICATION implementation for a consumer is declared?

RECEIVE_RESOURCE_AVAILABLE_NOTIFICATION is just other capability. For this reason, just in case the producer can nt response at once time, if they dont send the 303 because the process takes too much time, the producer has to use RECEIVE_RESOURCE_AVAILABLE_NOTIFICATION explicitly

vangelisv commented 9 years ago

Great, the picture gets more and more clear but let me clarify some things and ask more questions:

Clarification 1: Each USE of a capability is (made in the context of) a separate "instance" of a business process, i.e it has a separate bp_id. If consumer 8089 wants to make 2 requests to the 8088 provider, then it needs to create 2 business processes with the same capabilities? If this is correct then somehow the bp_id has to be included in the original request so FIspace knows which instance it is serving each time. This means that before using/posting to a capability, the consumer needs to call the SDI API to create a new business process. Correct? Also the consumer should "make a note" of the BP ID and then expect it in the response notification message?

Clarification 2:
What makes FISpace know that one capability in a business process is the consumer/final receiver and the other is the provider? I mean, in the general case, a business process can have 3 or more capabilities. How does FISpace know to which capability to send the 303 uri. Is it mandatory that in a business process there are always just 2 capabilities and the one is from the provider and the other from the receiver/consumer? Maybe that the consumer/receiver is ALWAYS the one with a capability with a capability type that has <ns2:requestMessageType>ResourceAvailableNotification</ns2:requestMessageType>

perezdf commented 9 years ago

Maybe now , you understand my concerns about the necessity first to have a rough business flow diagram before starting development or achive to register a cap.

Bp-id is part of any request. You can call several times but not concurrent. Consumer has to make the bp. Think about simple case of customer buying a piece of bread. Backeries has capability to sell bread (in general) . one specific customer decides to instanciate this capability and make use of it. Go to the backery an use this capability informing implicity about he is able to take the bread. Backer informs about availability of th resource and tell ... Here you have. Customer takes the bread a leave. You can repeat the process, other customer comes and he has a different capability to receive the resource, it is different target.

In a more complex business process with several caps and different resources, you can define roles.

vangelisv commented 9 years ago

2 other, not specific to the issue, questions: What is status of the new version of FISpace (0.15) and its documentation? Are you going to be in the FISpace face to face workshop on Sep 2nd in Bremen?

vangelisv commented 9 years ago

We have very specific business processes defined. We are running 3 projects with Fiware accelerators and we have detailed specs of our business processes. I don't want to take your time otherwise I could send you our technical specs. In our simplest project/case, your simple scenario is enough. We just send a request to a backend service (provider) and receive the result (that is directly available - the 303 case). The difference is that in our design the consumer is just a front-end client (javascript widget in Fispace's wirecloud) so it cannot implement the receive_notification capability. In this case we might need to just communicate directly with the backend (our provider).

perezdf commented 9 years ago

*2 other, not specific to the issue, questions: What is status of the new version of FISpace (0.15) and its documentation? Are you going to be in the FISpace face to face workshop on Sep 2nd in Bremen?"

Supposely, the training session will be based on 0.15, anyway, integration environment is also upgraded.

I am not involved in the official documentation activities but I have decided to go on my way creating the perezdf.bitbucket.org site, so the experience of development and for example this thread can be return to this site.

I am going to Bremen, I have a session, basically to show open-owm-adapter, so you can skip it since you know already all I am going to talk about ;-)

We have very specific business processes defined. We are running 3 projects with Fiware accelerators and we have detailed specs of our business processes. I don't want to take your time otherwise I could send you our technical specs.

it is up to you

_In our simplest project/case, your simple scenario is enough. We just send a request to a backend service (provider) and receive the result (that is directly available - the 303 case). The difference is that in our design the consumer is just a front-end client (javascript widget in Fispace's wirecloud) so it cannot implement the receivenotification capability. In this case we might need to just communicate directly with the backend (our provider).

if you widget(frontend) connect directly to backend (producer), you dont need Fispace in the middle.

For this reason, in the case of javascript, client scripts or smartphone application (for example), you need a backend in the frontend app side (at least until websocket will not be exported to client side)

vangelisv commented 9 years ago

Re. you last paragraph: This is the way we were also thinking to deal with it. Are there any plans to use websocket protocol for communication with clients? If so, I guess this will not be available any time soon...

perezdf commented 9 years ago

no a clear plan, but taking into account the trend on Market to have rich client application, it is an important feature in a commercial platform

vangelisv commented 9 years ago

Hi, I'd agree to close the issue, it is not related to a specific problem but in general about the lack of clear documentation and examples on how to use / set up SDI and BCM

SAF2 / documentation

Help with SDI - how to call a capability #45