Template messaging service

EliottPaillard commented 1 month ago

Hello, Following one of our previous discussion I tried to deploy a custom messaging handler. I have used the documentation you gave and I succeeded deploying the basic version of the template messaging service. I expected it to work exactly like the httpmessaging, however it's not the case. The only mentioned input required for the send is messageToSend and I didn't really understand with what this input should be fill exactly. I don't really know java and I couldn't find how to change this. Ideally I would like to provide the http_body, http_headers, http.uri and http.method elements the same way I used to with httpmessaging. Is there a simple way to do this? If not maybe you have some insights about how use the messageToSend element? Thank you :)

costas80 commented 1 month ago

Hi @EliottPaillard, indeed using a custom messaging handler is quite different to using something like the HttpMessaging built-in handler. When using a custom handler you are basically creating a separate app alongside the core Test Bed that will handle the sending and receiving of messages to other systems. The messageToSend input is a demo implementation (described here) to showcase how the messaging service can be used. There is also a sample test case showing how to use it via a TDL test case.

In your case what would be needed is an implementation of the send operation that would call your system under test, extract the payload and HTTP status code, and finally return them to the Test Bed as the step's report. We are in the process of creating a step-by-step guide on how to develop such services (should be published within the week).

I understand however that if you are not a developer, or at least not familiar with Java to follow the examples, this might look quite complex, and overkill compared to what you were doing previously. To better address the simple case of making HTTP calls we have started developing a new version of the HttpMessaging handler that will be very close to what you are using today but that will also be simpler to configure and use. If the custom service seems too big a change I would suggest you continue using the HttpMessaging handler and then consider switching to its new implementation once we have published it (we will do so as part of our nightly build channel). Until then let's keep this issue open and I'll ping you once its ready to use (with also a TDL snippet on how to use based on what you shared previously).

Hope this explains things better and covers you for now.

EliottPaillard commented 1 week ago

Hello there, any news about this?

costas80 commented 1 week ago

Hey @EliottPaillard. We're almost there. We took a bit more time on this as we are going for a complete replacement of the HTTP messaging (as opposed to only supporting the sending of messages as was the initial idea). We need an extra couple of days to make sure everything works as expected and then we'll publish the update on our nightly channel.

costas80 commented 1 week ago

Question on this @EliottPaillard. I recall from your other test case on #48 that you were using the HttpMessaging handler to have the Test Bed make calls to your SUT. Do you have any test cases at present where you need to have the Test Bed receive calls from the SUT? I'm not referring to using the receive step as you have done but, logically whether you need to receive calls from the external system.

The reason I am asking is that the "Test Bed sends to SUT" scenario is pretty much implemented and we could publish it right away for you to try out. The "SUT sends to Test Bed" scenario is more complex and needs a bit more work.

By the way, you will notice with the new implementation that you no longer need to (a) create a messaging transaction, (b) do a send and then (c) do a receive step to get the response. With the new HttpMessagingV2 you do a single send step that returns as part of its report a request and response map with the relevant data. Once we publish this I'll post here the sample test case from #48, adapted to use the new implementation (as documentation will only be updated when the next release is completed).

EliottPaillard commented 1 week ago

Sounds great to me. I would be interested later in ITB receiving calls from SUTs but so far I only need to make calls from ITB

costas80 commented 1 week ago

The update including the new HttpMessagingV2 handler is now published on our nightly build channel. We still need to support receiving SUT calls, and also adapting similarly other built-in messaging handlers (e.g. the SoapMessaging handler) but these will follow and be completed for the next release. As documentation is not updated yet I'm explaining below what you need to change to use this (based on your test from #48).

Please have a look below, let me know if anything is unclear, and feel free to share feedback on this (we can adapt this easily as it is only on our development branch).

Test case changes

Quite a few changes are made comparing to what you previously had:

You no longer need to open, close and refer to messaging transactions.
A single "Test Bed calls SUT" operation is no longer split into send and receive steps. The receive step in general should be used when you actually want to receive a call from a SUT, not to receive the synchronous response from a call the Test Bed has made (receiving a call from a SUT will be supported in the next update).
All config elements are replaced by regular inputs. This means that you can adapt their values on the fly (e.g. to determine dynamically the request URI or the HTTP method).
Receiving the response never blocks and will always succeed (unless these is a communication failure). In addition, you don't need to specify the expected response code to complete a call. Instead you should check the received response to see that the status code (and whatever else is needed) matches your expectations (I added a couple verify steps in the example below).
The URI to call is now set in full as an input. This can be a fixed string, an address you generate on the fly, a value coming from configuration, or a combination (e.g. a "base endpoint path" from the configuration with a suffix path appended to it during the test session). In my example I assume this is defined as a system-level configuration property that would be set as a custom property in your community by the community admin (a property set as required and included in tests), with the specific value set for the system being tested by the community or organisation admin.
The send step report includes two maps named request and response, each with further elements on the relevant data (e.g. HTTP method, headers, body, response status code etc.). If you make one call and view the step's report you will see everything that is returned.

In terms of the TDL, you go from your previous version of this ...

<steps>
    <btxn from="SimulatedActor" to="ContextBroker" txnId="t1" handler="HttpMessaging"/>

    <!-- Create header -->
    <assign to="headers{Content-Type}">"application/ld+json"</assign>

    <!-- Send data to create entity -->
    <send desc="Create entity to query" from="SimulatedActor" to="ContextBroker" txnId="t1">
        <config name="http.method">POST</config>
        <config name="http.uri">ngsi-ld/v1/entities</config>     
        <input name="http_body">$entity_to_create</input>
        <input name="http_headers" source="$headers" />
    </send>

    <receive id="dataReceived" desc="Send acknowledge (201 expected)" from="ContextBroker" to="SimulatedActor" txnId="t1">
        <config name="status.code">201</config>
    </receive>

    <etxn txnId="t1"/>

    <btxn from="SimulatedActor" to="ContextBroker" txnId="t2" handler="HttpMessaging"/>

    <!-- Delete created entity -->
    <send desc="Delete created entity" from="SimulatedActor" to="ContextBroker" txnId="t2">
        <config name="http.method">DELETE</config>
        <config name="http.uri">ngsi-ld/v1/entities/urn:ngsi-ld:example:001</config>
    </send>

    <receive id="dataReceived2" desc="Send acknowledge (204 expected)" from="ContextBroker" to="SimulatedActor" txnId="t2">
        <config name="status.code">204</config>
    </receive>

    <etxn txnId="t2"/>
</steps>

... to this:

<steps stopOnError="true">
    <!-- Create header -->
    <assign to="headers{Content-Type}">"application/ld+json"</assign>

    <!-- Send data to create entity -->
    <send id="create" desc="Create entity to query" from="SimulatedActor" to="ContextBroker" handler="HttpMessagingV2">
        <input name="method">"POST"</input>
                <!-- The || operator is the XPath string concatenation operator. -->
        <input name="uri">$SYSTEM{endpointAddress} || "ngsi-ld/v1/entities"</input>
        <input name="body">$entity_to_create</input>
        <input name="headers">$headers</input>
    </send>

    <log>"Received status code from create call: " || $create{response}{status}</log>
    <verify handler="StringValidator" desc="Check status code">
        <input name="actualstring">$create{response}{status}</input>
        <input name="expectedstring">"201"</input>
    </verify>

    <!-- Delete created entity -->
    <send id="delete" desc="Delete created entity" from="SimulatedActor" to="ContextBroker" handler="HttpMessagingV2">
        <input name="method">"DELETE"</input>
        <input name="uri">$SYSTEM{endpointAddress} || "ngsi-ld/v1/entities/urn:ngsi-ld:example:001"</input>
    </send>

    <log>"Received status code from delete call: " || $delete{response}{status}</log>
    <verify handler="StringValidator" desc="Check status code">
        <input name="actualstring">$delete{response}{status}</input>
        <input name="expectedstring">"204"</input>
    </verify>
</steps>

Test suite changes

Built-in messaging handlers needed previously to have a non-obvious configuration for the test case actors (especially if sending and receiving calls). This is now simplified as all configuration (the address and complete URI) can be defined as you like (see my earlier comment on this). This means that your test suite definition goes from ...

<?xml version="1.0" encoding="UTF-8"?>
<testsuite id="testSuite_ContextBroker" xmlns="http://www.gitb.com/tdl/v1/" xmlns:gitb="http://www.gitb.com/core/v1/">
    <metadata>
        <gitb:name>testSuite_ContextBroker</gitb:name>
        <gitb:description>example</gitb:description>
        <gitb:version>1.0</gitb:version>
    </metadata>
    <actors>
        <gitb:actor id="SimulatedActor">
            <gitb:name>SimulatedActor</gitb:name>
            <gitb:desc>The simulated Actor system used for testing purposes. Alternately has Producer and Consumer Role</gitb:desc>
        </gitb:actor>
        <gitb:actor id="ContextBroker">
            <gitb:name>ContextBroker</gitb:name>
            <gitb:desc>A context broker system that should be compliant to the specifications of the NGSILD API</gitb:desc>
            <gitb:endpoint name="address" desc="The address to contact the sut system">
            <!-- port = ip de connexion personnelle et host = 1026 -->
                <gitb:config name="network.host" desc="The host where the system is listening (personal IP)" kind="SIMPLE" use="R"/>
                <gitb:config name="network.port" desc="The port where the system is listening (context broker port)" kind="SIMPLE" use="R"/>
            </gitb:endpoint>
        </gitb:actor>
    </actors>
    <testcase id="example"/>
</testsuite>

... to:

<?xml version="1.0" encoding="UTF-8"?>
<testsuite id="testSuite_ContextBroker" xmlns="http://www.gitb.com/tdl/v1/" xmlns:gitb="http://www.gitb.com/core/v1/">
    <metadata>
        <gitb:name>testSuite_ContextBroker</gitb:name>
        <gitb:description>example</gitb:description>
        <gitb:version>1.0</gitb:version>
    </metadata>
    <actors>
        <gitb:actor id="SimulatedActor">
            <gitb:name>SimulatedActor</gitb:name>
            <gitb:desc>The simulated Actor system used for testing purposes. Alternately has Producer and Consumer Role</gitb:desc>
        </gitb:actor>
        <gitb:actor id="ContextBroker">
            <gitb:name>ContextBroker</gitb:name>
            <gitb:desc>A context broker system that should be compliant to the specifications of the NGSILD API</gitb:desc>
        </gitb:actor>
    </actors>
    <testcase id="example"/>
</testsuite>

Remember that this change means that you no longer provide configuration (the address/URI) at the level of the conformance statement, but rather you can use what works best for you. The typical case as I mentioned in my earlier comments and example would be a required system configuration property that is set as included in tests (at least as the base endpoint address). You could then also adapt this as needed by your API during test session execution (e.g. add path parameters dynamically based on your test session state).

HttpMessagingV2 supported inputs (send step)

As mentioned before, the new messaging handler only supports the send step currently (making a call from the Test Bed to the SUT - we'll be adding shortly the receive support). The supported inputs in this case are as follows:

Input	Required	Description	Default value
uri	Yes	The full URI to call (string).	-
method	No	The HTTP method (string).	`GET`
body	No	The body to use (any type, typically string or binary).	-
parameters	No	A map of key-value pairs (string values) of request parameters to include.	-
queryParameters	No	A map of key-value pairs of request parameters to include explicitly in the query string (for e.g. a POST with a body).	-
headers	No	A map of key-value pairs (string or list[string] values) of headers to set.	-
followRedirects	No	Whether any redirects should be followed (boolean).	`true`
parts	No	Parts to set as a multipart form submission (list of maps, with each map including the "name", "content", "fileName" and "contentType" - "fileName" and "contentType" needed for file parts).	-

For example, if you want to make a HTTP form-encoded POST submission you would do something like this:

<assign to="params{key1}">"val1"</assign>
<assign to="params{key2}">"val2"</assign>
<assign to="params{key3}" append="true">"val3a"</assign>
<assign to="params{key3}" append="true">"val3b"</assign>
<send desc="Send message to SUT" from="Actor1" to="Actor2" handler="HttpMessagingV2">
   <input name="uri">"http://localhost:8080/sut/endpoint"</input>
   <input name="method">"POST"</input>
   <input name="parameters">$params</input>
</send

Finally, note that certain inputs are determined based on other inputs. For example, if you omit the method and provide a body, it is assumed you are making a POST. Also, if you specify parameters and also a body, instead of a form-encoded POST submission you will have the body as the payload with the parameters set on the querystring.

EliottPaillard commented 1 week ago

Hello @costas80, thanks a lot. This new handler looks very good, however I didn't manage to use it so far.

I tried to build the simplest test suite I could think about thanks to the inputs you gave but ITB shows an error.

I was using this testSuite_020.zip with the version of (2024-05-24 08:00:57)

costas80 commented 1 week ago

Hi @EliottPaillard. Yes indeed! There was an issue with the implementation that has now been fixed. If you pull the latest nightly image it should work fine (I also tried with the test suite you uploaded).

EliottPaillard commented 5 days ago

Hello @costas80, after pulling the last nightly (2024-05-24 16:13:40), the first step of the example test case just never finish

costas80 commented 5 days ago

Hi @EliottPaillard - that's odd as I tried the fix with your example. Let me check again.

costas80 commented 5 days ago

Ok, so the problem was that with the updated implementation any connection errors were getting swallowed and the step was never completed (if the SUT could be contacted the step would complete as expected). The corrected error handling is now published in the latest nightly build.

@EliottPaillard, can you please pull the latest update and try again?

EliottPaillard commented 5 days ago

Ok, the first step fails now

I can confirm everything works fine when using curl to send the exact same request

EliottPaillard commented 5 days ago

Also, the previous HttpMessaging is not working anymore testSuite_020.zip

costas80 commented 5 days ago

If you get a connection failure and ClosedChannel (as you see in the logs) this means that the SUT is not accessible from the container. This could be because of Docker networking in case you have everything running on your localhost. From your example I see that you are sending to http://localhost:9090/ngsi-ld/v1/temporal/entities. If you are on a Windows workstation you would need to provide this as http://host.docker.internal:9090/ngsi-ld/v1/temporal/entities (host.docker.internal being the Windows Docker alias for localhost).

costas80 commented 5 days ago

Also, the previous HttpMessaging is not working anymore testSuite_020.zip

No changes were made to the HttpMessaging handler. I believe the issue here is that you removed the actor configuration for the network host and port (I see its missing in your test suite definition).

In any case I would suggest you use host.docker.internal with the new handler (HttpMessagingV2) to resolve this - although this is Docker-specific we'll also make it clear on the docs when the next release is published. By the way, if you are on a linux workstation localhost should work, but if it doesn't you can use Docker's 172.17.0.1 alias.

Note that the address of the target service should be defined as a configuration parameter. If you want to set this as a global config parameter, you can set it as a domain parameter (used then as $DOMAIN{address}), otherwise I think a custom system property would be best (used as $SYSTEM{address}).

Finally, in case you're wondering why using localhost or 127.0.0.1 works for the HttpMessaging handler, this is because it creates a socket connection directly between the internal test engine and the SUT. This seems to simplify things when you're accessing something on the localhost but for any other case its bad as it means you must make the internal test engine directly accessible to SUTs (as opposed to going via a reverse proxy for example). This is one of the reasons why we have deprecated these older messaging handlers.

EliottPaillard commented 4 days ago

Indeed, I had to change what you said.

It's still not working though. Now I have this error

costas80 commented 4 days ago

I'll try to replicate again and see if I get the same issue. In the meantime, could you try to get a bash shell in the gitb-srv container (could be named also itb-srv) and see if you can do a curl or wget from within it to the SUT address you are targeting? Whenever I have networking issues with Docker containers I try this as a first step to see if the address is reachable. If a curl/wget from within the container works then its an issue with the Test Bed software - otherwise its a networking issue.

To get a bash shell you would do e.g. docker exec -it gitb-srv bash.

EliottPaillard commented 4 days ago

I could have some issues with my SUT too. With another SUT I got another error

I'm going to do what you say

costas80 commented 4 days ago

I tried this on a fresh Test Bed instance built from the nightly images and everything seems to be working fine. I took your test suite and made some small adaptations, but the test step of interest is as follows:

<!-- Create header -->
<assign to="headers{Content-Type}">"application/ld+json"</assign>
<!-- Send data to create entity -->
<send id="create" desc="Create entity to query" from="SimulatedActor" to="ContextBroker" handler="HttpMessagingV2">
   <input name="method">"POST"</input>
   <input name="uri">$DOMAIN{sutAddress} || "/app/post"</input>
   <input name="body">"Test"</input>
   <input name="headers">$headers</input>
</send>

In the relevant domain I created a domain parameter as follows:

Note that I have a dummy server implementation listening on my localhost at http://localhost:8484/app/post. I basically set the base URI in the parameter and replaced localhost by host.docker.internal to account for Docker networking and my Windows PC.

With this in place, launching the test session I get a response as expected:

From this I assume you must have some issue with your SUT. I would suggest you do a wget/curl from your gitb-srv (or itb-srv depending on the naming) container to make sure it can be reached.

EliottPaillard commented 4 days ago

I managed to make it work ! The problem was about a faulty SUT + a bad way to call it. Thanks a lot for your support, I appreciate it. I will let you know if I have anything relevant to tell you about this handlerV2 :)

costas80 commented 4 days ago

Great to know @EliottPaillard! We're still working on the receive step for this new handler but as you don't need this immediately I'll close the current issue.

By the way, make sure you subscribe to this repo to be notified for releases so that you get the update when the new release is published (that will include all discussed changes).

ISAITB / gitb