`PATCH /thirdpartyRequests/transactions/{id}` Callback Design v2

[lewisdaly]

One of the challenges we face with the Thirdparty API is getting the notification back to a PISP to inform them of a transaction's outcome.

Our previous design relied on the switch to maintain state across a series of messages, which was a deal-breaker.

In this iteration, we will look at:

1. What information needs to be added to the API messages in order to get a callback to the PISP
2. How does the central-ledger's fulfilment notification fit into this picture?

1. API Changes:

POST /transfers

When the PayerDFSP issues this request, it includes the following fields:

Fields:

• transactionId - the id of the transaction
• transactionRequestId - the transactionRequestId this transfer is fulfilling. This id was originally decided by the PISP
• transactionParticipants - an array of participantIds and their role in the transaction.

POST /transfers

FSPIOP-Source: dfspa
FSPIOP-Destination: dfspb
{
  "transferId": "321",
  "payerFsp": "dfspa",
  "payeeFsp": "dfspb",
  "amount": {
    "amount": "100",
    "currency": "USD"
  },
  "expiration": "2020-06-15T13:00:00.000",
//  "ilpPacket": "...",
  "condition": "...",
  "transaction": {

  },
  //"transactionId": "987",
  "transactionRequestId": "123",
//determined by the Payee during the quote cycle
  "transactionParticipants": [
    { 
      "id": "dfspa",
      "role": "CREDITOR",
    },
    {
      "id": "pispa",
      "role": "INITIATOR"
    }
  ],
}

PUT /transfers/{id}

Q: Do we need to have the transactionParticipants in this request? Or can we assume that the central-ledger stores some state related to the original POST /transfers request, including the participants? Adrian: NO - the central-ledger will keep the transaction object

PUT /transfers/321

FSPIOP-Source: dfspb
FSPIOP-Destination: dfspa
{
  "fulfilment": "...",
  "completedTimestamp": "2020-06-15T12:01:00.000",
  "transferState": "COMMITTED"
}

2. Internal Processing

1. Upon transaction fulfilment at the central-ledger, the central-ledger emits a Notification event:

{
  "from": "dfspb",
  "to": "dfspa",
  "id": "bc1a9c36-4429-4205-8553-11f92de1919e",
  "content": {
    "uriParams": {
      "id": "bc1a9c36-4429-4205-8553-11f92de1919e"
    },
    "headers": {
      "content-type": "application/vnd.interoperability.transfers+json;version=1.0",
      "date": "2020-09-09T03:58:36.000Z",
      "fspiop-source": "dfspb",
      "fspiop-destination": "dfspa",
      "authorization": "Bearer 74b241a2-4200-3938-8dfc-0e26ba21dc22",
      "content-length": "136",
      "host": "ml-api-adapter:3000",
      "connection": "close"
    },
    "payload": "data:application/vnd.interoperability.transfers+json;version=1.0;base64,eyJjb21wbGV0ZWRUaW1lc3RhbXAiOiIyMDIwLTA5LTA5VDAzOjU4OjM2Ljg0NFoiLCJ0cmFuc2ZlclN0YXRlIjoiQ09NTUlUVEVEIiwiZnVsZmlsbWVudCI6Ii1TODBPZ0pMbEpSVElHUFAxMlpZTnFjZEhLQlQ3WHNVZDFjenVOMUI5RzQifQ"
  },
  "type": "application/json",
  "metadata": {
    "correlationId": "bc1a9c36-4429-4205-8553-11f92de1919e",
    "event": {
      "type": "notification",
      "action": "commit",
      "createdAt": "2020-09-09T03:58:36.859Z",
      "state": {
        "status": "success",
        "code": 0,
        "description": "action successful"
      },
      "id": "85756a1d-c159-4316-b8d0-d0a41ecbcfe1",
      "responseTo": "0ecc24f8-a617-4b60-b954-3cfb4b909ae8"
    },
    "trace": {
      "startTimestamp": "2020-09-09T03:58:36.932Z",
      "service": "cl_transfer_position",
      "traceId": "f67d1328e258e8fba2dacbbed098be48",
      "spanId": "0b142ddb9e695312",
      "parentSpanId": "e933005ac5ade0c6",
      "tags": {
        "tracestate": "acmevendor=eyJzcGFuSWQiOiIwYjE0MmRkYjllNjk1MzEyIiwidGltZUFwaUZ1bGZpbCI6IjE1OTk2MjM5MTY4NTcifQ==",
        "transactionType": "transfer",
        "transactionAction": "fulfil",
        "transactionId": "bc1a9c36-4429-4205-8553-11f92de1919e",
        "source": "dfspb",
        "destination": "dfspa"
      },
      "tracestates": {
        "acmevendor": {
          "spanId": "0b142ddb9e695312",
          "timeApiFulfil": "1599623916857"
        }
      }
    },
    "protocol.createdAt": 1599623916963
  }
}

Q: Should the central-ledger emit 2 events? One for the PayerDFSP and PISP? Or just 1 event that the ml-api-adapter and thirdparty-api-adapter can interpret?

• According to the 1.1 changes, there is just 1 message: https://docs.mojaloop.io/documentation/mojaloop-technical-overview/central-ledger/transfers/1.1.4.a-send-notification-to-participant-v1.1.html
• Message follows the following format: (ref)

{
  id: <transferMessage.transferId>,
  from: <transferMessage.payerFsp>,
  to: <transferMessage.payeeFsp>,
  type: application/json,
  content: {
    headers: <transferHeaders>,
    payload: <transferMessage>
  },
  metadata: {
    event: {
      id: <uuid>,
      responseTo: <previous.uuid>,
      type: transfer,
      action: commit || reserve,
      createdAt: <timestamp>,
      state: {
        status: "success",
        code: 0
      }
    }
  }
}

Perhaps this message can be changed to something along the following lines:

{
  id: <transferMessage.transferId>,
  participants: [
    { 
      id: "dfspa",
      role: "CREDITOR",
    },
    { 
      id: "dfspb",
      role: "DEBTOR",
    },
    {
      id: "pispa",
      role: "INITIATOR"
    }
  ],
  type: application/json,
  content: {
    headers: <transferHeaders>,
    payload: <transferMessage>
  },
  metadata: {
    event: {
      id: <uuid>,
      responseTo: <previous.uuid>,
      type: transfer,
      action: commit || reserve,
      createdAt: <timestamp>,
      state: {
        status: "success",
        code: 0
      },
      // add these fields so the thirdparty-api-adapter can correlate this message to a transaction
      transactionId: "987",
      transactionRequestId: "123",
    }
  }
}

Note: This somewhat alters the way we use the FSPIOP-Headers, which may not be desireable

2. The thirdparty-api-adapter sees this notification,

Known Issues with this design

1. The API messages are not backwards compatible for the PayeeFSP

2. We still need to keep some state in the central-ledger's database related to:

   • transactionId
   • transactionRequestId
   • transactionParticipants

3. There is somewhat redundant information between the FSPIOP-Headers (which are for routing messages), and the proposed transactionParticipants collection in the POST/PUT request bodies.

   • The original proposal was to use an FSPIOP-Initiator header, perhaps that is a simpler approach

4. When the thirdparty-api-adapter recieves the fulfilment notification, it must convert this to a PATCH /thirdpartyRequests/transactions/{id}

   • this requires the transactionRequestId (and maybe the transactionId) in the notification message

[mjbrichards]

So, looking at the POST: the Payer DFSP knows whether this is a PISP-initiated transfer or not, so it can fill in its own ID in any case and the PISP's if necessary. It doesn't have to do anything about the Payee DFSP, since it doesn't know whether the Payee DFSP wants an acknowledgement or not.

When the Payee DFSP responds with the PUT, it can decide whether it wants to add itself to the list of participants or not. Now the transactionParticipants object contains all the information the switch needs to know who to notify: the switch doesn't need to persist any information itself...

I think I'd like to replace the two distinct calls in the sequence diagram with a single loop that covers all the responses requested. The content of the message will be the same in each case, but the headers will be different and the HTTP command will vary too - PUT for the payer, PATCH for the payee, ? for the PISP. We could make that implicit (switch works out what to put from the role,) or we could add it as a field to the transactionParticipant entries. We could also add that as processing conditional on the presence of a transactionParticipant list to maintain backwards compatibility, so payee DFSPs could use either mode to request confirmations...

[lewisdaly]

To recap last weeks discussion:

1. We plan to use add a list of subscribers to the transaction object
2. During the Quoting phase, both the Payer and Payee will have an opportunity to add themselves (and any other 3rd party they wish) to the subscribers list
3. The transaction object will be also in plaintext in the request bodies of the PUT /quotes/{id} and POST /transfers
4. During the reserve step of the transfer, the central-ledger will store the transaction object in it's database (it needs to do this already in some form to verify the transfer hasn't been tampered with
5. When the central-ledger commits the transaction, it looks up the transaction object, and includes it in the Fulfil Notification it emits

@MichaelJBRichards @jgeewax does this sound about right to you?

[millerabel]

Since all parties see the transaction object when it is presented to the central-ledger as “prepared,” why does the payer need it returned during fulfill?

The central-ledger provides a service guarantee that the transaction object has not been tampered with, if the fulfillment covers the transaction object content. The transaction object does not need to be sent with notifications to anyone. The central ledger should not traffic in transaction details, but confine itself to the payment condition and fulfillment. The transaction object digest ensures integrity and should be included in the fulfillment calculation.

If a third party needs to see the quotation response details, it should receive them prior to submitting the prepared payment, to ensure the end-user can see the consequences of quotation (filtered through the third-party app). Once the payment is prepared with a payment condition, nothing can change and so duplicating and sending the transaction object to external parties is a huge burden on the central-ledger.

Is there a data flow detail here I’m missing?

— Miller

On Oct 4, 2020, at 11:12 PM, Lewis Daly notifications@github.com wrote:

To recap last weeks discussion:

We plan to use add a list of subscribers to the transaction object During the Quoting phase, both the Payer and Payee will have an opportunity to add themselves (and any other 3rd party they wish) to the subscribers list The transaction object will be also in plaintext in the request bodies of the PUT /quotes/{id} and POST /transfers During the reserve step of the transfer, the central-ledger will store the transaction object in it's database (it needs to do this already in some form to verify the transfer hasn't been tampered with When the central-ledger commits the transaction, it looks up the transaction object, and includes it in the Fulfil Notification it emits @MichaelJBRichards https://github.com/MichaelJBRichards @jgeewax https://github.com/jgeewax does this sound about right to you?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/mojaloop/pisp/issues/71#issuecomment-703419422, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB6OJ6B55OGJ3TL5QBDIJBLSJFPTJANCNFSM4RWSXQIQ.

[lewisdaly]

Thanks for your comments @millerabel

Since all parties see the transaction object when it is presented to the central-ledger as “prepared,” why does the payer need it returned during fulfill?

I think I wasn't clear that I simply mean to include the Subscriptions object in the internal messaging that occurs between the central-ledger and ml-api-adapter and not a part of the external messaging between the switch and participants (i.e. the API Definition)

Indeed the transaction object does not need to be send along with notifications to everyone, but we do need a way to deliver those notifications. There are 2 options I forsee:

1. The central-ledger emits 1 fulfillment notification message, and it's up to the ml-api-adapter to interpret this message and send one or more HTTP calls (be they PUT /transfers/{id} or PATCH /transfers/{id} or something else)
2. The central-ledger emits a fulfilment notification for each participant specified in the Subscription object. So if there are 3 participants in the Subsriptions list, then it would emit 3 separate notifications.

1 is the approach we have today as of v1.1 of the FSPIOP-API. The central-ledger emits a single fulfilment notification. If the message has event.action == 'reserve', then the ml-api-adapter will generate 2 HTTP calls, one to the Payer, and another to the Payee (this sequence diagram has more information)

In the future, we have other api-adapters for other APIs (e.g. the Thirdparty API), my proposal was to extend the existing functionality by including the subscriptions list in the fulfilment notification, so that the ml-api-adapter (or any other api adapter) can simply listen in for fulfilment notifications, and generate the appropriate HTTP calls for the relevant participants.

[lewisdaly]

@adrianhopebailie I've got a PR for your PR here: https://github.com/adrianhopebailie/mojaloop-specification/pull/1