Changes in Notify protocol after DSpace implementation

[steph-ieffam]

Hello, I recently noticed that the protocol has changed in the Rejection/Acceptance patterns. The current pattern implementation is but the old ones were using this longer format:

  "@context": [
    "https://www.w3.org/ns/activitystreams",
    "https://purl.org/coar/notify"
  ],
  "actor": {
    "id": "https://review-service.com",
    "name": "Review Service",
    "type": "Service"
  },
  "context": {
    "id": "https://dspace-coar.4science.cloud/items/24d50450-9ff0-485f-82d4-fba1be42f3f9",
    "ietf:cite-as": "https://doi.org/10.5555/12345680",
    "type": "sorg:AboutPage",
    "url": {
      "id": "https://research-organisation.org/repository/preprint/201203/421/content.pdf",
      "mediaType": "application/pdf",
      "type": [
        "Article",
        "sorg:ScholarlyArticle"
      ]
    }
  },
  "id": "urn:uuid:5fb3af44-d4f8-4226-9475-2d09c2d8d9e0",
  "inReplyTo": "urn:uuid:f7289ad5-0955-4c86-834c-fb54a736778b",
  "object": {
    "id": "urn:uuid:0370c0fb-bb78-4a9b-87f5-bed307a509dd",
    "object": "https://research-organisation.org/repository/preprint/201203/421/",
    "type": [
      "Offer",
      "coar-notify:ReviewAction"
    ]
  },
  "origin": {
    "id": "https://httpdump.app/dumps/29fc0d4c-aa11-4107-9f1f-e733d569f274",
    "inbox": "https://httpdump.app/dumps/29fc0d4c-aa11-4107-9f1f-e733d569f274",
    "type": "Service"
  },
  "target": {
    "id": "https://research-organisation.org/repository",
    "inbox": "https://research-organisation.org/inbox/",
    "type": "Service"
  },
  "type": [
    "TentativeReject",
    "coar-notify:ReviewAction"
  ]
}

I have noticed some sections were removed and the type has also changed. You have moved from String array to String. As far as improvements are fine and useful keep in mind that we have developed some feature using the previous "version" and this new feature has also been implemented in DSpace 8. Are you sure you want to change the pattern at this point? without even considering creating a version for each "protocol" I think there will be a lot of issues while communicating between systems.

My suggestion is to restore the previous version and create a version 2.0 (or different version naming) with the current changes available on the COAR Notify website. The old patterns were shared with the DSpace community, a Postman collection is also available on the DSpace 8 documentation and uses the previous version.

[paulwalk]

Hi @steph-ieffam

I think there are two (related) issues:

1. There seems to have been some confusion with the implementation of this pattern - with discrepancies between implementations in HAL, PCI and DSpace. I am trying to clarify the situation and will arrange a call to discuss this.
2. Versioning. There has been a lot of discussion about how to implement versioning. This is difficult dues to the "loose" nature of the RDF-based protocol. I am currently discussing how we might use JSON schema as a way to make this more precise.

[paulwalk]

The decisions about how to address this issue are documented in this discussion item

[paulwalk]

Hi @steph-ieffam,

we have made a lot of progress with implementing the changes that were summarised here

You can see these changes in a temporary development/staging site for the protocol and workflow documentation here:

https://notify.antleaf.com

You will see that the approach to documenting workflows has changed considerably (using flow-charts instead of the "swim lanes"), e.g. https://notify.antleaf.com/catalogue/workflows/repository-pci/

I want to particularly focus now on ensuring that the site has documentation which accords with what has been implemented in DSpace.

I am trying to document a "legacy" version of the protocol as 0.9.0: https://notify.antleaf.com/specification/0.9.0/

I have a question for you: What has actually been implemented in DSpace 8?

I think it is one of these two:

1. actual support for some version of the PCI workflow, or
2. just support for the patterns which are used in the workflow?

If (2) is the current state of DSpace 8 support, then (perhaps with a little advice from you) I can address this by accurately documenting the patterns under the 0.9.0 version.

If (1) is the current state of DSpace 8 support, then I will need to document both the patterns (under 0.9.0) but also a different version of the PCI workflow. This may be problematic, because PCI may not support that workflow, but we can discuss how that might be possible.

Please can you advise? I am happy to discuss in a call if that is easier.

[steph-ieffam]

Hey @paulwalk thank a lot, I really appreciate. Yes, I agree that the flow-charts are much clearer and easier to understand.

In reply to your question about what was implemented in DSpace 8 I can tell you that DSpace 8 is really flexible in hadling LDN messages and we didn't restrict the support to a single (or more) workflow. So, since we have a full flexible support to this I can say that the correct option is (2). I am also sharing this technical documentation with you where you might find helpful info. I am also attaching a Postman collection used for testing purposes. The Postman collection contains exactly the patterns that we support Coar Notify.postman_collection.json

Let me know if you still have any doubt

[paulwalk]

Thanks @steph-ieffam !

This is reassuring :-)

I have had a look at the technical documentation. I think this needs some revision. To be honest, I don't think this is because of any changes to the protocol - I think it's just a little unclear/inaccurate. For example, it implies that context is more important than object which I don't think is correct, and also that actor "doesn't include important information" which I think is also not necessarily correct (for example in the PCI Endorsement workflow the actor is very important).

However, these are just description issues and are easily changed. I'm happy to help/advise on this.

Regarding the Postman file - is there a way to get a more easily readable view of the "raw" payloads? It's quite difficult to read, for example, this:

"body": {
                            "mode": "raw",
                            "raw": "{\n  \"@context\": [\n    \"https://www.w3.org/ns/activitystreams\",\n    \"https://purl.org/coar/notify\"\n  ],\n  \"actor\": {\n    \"id\": \"https://review-service.com\",\n    \"name\": \"Review Service\",\n    \"type\": \"Service\"\n  },\n  \"context\": {\n    \"id\": \"http://localhost:4000/handle/123456789/3\",\n    \"ietf:cite-as\": \"https://doi.org/10.5555/12345680\",\n    \"type\": \"sorg:AboutPage\",\n    \"url\": {\n      \"id\": \"https://research-organisation.org/repository/preprint/201203/421/content.pdf\",\n      \"mediaType\": \"application/pdf\",\n      \"type\": [\n        \"Article\",\n        \"sorg:ScholarlyArticle\"\n      ]\n    }\n  },\n  \"id\": \"urn:uuid:2f4ec583-109e-4952-a94a-b7d7625b7c63\",\n  //\"inReplyTo\": \"urn:uuid:0370c0fb-bb78-4a9b-87f5-bed307a509fd\",//another existing ldn_message->ldn_id \n  \"object\": {\n    \"id\": \"https://www.example.com/\",\n    \"ietf:cite-as\": \"https://doi.org/10.3214/987654\",\n    \"type\": [\n      \"Document\",\n      \"sorg:Review\"\n    ]\n  },\n  \"origin\": {\n    \"id\": \"https://www.example.com\",\n    \"inbox\": \"https://www.example.com\",//notifyservice->ldn_url\n    \"type\": \"Service\"\n  },\n  \"target\": {\n    \"id\": \"https://www.local.com\",\n    \"inbox\": \"https://www.local.com\",//notifyservice->ldn_url\n    \"type\": \"Service\"\n  },\n  \"type\": [\n    \"Announce\",\n    \"coar-notify:ReviewAction\"\n  ]\n}",
                            "options": {
                                "raw": {
                                    "language": "json"
                                }
                            }
                        },

[steph-ieffam]

The documentation I have shared to you is a more technical section of the Notify feature which is not really meant to explain people how the Notify protocol works or what's more important in the LDN Message. It was made just to give a quick glimpse to people that are about to test the Notify system. If you also take a look at the bottom of the page there's another postman collection for general tests. A more high-level documentation is available on other sections and explains also the usage of the Notify dedicated sections on Dspace. But if you think there's any way we could improve it please feel free to suggest any change that looks needed. It's really important to be as clear as possible as you're pointing out.

About the postman collection, I think the easiest way is to import it in the Postman application. We have more than 10 different requests so I don't want to make things messy in the comment section but I think I can share the most important with you in a new comment

[steph-ieffam]

Announce Review

  "@context": [
    "https://www.w3.org/ns/activitystreams",
    "https://purl.org/coar/notify"
  ],
  "actor": {
    "id": "https://review-service.com",
    "name": "Review Service",
    "type": "Service"
  },
  "context": {
    "id": "http://localhost:4000/handle/123456789/3",
    "ietf:cite-as": "https://doi.org/10.5555/12345680",
    "type": "sorg:AboutPage",
    "url": {
      "id": "https://research-organisation.org/repository/preprint/201203/421/content.pdf",
      "mediaType": "application/pdf",
      "type": [
        "Article",
        "sorg:ScholarlyArticle"
      ]
    }
  },
  "id": "urn:uuid:2f4ec583-109e-4952-a94a-b7d7625b7c63",
  //"inReplyTo": "urn:uuid:0370c0fb-bb78-4a9b-87f5-bed307a509fd",//another existing ldn_message->ldn_id 
  "object": {
    "id": "https://www.example.com/",
    "ietf:cite-as": "https://doi.org/10.3214/987654",
    "type": [
      "Document",
      "sorg:Review"
    ]
  },
  "origin": {
    "id": "https://www.example.com",
    "inbox": "https://www.example.com",//notifyservice->ldn_url
    "type": "Service"
  },
  "target": {
    "id": "https://www.local.com",
    "inbox": "https://www.local.com",//notifyservice->ldn_url
    "type": "Service"
  },
  "type": [
    "Announce",
    "coar-notify:ReviewAction"
  ]
}

Announce Endorsement

{
    "@context": [
       "https://www.w3.org/ns/activitystreams",
       "https://purl.org/coar/notify"
     ],
     "actor": {
       "id": "https://overlay-journal.com",
       "name": "Overlay Journal",
       "type": ["Service"]
     }, 
     "context": {
        "id": "https://dspace-coar.4science.cloud/items/24d50450-9ff0-485f-82d4-fba1be42f3f9",
       "ietf:cite-as": "https://doi.org/10.5555/12345680",
       "type": ["sorg:AboutPage"],
       "url": {
         "id": "https://research-organisation.org/repository/preprint/201203/421/content.pdf",
         "mediaType": "application/pdf",
         "type": [
           "Article",
           "sorg:ScholarlyArticle"
         ]
       }
     },
    "id": "urn:uuid:2f4ec582-109e-4952-a94a-b7d7615f9c72",
    "inReplyTo": "urn:uuid:570eb7ad-68e0-4870-8f28-2a2c8a4248a6",//another existing ldn_message->ldn_id 
    "object": {
        "id": "https://www.example.com/",
       "ietf:cite-as": "https://overlay-journal.com/articles/00001/",
       "type": [
         "Page",
         "sorg:WebPage"
       ]
     },
    "origin": {
    "id": "https://notify-inbox.info/",
    "inbox": "https://notify-inbox.info/inbox/",//ldn_url
    "type": "Service"
    },
    "target": {
    "id": "https://generic-service.com/system",
    "inbox": "ldn_url",//notifyservice->ldn_url
    "type": "Service"
    },
     "type": [
       "Announce",
       "coar-notify:EndorsementAction"
     ]
 }

Announce Release

{
  "@context": [
    "https://www.w3.org/ns/activitystreams",
    "https://purl.org/coar/notify"
  ],
  "actor": {
    "id": "https://dspace-coar.4science.cloud/",
    "name": "DSpace at My University",
    "type": "Service"
  },
  "context": {
    "id": "https://dspace-coar.4science.cloud/entities/publication/f5846486-7b49-43d2-ab9f-7d59ae7e6c02",
    "ietf:cite-as": "https://dspace-coar.4science.cloud/entities/publication/f5846486-7b49-43d2-ab9f-7d59ae7e6c02",
    "ietf:item": {
      "id": "http://localhost:4000/bitstreams/73b1ecbd-94d1-4438-a6fc-fb00f550bcac/download",
      "mediaType": "",
      "type": [
        "Article",
        "sorg:ScholarlyArticle"
      ]
    },
    "type": "sorg:AboutPage"
  },
  "id": "urn:uuid:43092174-d9b1-4768-9192-fab1b3a68d69",
  "object": {
    "as:object": "http://localhost:4000/handle/123456789/14",
    "as:relationship": "http://purl.org/vocab/frbr/core#supplement",
    "as:subject": "https://dspace-coar.4science.cloud/entities/publication/f5846486-7b49-43d2-ab9f-7d59ae7e6c02",
    "id": "urn:uuid:b4258563-fab4-466a-8549-3497f851e86b",
    "type": "Relationship"
  },
  "target": {
    "id": "http://localhost:4000",
    "inbox": "http://localhost:8080/server/ldn/inbox",
    "type": "Service"
  },
  "origin": {
    "id": "https://dspace-coar.4science.cloud/home",
    "inbox": "https://dspace-coar.4science.cloud/server/ldn/inbox",
    "type": "Service"
  },
  "type": [
    "Announce",
    "coar-notify:RelationshipAction"
  ]
}

Request Review

{
  "@context": [
    "https://www.w3.org/ns/activitystreams",
    "https://purl.org/coar/notify"
  ],
  "actor": {
    "id": "https://orcid.org/0000-0002-1825-0097",
    "name": "Josiah Carberry",
    "type": "Person"
  },
  "id": "urn:uuid:0370c0fb-bb78-4a9b-87f5-bed307a509df",
  "object": {
    "id": "http://localhost:4000/handle/123456789/1119",
    "ietf:cite-as": "https://doi.org/10.5555/12345680",
    "type": "sorg:AboutPage",
    "url": {
      "id": "https://research-organisation.org/repository/preprint/201203/421/content.pdf",
      "mediaType": "application/pdf",
      "type": [
        "Article",
        "sorg:ScholarlyArticle"
      ]
    }
  },
  "origin": {
    "id": "https://research-organisation.org/repository",
    "inbox": "2f4ec582-109e-4952-a94a-b7d7615a8c69",//notifyservice->ldn_url
    "type": "Service"
  },
  "target": {
    "id": "https://review-service.com/system",
    "inbox": "https://review-service.com/inbox/",
    "type": "Service"
  },
  "type": [
    "Offer",
    "coar-notify:ReviewAction"
  ]
}

Request Endorsement

{
  "@context": [
    "https://www.w3.org/ns/activitystreams",
    "https://purl.org/coar/notify"
  ],
  "actor": {
    "id": "https://orcid.org/0000-0002-1825-0097",
    "name": "Josiah Carberry",
    "type": "Person"
  },
  "id": "urn:uuid:c8ea494b-3fe4-4494-a06f-4aa423cbb66e",
  "object": {
    "id": "http://localhost:4000/handle/123456789/1119",
    "ietf:cite-as": "https://doi.org/10.5555/12345680",
    "type": "sorg:AboutPage",
    "url": {
      "id": "https://research-organisation.org/repository/preprint/201203/421/content.pdf",
      "mediaType": "application/pdf",
      "type": [
        "Article",
        "sorg:ScholarlyArticle"
      ]
    }
  },
  "origin": {
    "id": "https://research-organisation.org/repository",
    "inbox": "2f4ec582-109e-4952-a94a-b7d7615a8c69",//notifyservice->ldn_url
    "type": "Service"
  },
  "target": {
    "id": "https://review-service.com/system",
    "inbox": "https://review-service.com/inbox/",
    "type": "Service"
  },
  "type": [
    "Offer",
    "coar-notify:EndorsementAction"
  ]
}

ACK Reject Review

{
  "@context": [
    "https://www.w3.org/ns/activitystreams",
    "https://purl.org/coar/notify"
  ],
  "actor": {
    "id": "https://review-service.com",
    "name": "Review Service",
    "type": "Service"
  },
  "context": {
    "id": "https://dspace-coar.4science.cloud/items/24d50450-9ff0-485f-82d4-fba1be42f3f9",
    "ietf:cite-as": "https://doi.org/10.5555/12345680",
    "type": "sorg:AboutPage",
    "url": {
      "id": "https://research-organisation.org/repository/preprint/201203/421/content.pdf",
      "mediaType": "application/pdf",
      "type": [
        "Article",
        "sorg:ScholarlyArticle"
      ]
    }
  },
  "id": "urn:uuid:5fb3af44-d4f8-4226-9475-2d09c2d8d9e0",
  "inReplyTo": "urn:uuid:f7289ad5-0955-4c86-834c-fb54a736778b",
  "object": {
    "id": "urn:uuid:0370c0fb-bb78-4a9b-87f5-bed307a509dd",
    "object": "https://research-organisation.org/repository/preprint/201203/421/",
    "type": [
      "Offer",
      "coar-notify:ReviewAction"
    ]
  },
  "origin": {
    "id": "https://httpdump.app/dumps/29fc0d4c-aa11-4107-9f1f-e733d569f274",
    "inbox": "https://httpdump.app/dumps/29fc0d4c-aa11-4107-9f1f-e733d569f274",
    "type": "Service"
  },
  "target": {
    "id": "https://research-organisation.org/repository",
    "inbox": "https://research-organisation.org/inbox/",
    "type": "Service"
  },
  "type": [
    "TentativeReject",
    "coar-notify:ReviewAction"
  ]
}

ACK Endorsement Reject

{
  "@context": [
    "https://www.w3.org/ns/activitystreams",
    "https://purl.org/coar/notify"
  ],
  "actor": {
    "id": "https://review-service.com",
    "name": "Review Service",
    "type": "Service"
  },
  "context": {
    "id": "https://research-organisation.org/repository/preprint/201203/421/",
    "ietf:cite-as": "https://doi.org/10.5555/12345680",
    "type": "sorg:AboutPage",
    "url": {
      "id": "https://research-organisation.org/repository/preprint/201203/421/content.pdf",
      "mediaType": "application/pdf",
      "type": [
        "Article",
        "sorg:ScholarlyArticle"
      ]
    }
  },
  "id": "urn:uuid:4fb3af44-d4f8-4226-9475-2d09c2d8d9e0",
  "inReplyTo": "urn:uuid:0370c0fb-bb78-4a9b-87f5-bed307a509dd",
  "object": {
    "id": "urn:uuid:0370c0fb-bb78-4a9b-87f5-bed307a509dd",
    "object": "https://research-organisation.org/repository/preprint/201203/421/",
    "type": [
      "Offer",
      "coar-notify:ReviewAction"
    ]
  },
  "origin": {
    "id": "https://review-service.com/system",
    "inbox": "https://review-service.com/inbox/",
    "type": "Service"
  },
  "target": {
    "id": "https://research-organisation.org/repository",
    "inbox": "https://research-organisation.org/inbox/",
    "type": "Service"
  },
  "type": [
    "TentativeReject",
    "coar-notify:EndorsementAction"
  ]
}

ACK Review Accept

{
  "@context": [
    "https://www.w3.org/ns/activitystreams",
    "https://purl.org/coar/notify"
  ],
  "actor": {
    "id": "https://review-service.com",
    "name": "Review Service",
    "type": "Service"
  },
  "context": {
    "id": "https://research-organisation.org/repository/preprint/201203/421/",
    "ietf:cite-as": "https://doi.org/10.5555/12345680",
    "type": "sorg:AboutPage",
    "url": {
      "id": "https://research-organisation.org/repository/preprint/201203/421/content.pdf",
      "mediaType": "application/pdf",
      "type": [
        "Article",
        "sorg:ScholarlyArticle"
      ]
    }
  },
  "id": "urn:uuid:4fb3af44-d4f8-4226-9475-2d09c2d8d9e0",
  "inReplyTo": "urn:uuid:0370c0fb-bb78-4a9b-87f5-bed307a509dd",
  "object": {
    "id": "urn:uuid:0370c0fb-bb78-4a9b-87f5-bed307a509dd",
    "object": "https://research-organisation.org/repository/preprint/201203/421/",
    "type": [
      "Offer",
      "coar-notify:ReviewAction"
    ]
  },
  "origin": {
    "id": "https://review-service.com/system",
    "inbox": "https://review-service.com/inbox/",
    "type": "Service"
  },
  "target": {
    "id": "https://research-organisation.org/repository",
    "inbox": "https://research-organisation.org/inbox/",
    "type": "Service"
  },
  "type": [
    "Accept",
    "coar-notify:ReviewAction"
  ]
}

ACK Endorsement Accept

{
  "@context": [
    "https://www.w3.org/ns/activitystreams",
    "https://purl.org/coar/notify"
  ],
  "actor": {
    "id": "https://review-service.com",
    "name": "Review Service",
    "type": "Service"
  },
  "context": {
    "id": "https://dspace-coar.4science.cloud/items/e657437a-0ee2-437d-916a-bba8c57bf40b",
    "ietf:cite-as": "https://doi.org/10.5555/12345680",
    "type": "sorg:AboutPage",
    "url": {
      "id": "https://research-organisation.org/repository/preprint/201203/421/content.pdf",
      "mediaType": "application/pdf",
      "type": [
        "Article",
        "sorg:ScholarlyArticle"
      ]
    }
  },
  "id": "urn:uuid:4fb3af44-d4f8-4226-9486-2d09c2d8d9e0",
  "inReplyTo": "urn:uuid:c2d120a4-ee5d-4756-b4bf-8846406059d8",
  "object": {
    "id": "urn:uuid:0370c0fb-bb78-4a9b-87f5-bed307a509dd",
    "object": "https://research-organisation.org/repository/preprint/201203/421/",
    "type": [
      "Offer",
      "coar-notify:ReviewAction"
    ]
  },
  "origin": {
    "id": "https://httpdump.app/dumps/29fc0d4c-aa11-4107-9f1f-e733d569f274",
    "inbox": "https://httpdump.app/dumps/29fc0d4c-aa11-4107-9f1f-e733d569f274",
    "type": "Service"
  },
  "target": {
    "id": "https://research-organisation.org/repository",
    "inbox": "https://research-organisation.org/inbox/",
    "type": "Service"
  },
  "type": [
    "Accept",
    "coar-notify:EndorsementAction"
  ]
}

[steph-ieffam]

@paulwalk let me know I case you need something else

[paulwalk]

This is really useful @steph-ieffam - thanks very much for taking the time to make this easier for me to review.

[paulwalk]

Hi @steph-ieffam - this is my review of the above:

DSpace Pattern	v0.9.0?	v1.0.0?	Notes
Announce Review	✅	✅
Announce Endorsement	✅	✅
Announce Relationship	✅	✅	(you called this "Announce Release" above - but probably just a typo?)
Request Review	✅	✅
Request Endorsement	✅	✅
ACK Reject Review	✅	partially	Not entirely compliant with v1.0.0. v1.0.0 prefers the details of any resource object to be included in the detail of the top-level object (which represents the activity being rejected) instead of in the context . The use of context here does not break the specification, but it is likely to be ignored by other v1.0.0 compliant systems.
ACK Endorsement Reject	✅	partially	Not entirely compliant with v1.0.0. v1.0.0 prefers the details of any resource object to be included in the detail of the top-level object (which represents the activity being rejected) instead of in the context . The use of context here does not break the specification, but it is likely to be ignored by other v1.0.0 compliant systems.

There are also some patterns not mentioned above, e.g.

• Tentative Accept
• Reject
• Unprocessable Notification

Currently, therefore, DSpace does not break anything, but the use of context instead of the richer inner object in the "ack" patterns is not recommended.

So, the situation is not too bad really, and hopefully we can get DSpace to update its code in due course.

In the meantime, I have updated the 0.9.0 spec to align with the DSpace implementation of those "ack" patterns.

[steph-ieffam]

@paulwalk thanks for your check. I have one last question, what about the changes in the type field for the LDN Messages? If you check the provided screenshot you will see there's a difference in the type. Currently on DSpace 8 we have mapped the types to understand what action is being received by the system (this is needed to correctly process it). If this has been changed in the new version we will need to update the mapping as well in the next version to make it work with v1.0.0

for

• Tentative Accept (this is the same as for the other acceptance LDN so there's actually no big difference with other ACK)
• Reject (this is the same as for the other rejection LDN)
• Unprocessable Notification (I think this was never mapped in DSpace 8 prob. was not even available during the developments but I am not fully sure)

Yes sorry, that's a typo "Announce Release" is actually "Announce Relationship"

[paulwalk]

"Unprocessable Notification" was a recent addition, so that's why it was not implemented I think

However, there is also "Undo" which has been there since the start

Regarding the type property having two values in DSpace, e.g

"type": [
    "Accept",
    "coar-notify:ReviewAction"
  ]

I don't see this a problem, because the spec always allow more type values to be added to any notification. The only way I can see that this could be a problem would be if DSpace required the additional type value in order to process an incoming notification. But that seems unlikely? In any case, the type is in the object property in these cases.

@steph-ieffam - what do you think?

[steph-ieffam]

Undo was never mapped on DSpace 8 even though it could be supported so there's no prob. For type change we'll only have to change the mapping on DSpace 8 since now they have been mapped with two specific value for each notification type

[paulwalk]

OK - I am going to close this now. Thanks very much for engaging with this @steph-ieffam 👍