Demonstrating Federation – discussion

[gsvarovsky]

Ticket for discussion of /wiki/Demonstrating-Federation

[gsvarovsky]

(Similar question asked by @kroky on Gitter)

Proposed tests

1. Data update request

This workflow implies a "request" end-point (or other API) on the other system, which is not something we've discussed yet and is additional to our proposed HTTP POST approach.

Perhaps it would be better to jump straight into systems making the POST requests to other federated systems on their own schedule, without user prompting; ideally, immediately after the data changes. This way, the integration will be fully automatic, i.e.

User makes a change in system1 -[system1-defined schedule]-> system1 POSTs to system2 --> Manager observes change in system2

In this scheme we also do not need the "enact transfer" and "initiate update" actions in the Demo scripts section, since these triggers happen automatically.

[mcalligator]

Thanks for highlighting the request endpoint consideration - we should discuss this during our next Federated Timesheets call and agree jointly on the most appropriate approach.

The replication behaviour you suggest would apply best to Proposed test 3, so I have updated the Wiki page accordingly, including the Demo scripts.

[michielbdejong]

Paging and filtering are also relevant here, most APIs have something like that. Prejournal has filtering per project and paging by local entry ID. Do we have a wiki page yet that details the 3 APIs of the 3 live systems?

[michielbdejong]

See also https://github.com/federatedbookkeeping/timesheets/issues/41 for discussion of how to push to / pull from time.pondersource.com.

[mcalligator]

Do we have a wiki page yet that details the 3 APIs of the 3 live systems?

Currently, no; each of the systems' APIs is documented separately (and incompletely). I've created a page in this repo's Wiki where that information can be captured.

[michielbdejong]

Re https://github.com/federatedbookkeeping/timesheets/wiki/Demonstrating-Federation/b0e4fd5f5a59cdee5a207a3f9112e801a4ef96d6 point 1, it seems that "when a data update occurs in the tested system" describes 3 of our 9 milestones (namely milestones 1a, 2a, and 3a). This is when data comes into the federation. We should demonstrate each supported time tracker app separately. Not only our in-house ones but also the ones we support import from.

[michielbdejong]

To demonstrate point 2 we would have to deliberately misconfigure one of the systems involved. This should never happen in the real federation, I think? It's good to demonstrate though, of course.

Point 3 is more a design decision, but yes, also good to demonstrate how each of the tier-2 systems implements this. It shouldn't be possible to edit data unless that data is "at home" at the current system.

Point 4 seems to be about digital signatures, which is milestone 6, right? We're not demonstrating this for 1a/1b/2a/2b/3a/3b, right?

Point 5 also a good point.

Also, we should demonstrate that in case of a scheduled sync, running it twice doesn't lead to duplicate data in the receiving system.

I'll have a go at demonstrating these things for milestone 1a.

[mcalligator]

Note: the numbering of the points has changed with the update to this Wiki page. The correspondence between previous and current numbering has been made clear where that affects quoted replies here:

To demonstrate point 2 we would have to deliberately misconfigure one of the systems involved. This should never happen in the real federation, I think? It's good to demonstrate though, of course.

Most definitely; robust federation relies on equally robust data submission, which includes notifying a sending system if an invalid data format has been used.

Point 3 [now Point 4] is more a design decision, but yes, also good to demonstrate how each of the tier-2 systems implements this. It shouldn't be possible to edit data unless that data is "at home" at the current system.

Given the differences in approach to access control that have emerged between PreJournal and the other two federated systems in the project, this test needs to be updated to reflect those.

Point 4 [now Point 5] seems to be about digital signatures, which is milestone 6, right? We're not demonstrating this for 1a/1b/2a/2b/3a/3b, right?

That's correct; the inclusion of digital signatures across the board for data transfer between federated systems is not in scope for milestones 1 through 3. We will need to elaborate on precisely how data integrity can be demonstrably validated through he use of signatures.

Also, we should demonstrate that in case of a scheduled sync, running it twice doesn't lead to duplicate data in the receiving system.

Good point; that's now in the list.

[mcalligator]

The only unresolved matter currently on that Wiki page is that of testing and demonstrating data deletion. I'm happy to take suggestions here, and will update the page accordingly once we've reached a consensus.

[kroky]

Deletion makes sense to be push-only approach as pulling won't show the absence of an entry unless you compare your own set of entries to all remote entries (which will be inefficient). How about a push approach to a delete entry endpoint of remote API that is triggered whenever a local entry is deleted?