Entity change events: Be able to detect source of change

[iLLiCiTiT]

Story

For 2 way sync of production trackers it is necessary to know "Was this change on entity caused by my service?". If it was caused by my service, then I can ignore the entity change and not sync it back.

Problems

There are many, sync is usually processing events in some order, and the queue of events is not ideal. That can lead to infinite loops of event processing. Imagine events queue with EventA changing status to "In Progress" and EventA to "Approved" on same entity, when those are started to be processed, then statuses are changed it in AYON, but because of 2 ways sync I also will get information from AYON that I should change the status in the prod tracker too -> this small example will cause that the entity will switch statuses between "In Progress" and "Approved" until someone manually stops it.

Also it usual that both sides AYON and other production tracker have automations that are changing something based on previous change (e.g. change version status based on task and vica versa), which can cause infinite loops on another level.

Currently this can be achieved by checking user who did it, but services should in ideal case work "as different user", if UserA did change in AProdTracker, it should look like UserA did the same cange in AYON too, so that is not really possible. Also the same service user can be used for multiple different services.

Proposal

Maybe the event related to entity changes should have additional information source. Source could be, ayon-editor (Editor frontend). ftrack-sync, shotgrid-sync, publishing etc. Which could be None (to be backwards compatible). It could be based on header information.

[martastain]

Default could be None, that would mean it originated from the user interaction, while services/python-api could add an additional header. Are you planning using that in enroll filter somehow?

Also. Is this a duplicate of https://github.com/ynput/ayon-backend/issues/132 ?

[iLLiCiTiT]

Default could be None, that would mean it originated from the user interaction, while services/python-api could add an additional header.

Or older ayon api was used (not relevant for my use-case). I would say it would be good to know how did user change it even if it was done via frontend, for reference (personal opinion).

Are you planning using that in enroll filter somehow?

I did think about it as, ignore event after enroll, but if it would be part of enroll filters it would work too.

Also. Is this a duplicate of https://github.com/ynput/ayon-backend/issues/132 ?

Yes, and I don't understand how I couldn't find it. I scrolled through issues twice.

[martastain]

Adding original @jakubjezek001 comment from here and closing the original https://github.com/ynput/ayon-backend/issues/132

At the moment it is impossible to filter out all events with all service users from particular service. This way we could avoid processing loops only from particular services and keep others.

In ShotGrid for example we are no filtering all events for entity attrib changes but we can only avoid loop with getting all users which are set as isService and then enroll an event which would be avoiding those so we are not looping events. But this will in future probably create some issue if for example under the same worker will run Ftrack and what ever is changed in here we will need to be reflected in ShotGrid.

[martastain]

Couldn't we use deterministic sender values? if a service just identifies itself as ftrack-sync.{service_name} it will be backwards compatible, we store sender in the database and it should be easy to query.

[martastain]

we use a hash (unique to a tab) as a sender of events originating from the web interface, but we could add webui. prefix there as well.

[iLLiCiTiT]

If we would use sender, are we able to get user who did it?

[martastain]

After a discussion with @iLLiCiTiT we think, we should add additional field sender_type to events. The field would be nullable, set to api whenever the action was triggered using rest and overridable using x-sender-type header. Web interface would for example populate it with webui, FTrack services would use ftrack.sync or whatever necessary. We should also have a simple way of filtering by this value both in graphql and enroll

[iLLiCiTiT]

I think this is the best as it allows easier indexing and is more "explicit". I would maybe use source_name instead of sender_type but that is technicality.