Enhancement: Implement event system for callbacks for publishing

BigRoy commented 1 year ago

Is there an existing issue for this?

[X] I have searched the existing issues.

Please describe the feature you have in mind and explain what the current shortcomings are?

The new publishing system of OpenPype should implement an Event system so custom callbacks can be implemented to publishing. This would be similar to what Pyblish API offers with its Events.

That would allow e.g. things like the psuedocode below for a studio to trigger custom events whenever anything happens during publishing.

import pyblish.api

def report_errors(context):
  has_error = any(result["error"] is not None for result in context.data["results"])
  if not has_error:
    return

  # some psuedocode examples below
  json_report = get_json_report(context)
  post_to_ayon_logs(json_report)
  post_to_sentry(json_report)
  post_to_slack(json_report, channel="#publish-errors")
  post_to_ayon_community_forum(json_report, topic="OMG. IT CRASHED AGAIN!")

pyblish.api.register_callback("published", report_errors)

Whether it's publishing through UI or standalone (e.g. in batch) we should be able to attach custom callbacks to these events occurring.

How would you imagine the implementation of the feature?

Basically mimic how Pyblish exposed it as well with its Events.

Are there any labels you wish to add?

[X] I have added the relevant labels to the enhancement request.

Describe alternatives you've considered:

I've thought of creating a custom plug-in but unfortunately that only triggers at that particular plug-in's order and thus won't capture hard errors which stop the new publisher, etc.

Additional context:

Previously Avalon used Pyblish and it supported to Pyblish-native feature and I'm not entirely sure but I believe pyblish_pype still supported it too. Would be great to still see support for a feature like this.

Also mentioned on discord.

[cuID:OP-6941]

BigRoy commented 10 months ago

Having the pluginProcessed events would be needed also for setting up a simple debug stepper.

https://user-images.githubusercontent.com/2439881/193165044-1eb37617-5096-49b8-ac95-676b4e8a06b2.mp4

It would be sweet however, if unlike Pyblish API that particular event also received the context as an argument instead of only results so hacks like this are not needed.

Old prototype for Pyblish QML here.

Would also be perfect if we can get a Create Context Created or Reset event so we can detect when the new publisher has just reset but has not yet started publishing processing plug-ins.

So events would be:

pluginProcessed: Emitted once per processed plug-in. (thus runs once for context plugins and once per instance for instance plugins)
pluginFailed: Emitted once per failed plug-in.
createContextCreated Emitted upon publish reset when it finished creating its create context
published: Emitted upon finished publish, regardless of failure

BigRoy commented 4 months ago

Working prototypes of the Pyblish debugger have been made with AYON by @MustafaJafar on the community forum here: https://community.ynput.io/t/pyblish-plugins-debugging/1447

With a recent release in pyblish-base this has also been greatly simplified to access the context, etc.

So this should be somewhat more accessible now. However the exact use case of triggering an event on any publish 'finished' I'm not sure currently exists in AYON. @iLLiCiTiT do you know? If it does exist, we can close it - otherwise we might want to transfer the issue to ayon core.

iLLiCiTiT commented 4 months ago

I don't know either. Transfer for double-check?

ynput / ayon-core