Pyblish's registered test function not respected in pyblish-qml UI

[jboisvertrodeofx]

If we register a custom test function using pyblish.logic.register_test, it is respected by the pyblish.util.publish function and by the pyblish-lite UI, but it is completely ignored by the pyblish-qml UI.

It seems that pyblish_qml.control.Controller.iterator always uses the default default_test function instead of the one that was registered with pyblish.logic.register_test.

We use a custom test function to completely stop the execution of all plugins as soon as an error is raised, except for validator plugins. For example, if an error is raised during the execution of an extractor plugin, we stop the execution so that not only the other extractors are not executed, but also the integrators.

pyblish-lite

pyblish-qml

Here is a code sample to replicate the problem:

import pyblish.api
import pyblish.logic
import pyblish_lite
import pyblish_qml

class TestCollector1(pyblish.api.ContextPlugin):

    order = pyblish.api.CollectorOrder

    def process(self, context):
        print("### {0}".format(self.__class__.__name__))
        context.create_instance("instance1")

class TestCollector2(pyblish.api.ContextPlugin):

    order = pyblish.api.CollectorOrder + 0.01

    def process(self, context):
        print("### {0}".format(self.__class__.__name__))

class TestValidator1(pyblish.api.InstancePlugin):

    order = pyblish.api.ValidatorOrder

    def process(self, instance):
        print("### {0}".format(self.__class__.__name__))

class TestValidator2(pyblish.api.InstancePlugin):

    order = pyblish.api.ValidatorOrder + 0.01

    def process(self, instance):
        print("### {0}".format(self.__class__.__name__))

class TestExtractor1(pyblish.api.InstancePlugin):

    order = pyblish.api.ExtractorOrder

    def process(self, instance):
        print("### {0}".format(self.__class__.__name__))
        raise pyblish.api.PyblishError("bla bla")

class TestExtractor2(pyblish.api.InstancePlugin):

    order = pyblish.api.ExtractorOrder + 0.01

    def process(self, instance):
        print("### {0}".format(self.__class__.__name__))

class TestIntegrator1(pyblish.api.InstancePlugin):

    order = pyblish.api.IntegratorOrder

    def process(self, instance):
        print("### {0}".format(self.__class__.__name__))

class TestIntegrator2(pyblish.api.InstancePlugin):

    order = pyblish.api.IntegratorOrder + 0.01

    def process(self, instance):
        print("### {0}".format(self.__class__.__name__))

def testStrictPluginContinuation(**kwargs):
    # Stop execution if any of the collectors fail.
    for order in sorted(kwargs["ordersWithError"]):
        if pyblish.api.CollectorOrder <= order < pyblish.api.ValidatorOrder:
            return "failed collection"

    # When validators are all executed, stop execution as soon as there is an error.
    if kwargs["nextOrder"] >= pyblish.api.ExtractorOrder:
        for order in sorted(kwargs["ordersWithError"]):
            # Check if validators failed.
            if pyblish.api.ValidatorOrder <= order < pyblish.api.ExtractorOrder:
                return "failed validation"

            # Check if extractors failed.
            if pyblish.api.ExtractorOrder <= order < pyblish.api.IntegratorOrder:
                return "failed extraction"

            # Check if integrators failed.
            if pyblish.api.IntegratorOrder <= order:
                return "failed integration"

pyblish.api.deregister_all_paths()
pyblish.api.deregister_all_plugins()

pyblish.api.register_plugin(TestCollector1)
pyblish.api.register_plugin(TestCollector2)
pyblish.api.register_plugin(TestValidator1)
pyblish.api.register_plugin(TestValidator2)
pyblish.api.register_plugin(TestExtractor1)
pyblish.api.register_plugin(TestExtractor2)
pyblish.api.register_plugin(TestIntegrator1)
pyblish.api.register_plugin(TestIntegrator2)

pyblish.logic.register_test(testStrictPluginContinuation)

# pyblish_lite.show()

server = pyblish_qml.show()
server.wait()

[mottosso]

Thanks for reporting this @jboisvertrodeofx, this is an oversight. The test is being used, it's just not coming from the host.

https://github.com/pyblish/pyblish-qml/blob/835bf3bcc210fb181392502159438237597698ec/pyblish_qml/control.py#L321-L336

This is being called from the QML process. So what you could do, as a temporary and perhaps permanent workaround, is register your test in the QML parent process. For example, by putting it in a userSetup.py that's on your PYTHONPATH prior to starting QML.

One approach would be something like..

1. Launch e.g. Maya/Houdini as per usual
2. In Maya's userSetup.py, append another userSetup.py to PYTHONPATH
3. Launch QML

That way, it'll be passed onto QML, and Maya et. al. won't see it.

[jboisvertrodeofx]

Thanks for the quick reply @mottosso ! I think your workaround could definitely work and we'll probably end up doing something similar to that on our end.

I'm still a bit curious about how we could go about fixing this in a permanent proper way though. I'm still looking through the code, and I know we're at least passing targets to the server from the client, so I'm wondering if we could pass the registered test function as a serialized string to the server as well. And of course, there might be other ways to do this as well.

[mottosso]

Properly would mean communicating with the host, like Maya. Communication currently happens through a "service", which exposes a select few function calls to QML from the host. And from the looks of it, there is already support for doing this.

Called from QML

https://github.com/pyblish/pyblish-qml/blob/835bf3bcc210fb181392502159438237597698ec/pyblish_qml/ipc/client.py#L49-L51

Called from Host

https://github.com/pyblish/pyblish-qml/blob/835bf3bcc210fb181392502159438237597698ec/pyblish_qml/ipc/service.py#L34-L36

So, it should only be a matter of replacing..

https://github.com/pyblish/pyblish-qml/blob/835bf3bcc210fb181392502159438237597698ec/pyblish_qml/control.py#L358

..with

if self.host.test(**state):

And things should just magically work. From the looks of it, this was already the case way back when, and I'm not sure why it wasn't kept.

https://github.com/pyblish/pyblish-qml/blob/835bf3bcc210fb181392502159438237597698ec/pyblish_qml/control.py#L1159

If you get around to testing (pun!) it, I'd be curious to know how it goes!

[jboisvertrodeofx]

Thanks again @mottosso ! It was slightly more complicated, but I got it to work following your advice!

I had to add the kwargs argument back to both of the _dispatch functions, add some missing double asterisks to the test function, and fix some JSON serialization issue with the set from the state dictionary's "ordersWithError" key (I made a custom Encoder/Decoder to support Python sets).

I made a branch on our work repo, but I'll do some more testing to make sure there are no new problems.

[mottosso]

Nice one!

[hannesdelbeke]

can this issue be closed since the PR went in?

[mottosso]

Yes, good idea!