Optional Plugins

[mottosso]

Goal

To facilitate for plugins that aren't showstoppers, but "nice to have"

Some plugins may not be critical to production, but may help the user in better understanding his or her work. For example, maybe the rendering pipeline has counter-measures for dealing with inverted normals. Thus, for modelers to produce correctly pointed normals is a definite plus, but not critical to production and may thus be skipped.

It also facilitates plugins that aren't a 100% bullet-proof or for plugins to have a "beta-testing" period and thus be optional until stable.

Architecture

As a user would need to be informed of plugins being optional, we will most likely be dependent on a GUI to visualise this. In the GUI, a checkbox-type indicator could signal to the user that the plugin is optional and may be skipped.

[tokejepsen]

Just to have a different use-case; reference versioning.

The plugin wouldn't be a publish stopper, but rather flag to the user that there is a new version of a file with the "Repair" button doing the updating of the reference.

[mottosso]

Good point, Toke, and probably one for much discussion. Such as, should a user be required to have updated to the latest version of an asset prior to publishing his work which is based on it?

In cases of pointcaches, publishing from an old reference might result in subsequent pipeline steps not being able to use the cache, in cases of topology or UV changes for instance.

Much dependent on the facility of course, and potentially varying per project, which means a good opportunity for a plugin with "loose" and "strict" options. :)

[mottosso]

For the record, here is a potential method of implementing optional plugins.

import pyblish

class MyValidator(pyblish.Validator):
     optional = True

In which case the optional defaults to False

[mottosso]

In the pull-request @ #114, optional plug-ins are never processed. However, should they process, but simply not be allowed to cancel a publish?

Picture this scenario; a user has a number of assets to be published and chooses not to include optional plug-ins. If an optional plug-in would have failed, the user would never know. Even though the error might not be a show-stopper, it could still be useful for the user to know that some validations are failing.

So the alternative might be to have optional plug-ins still process, but not cancel the publish. The potential downside however is the added time it takes to process even optional plug-ins. Especially if the process takes a substantial amount of time.

Should be also introduce a "skip" flag that is only applicable to optional plug-ins?

[tokejepsen]

However, should they process, but simply not be allowed to cancel a publish?

I can't think of a use-case for this. What could optional plugins that run on the users-choice be?

So the alternative might be to have optional plug-ins still process, but not cancel the publish

I vote for this. Mainly because I can think of a personal use-case for it; updating of references. I know... selfish:)

[mottosso]

I vote for this. Mainly because I can think of a personal use-case for it; updating of references. I know... selfish:)

This was what gave me the idea, that's one scenario where we're interested in the outcome of the validator, but possibly not concerned enough to interrupt the publish.

They would at best provide warnings as opposed to errors.

I can't think of a use-case for this. What could optional plugins that run on the users-choice be?

Sorry, could you rephrase or expand upon this?

[tokejepsen]

I can't think of a use-case for this. What could optional plugins that run on the users-choice be?

I seems a bit outside the scope of Pyblish to provide optional plugins, that the users run by choice. This seems just like tools, rather than dedicated validation. But maybe I just can't think of a use-case for it?

[mottosso]

Oh, you mean if there is a reason to have some plug-ins be optional at all?

I think you've just proved this, with the validation of latest version, no? By making the version validation optional, the responsibility is left to the user who can choose whether or not to be reminded in case of an out-dated asset.

Being too hard-core in validation can in some cases prove counter-productive, especially upon closing in on deadlines. For example, if you are at the end of production and need to update animation on a very complex rig. A rig that crashes when you update it due to unknown factors in your working file. Both you and the rest of production knows that the current version produce a point-cache that is compatible with other versions, however due to strict validation you cannot update and are forced to either investigate or re-do. Both of which takes time and effort and sometimes, perfection simply isn't worth either.

Whether or not to spend the extra minutes/hours on fixing this is of course up to company policy, and that's the point. Having optional plug-ins allows a company to have such a policy.

[mottosso]

Great questions, Toke, this is exactly the kind of things we should work out before releasing any new feature. :)

[tokejepsen]

Oh, you mean if there is a reason to have some plug-ins be optional at all?

So I vote for having optional plugins that do not cancel the publish, but still flags to the user any validation information. What I don't see a point in having is optional plugins, that the user decide to run or not. This would just be an external tool as far as I can see.

[mottosso]

all?

Are you not seeing the full message via email? Could you check that you're getting what's in the issue?

What I don't see a point in having is optional plugins

Maybe this will become more clear as we go. Here is one last example.

Here, validators "Muted channels" and "Custom constraints" are optional. Because regardless of whether or not there are muted channels in the scene, it might not have any affect on the resulting output; but merely give the user an indication of whether or not there may be a difference in output from what he is seeing in his scene.

Here, it is ultimately up to the user to decide whether or not having muted channels in his scene is okay or not.

Whether or not this should remain optional is left up to company policy. I suppose another way of putting is, with optional plug-ins, some validations may be for "best practices" whereas others are for "quality control".

[tokejepsen]

Are you not seeing the full message via email? Could you check that you're getting what's in the issue?

I am, but emailing back seems to mess with the text.

What I'm after is to have optional plugins, and then skippable plugins. The optional plugins the user can choose to enable as you have shown. Skippable plugins are not optional for the user, so they can't enable/disable them by choice, but when it comes to executing the publish these plugins won't cancel the publish but simply flag to the user any errors.

[mottosso]

Skippable plugins are not optional for the user, so they can't enable/disable them by choice, but when it comes to executing the publish these plugins won't cancel the publish but simply flag to the user any errors.

Ok, so sum things up, I think we have three categories of validators at this point.

1. Validators that always run and if it fails, the publish will not proceed.
2. Validators that always run and if it fails, the publish proceeds, but spits out a warning.
3. Validators that cancels a publish at failure, but are optional and may be bypassed by the user.

How about this.

Validators that should only provide a warning to the user may do so via logging.warning("This version is old") and simply not throw an exception, which would cancel the publish.

This way, it is a validator that can never fail but still communicate a warning. This could be a "ValidateLatestVersion" plug-in for example.

[tokejepsen]

good idea:)

[mottosso]

Cool. :)

[mottosso]

Implemented in 1.0.8, #114