search

getavalon / core

The safe post-production pipeline - https://getavalon.github.io/2.0
MIT License
218 stars 49 forks source link

Background Publishing #235

Open mottosso opened 7 years ago

mottosso commented 7 years ago

Goal

Free up the application during long-running publishes and enable making changes to a scene during publish, without affecting the user.

Motivation

Down the line, one could run multiple operations too heavy for a single workstations, including caches that occupy most if not all of available RAM, on multiple remote machines, similar to how rendering works today.

Until then, here's an intermediate approach applicable today and without making too many assumptions on (1) where files run in the background are initially saved and (2) what mechanism to use for actually distributing jobs to computers.

Implementation

Offer an option-box to the standard method of publishing, in which the user may choose to publish in the foreground (default) or background.

untitled

Background in this context means cloning the current session on the local machine, opening up a temporarily saved version of the current scene and publishing it without options.

import os
import tempfile
import subprocess

from maya import cmds

tempdir = tempfile.mkdtemp()
fname = os.path.join(tempdir, "some_file.ma")

print("Saving..")
fname = cmds.file(fname, exportAll=True)

print("Dispatching..")
subprocess.Popen([sys.executable, "background_cli.py", fname])
mottosso commented 7 years ago

What about when validation fails? What about when publishing takes 30 minutes and you'd like to know when it finishes?

One approach is utilising a cluster management system, such as Thinkbox's Deadline or Pixar's Tractor. The process would be monitored at a high-level (e.g. CPU and memory use) and be able to indicate completion, possibly via notification. But an implementation would be highly tailored to the system itself.

Ideally, publishing in the background would work without additional tooling, using only what is already available at a workstation capable of publishing locally.

Here's a revised idea, this time launching the same GUI accessible to the local machine, but from a background process.

untitled

In this example, the user is notified that publishing has started successfully and can click the notification to view the full GUI. The GUI would be the same GUI run locally, as per the PYBLISH_GUI environment variable. If no variable is set, such as if publishing was happening elsewhere such as on a headless machine, then a GUI would simply not be provided.

If a GUI is available locally, then the background GUI is guaranteed to function as the background process inherits the parent process's environment.

In short, the spawned Maya would call the GUI much like the GUI is called from a local instance.

untitled

For this to happen, a few additions must be made to Pyblish QML.

import pyblish_qml
pyblish_qml.show(autostart=True, autoclose=True, hidden=True)

Where autostart means to start publishing as soon as the GUI appears and autoclose means to close the GUI upon finish. We'll need hidden as well as the GUI shouldn't be necessary unless there's a problem.

QML would also be responsible for the balloon notification, which should only be available when hidden=True in which case the balloon acts as a button to un-hide the GUI.

At the moment, QML would shut down alongside its host, but in this case we need it to remain open in case of failure so that the user is given a chance to inspect relevant messages. That means listening in on when the host closes and disable any mechanism to interact with it, such as Reset and Play.