Open BigRoy opened 5 years ago
inputs
and outputs
:)I've implemented the logic for collecting the inputs
data for Maya, Houdini and Fusion in our colorbleed-config as an example. Code can be found in this commit however note that it hasn't been tested thorougly but it did work for my current tests. So should be a good reference to support it after it's implemented in Avalon core to be visualized.
In short the changes required in a studio config to add support for this for a host:
integrate.py
in the example commit above)Here's a little preview of a potential visualization for it in action. A little Input/Output widget for publishes. Also shared in on Avalon Gitter here
The top entries that are not indented are the ones the inputs/outputs are being listed for. In this image it's listing for the pointcacheExplode
at the top.
Some input would be very welcome! :)
Here's a quick preview of how it could appear in the Loader - currently implemented it quickly as just a Loader in the studio config for testing.
Also rolled out the collection of inputs from two comments above (with some minor bug fixes) so our pipeline is currently collecting data for current production. In that way I have something useful to start visualizing... ;)
This is great. The only thing I'd change would be trying to display it as node graph to make it clearer how the data flow, but that's just a UI thing. The data collection and data member names are solid.
We can give this a shot on our end, but I think this is a great candidate for feature that should be in the core.
I'd personally give people the option to show it in both loader and scene manager. I can think of situations where it's useful in both places and if it runs as action anyway, that it's trivial to show in multiple places.
that being said, it would deserve to be in the main avalon menu as well, with added checkbox to let you display inputs and outputs based on scene selection
It's useful in so many scenarios. At this case I'm mostly interested in rolling out at least a first design so we can iteratively start improving it in production!
The only thing I'd change would be trying to display it as node graph to make it clearer how the data flow, but that's just a UI thing.
This is one of those things that's definitely much prettier to debug in a good node/flow graph however I've never seen a trivial light-weight Python implementation of one that we can easily embed with Avalon or the tool - or at least I haven't used them. Here are some links just as a reference:
The first one already seems to be using Qt.py and has no other dependencies so could be a potentially usable one.
I did a quick prototype with NodeGraphQt
:
The layout/positioning unfortunately is done manually
However, there's no auto-layout functionality in NodeGraphQt
which makes it pretty much useless at this point for generating a nice and tidy graph. Visually the design looks pretty good. Auto-layout could potentially be implemented by using networkx
to compute a formatted graph layout and just using those values to position the graph itself. With networkx
this would then become trivial.
Some references/options for layout graph algorithms (not all Python):
It's just a regular Qt widget and it has the necessary callbacks to also allow more debugging information to pop-up as you click a specific node in the graph (like a widget showing the representation's information).
Here's the code I used to generate this prototype. Please note that I hardcoded a representation id to test with.
This is related to #159 :)
@davidlatwe did you make any progress on your side regarding Input/Output tracking?
Hi @BigRoy , I haven't !
To keep this alive - we recently required visualizing dependencies for a project and I've tweaked a prototype to have a search bar and some filters.
Here's a preview of that prototype:
Some explanations:
teun
is in the searchbar at top to find only assets/subsets in history that have that in the namehost.ls()
of the registered host in Avalon) so the Artist could easily find what history is related to what he's loaded in his sceneAdding search and filters helps the artists to find in the input history the information he needs to compare with.
In this case we wanted to retrieve the information from a rendered comp which used renders from Maya to detect which versions of animation pointcaches were used at that stage. So the Houdini FX could make sure to match up the pointcache versions with the comp plate he's using.
I've made an explicit "refresh inputs/outputs" button due to the history possibly being a LOT of data. A comp could have many renderlayers, which all could have many characters, which in turn all have rigs or models in history.
Here's the relevant code to reproduce this Avalon representations inputs/outputs prototype UI in a gist. It's not a PR/branch since it contains code snippets from config, local development prototypes, etc. and it's quite quickly thrown together so the code might be a bit ugly here and there.
Feature
I would like to implement some sort of Tracking of the inputs that are used to generate a specific publish. This will allow upstream input and downstream output tracking to be visualized.
For example it should allow us to trace:
Similarly it allows you to trace wherever
pointcache v007
was used to generate new outputs.Example uses cases would be:
Terminology
I'm going for Inputs/Outputs as regards to terminology, but there are plenty of other options.
Other ideas that passed:
One of the other expected names for it would be "dependencies" however that's currently already taken as definition for those dependencies of a representation that it requires to operate after load (see representation schema) which was also used in https://github.com/getavalon/core/issues/129.
Implementation
We'd only need to store
representation["data"]["inputs"]
as a list of MongoDB document ids (avalon.io.ObjectId
) and the data should be ready for visualization.With only that data present I've prepared the following visual design with Qt:
This shows the inputs to
pointcacheExplode
. A toggle to switch between "inputs" and "outputs" should still be added to visually see wherever that publish was used to generate other publishesPreview Pull Request coming soon..
I'll prepare a PR for both Avalon core and share a link to a PR on colorbleed-config to see how a studio could manage collecting these inputs at publish time for Maya, Fusion and Houdini. Expected this Friday or next Monday.
References: