Major Rewrite

[tokejepsen]

First off this is not backwards compatible. As I've talked about in the forums, I've remove the remote submission feature. I've also simplified the plugin resposibilities to formatting of data for Deadline submission.

When it comes to updating plugins for Deadline submission it should only be a matter of adding Name, UserName and Plugin to the deadlineData["job"]. Also very important is to add a deadline family to the instance.

This PR also introduces a proper integration with Deadline with an added event plugin. Please see the readme docs for more details.

[tokejepsen]

There will be some updates to this before merging, so this is just to get a discussion / making people aware of changes.

[tokejepsen]

I've tested the rest of our pipeline with these latest commits, and everything is working as expected.

[tokejepsen]

Merging this as its in production both at Bait Studio and Kredenc

[mottosso]

Sorry I can't help with testing this, but if it helps, I could review the overall code quality and maybe make a few suggestions?

[tokejepsen]

Sorry I can't help with testing this, but if it helps, I could review the overall code quality and maybe make a few suggestions?

Go for it:)

[mottosso]

One question, what is new_context?

[tokejepsen]

One question, what is new_context?

We are reordering the instances dependent on the order data member. This is so we can submit instances that are dependent on each other. Practical case is to precomp something in Nuke.

[mottosso]

Ah, I see. Perhaps there is room for clarification there.

What do you think about something like this?

for instance in sorted(context, key=lambda instance: instance["deadlineData"]["order"]):
   # do things in your custom order.

That might allow you to strip everything here and above.

[tokejepsen]

What do you think about something like this?

Yeah, that would probably work well, since we are now using a specific family for this plugin.

[mottosso]

Do you know if there's a trail version or the like that one could setup via the command-line, so that we can test things?

Like, if we make this change, could we have a series of tests to ensure we didn't break anything? I realise it's farm-software, but I would try starting a server locally and submitting things to that. And shut it down once the tests are complete. Does it require Maya, or can it be used standalone?

[tokejepsen]

Do you know if there's a trail version or the like that one could setup via the command-line, so that we can test things?

They used to have a version where you could run maximum of two machines as slaves. Querying on the forums; https://forums.thinkboxsoftware.com/viewtopic.php?f=11&t=14738

Does it require Maya, or can it be used standalone?

All plugins are optional, so we could probably run the python plugin for testing.

[tokejepsen]

Seems like you can still get a trial version with two slaves running. You just need to contact sales; http://www.thinkboxsoftware.com/sales-contact/

[mottosso]

That's great, but probably not what we're looking for. We'd need something we can setup on Travis, where we would install it, prior to running the tests. If licenses are involved, then we can't publish it online.

This is the same problem we're facing with unit-testing the ftrack extension, or any package that depends on software without a public demo. :( It's unfortunate, it means we'll never be able to build tests for it.

[tokejepsen]

It's unfortunate, it means we'll never be able to build tests for it.

Yeah, I know. Think its worth setting up tests, that can't be run on Travis? So basically we need to manually run a test suite to make sure it works.

[mottosso]

Having tried this with maya-capture, I can tell you from experience that it won't work. It will be forgotten, overlooked or ignored. And then one day, bam, code is a mess.

What we could do however is either build or request a mock of Deadline. Something that acts like it, that we can run locally. Something that doesn't really schedules anything, but acts like it does.

They might already have it, and otherwise if you can narrow down the points in your code where you communicate with Deadline, then we could have a look at trying to build a simple HTTP server that imitates it.

Odds are, in particular with those event plug-ins, that it'd be too great of a challenge to be worth it.

[tokejepsen]

I'll add this as an issue for the future.

[mottosso]

Ok, I'll add some reference to the issue once you've got it.

[tokejepsen]

https://github.com/pyblish/pyblish-deadline/issues/32