rez-env does not apply the environment in 2.0.0 on windows

[instinct-vfx]

I deployed 2.0.0 from master in exactly the same way i did with the RC before. When using rez-env package it seems the shell is never really created/applied. (e.g. there are no REZ_* variables and commands() has not been called). When using rez-env package -- command everything seems to work as expected.

[instinct-vfx]

This is due to this change:

https://github.com/nerdvegas/rez/commit/f764025489d946a5e95f381f0a5fefa190d1393e

Basically the new version seems to just exit the shell if you do not provide a command directly which kind of defeats the purpose of spawning an interactive shell.

@skrall @michael.morehouse : Michael seems to have commited the change based on a change by Sebastian. Could you please elaborate what the goal of this change is and if it also breaks the same way for you than it does for me?

[yawpitch]

I'm having the same issue and that change was the cause; the problem is really you want /C in any situation in which you're passing in a command and /K if what you're doing is spawning a subshell to enter commands in interactively, and as is you're getting either one compromise behavior or the other. I think Sebastian had noticed that /K was leaving your command hanging in a subshell that was waiting for an explicit "exit" call.

That said I've also noticed some problems with the way --detached and --new-session work, and I think we're passing subprocess.NEW_CONSOLE where we should be passing subprocess.NEW_PROCESS_GROUP, as even with /K I'm getting weird behavior when I create a new and detached session.

I'll take a look at some new logic that attempts to account for an incoming command.

[instinct-vfx]

But isn't that treated differently already? I thought when providing a command to run there is no actual shell created, but ResolvedContext directly applies to the python environment and runs the command which is the reason for certain shell features not working with double-dash commands (like alias).

[yawpitch]

I'll have to do some more testing; I've been trying out all the different permutations of calls and I've ended up in several states where I've got a command resulting in a subshell that's requiring an extra "exit" when it really shouldn't.

Reverting the /C to /K should give back the old behavior, but I do think there's something buggy in the behavior when --detached and/or --new-session are passed in.

On Aug 4, 2016, 00:31 -0700, Thorsten Kaufmann notifications@github.com, wrote:

But isn't that treated differently already? I thought when providing a command to run there is no actual shell created, but ResolvedContext directly applies to the python environment and runs the command which is the reason for certain shell features not working with double-dash commands (like alias).

— You are receiving this because you commented. Reply to this email directly, view it on GitHub (https://github.com/nerdvegas/rez/issues/336#issuecomment-237475265), or mute the thread (https://github.com/notifications/unsubscribe-auth/AAnEp7BIIt6maI1TM9QHdous5_0sPp7Hks5qcZVNgaJpZM4JbYFf).

[nerdvegas]

No, a shell is actually created, otherwise the only effect on the env would be environment variables. You can do that in the API - see ResolvedContext.execute_command() - but that isn't what's used when you run "rez-env pkgs -- command".

As an aside though it could be useful to expose this functionality in rez-env.

A

On Thu, Aug 4, 2016 at 12:31 AM, Thorsten Kaufmann <notifications@github.com

wrote:

But isn't that treated differently already? I thought when providing a command to run there is no actual shell created, but ResolvedContext directly applies to the python environment and runs the command which is the reason for certain shell features not working with double-dash commands (like alias).

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/nerdvegas/rez/issues/336#issuecomment-237475265, or mute the thread https://github.com/notifications/unsubscribe-auth/ABjqSlaBcfzEl70O4IugA7Fa1N6E2cUtks5qcZVLgaJpZM4JbYFf .

[yawpitch]

Seems to me there's basically six desirable outcomes, which are to resolve the env and:

1) create an interactive child shell process in the existing console window

2) create an interactive child shell process in a new console window

3) create an interactive shell process in a new process tree in the existing console window

4) create an interactive shell process in a new process tree in a new console window

5) create a non-interactive version of 1

6) create a non-interactive version of 3

7) do not create a new shell at all, but spit back the appropriate shell code to create one (i.e. allow something like "rez env --shell=bash --pipe python | bash" ... this being most useful to someone like me who is trying to wrap Rez in an artist-facing contextual configuration manager.

Basically on Windows you'll ultimately need /C if your intent is to have the subshell run a command and exit back to an unmodified parent shell, and /K for any interactive work.

M

On Aug 4, 2016, 08:34 -0700, allan johns notifications@github.com, wrote:

No, a shell is actually created, otherwise the only effect on the env would be environment variables. You can do that in the API - see ResolvedContext.execute_command() - but that isn't what's used when you run "rez-env pkgs -- command".

As an aside though it could be useful to expose this functionality in rez-env.

A

On Thu, Aug 4, 2016 at 12:31 AM, Thorsten Kaufmann <notifications@github.com

wrote:

But isn't that treated differently already? I thought when providing a command to run there is no actual shell created, but ResolvedContext directly applies to the python environment and runs the command which is the reason for certain shell features not working with double-dash commands (like alias).

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/nerdvegas/rez/issues/336#issuecomment-237475265, or mute the thread https://github.com/notifications/unsubscribe-auth/ABjqSlaBcfzEl70O4IugA7Fa1N6E2cUtks5qcZVLgaJpZM4JbYFf .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub (https://github.com/nerdvegas/rez/issues/336#issuecomment-237590880), or mute the thread (https://github.com/notifications/unsubscribe-auth/AAnEp5RMkmgD-1lGnFTzwyvPjFWjgciFks5qcgZogaJpZM4JbYFf).

[nerdvegas]

Just a note: Adding the rez-env mode that just prints out the resulting shell code would be pretty trivial to add. It's something I've considered also. But, you can actually already do it, though it's a bit obfuscated:

]$ rez-env mypackages -o - | rez-context -i -

Use rez-context -f to change the format/shell language.

Hth A

On Thu, Aug 4, 2016 at 11:11 AM, Michael Morehouse <notifications@github.com

wrote:

Seems to me there's basically six desirable outcomes, which are to resolve the env and:

1) create an interactive child shell process in the existing console window

2) create an interactive child shell process in a new console window

3) create an interactive shell process in a new process tree in the existing console window

4) create an interactive shell process in a new process tree in a new console window

5) create a non-interactive version of 1

6) create a non-interactive version of 3

7) do not create a new shell at all, but spit back the appropriate shell code to create one (i.e. allow something like "rez env --shell=bash --pipe python | bash" ... this being most useful to someone like me who is trying to wrap Rez in an artist-facing contextual configuration manager.

Basically on Windows you'll ultimately need /C if your intent is to have the subshell run a command and exit back to an unmodified parent shell, and /K for any interactive work.

M

On Aug 4, 2016, 08:34 -0700, allan johns notifications@github.com, wrote:

No, a shell is actually created, otherwise the only effect on the env would be environment variables. You can do that in the API - see ResolvedContext.execute_command() - but that isn't what's used when you run "rez-env pkgs -- command".

As an aside though it could be useful to expose this functionality in rez-env.

A

On Thu, Aug 4, 2016 at 12:31 AM, Thorsten Kaufmann < notifications@github.com

wrote:

But isn't that treated differently already? I thought when providing a command to run there is no actual shell created, but ResolvedContext directly applies to the python environment and runs the command which is the reason for certain shell features not working with double-dash commands (like alias).

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/nerdvegas/rez/issues/336#issuecomment-237475265, or mute the thread https://github.com/notifications/unsubscribe-auth/ ABjqSlaBcfzEl70O4IugA7Fa1N6E2cUtks5qcZVLgaJpZM4JbYFf .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub ( https://github.com/nerdvegas/rez/issues/336#issuecomment-237590880), or mute the thread (https://github.com/notifications/unsubscribe- auth/AAnEp5RMkmgD-1lGnFTzwyvPjFWjgciFks5qcgZogaJpZM4JbYFf).

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/nerdvegas/rez/issues/336#issuecomment-237636236, or mute the thread https://github.com/notifications/unsubscribe-auth/ABjqShWWWqb_sD3SKY7jkXxMTthsAKNCks5qcitcgaJpZM4JbYFf .

[yawpitch]

Ahh, cool. Non-obvious, but cool.

On Aug 4, 2016, 11:43 -0700, allan johns notifications@github.com, wrote:

Just a note: Adding the rez-env mode that just prints out the resulting shell code would be pretty trivial to add. It's something I've considered also. But, you can actually already do it, though it's a bit obfuscated:

]$ rez-env mypackages -o - | rez-context -i -

Use rez-context -f to change the format/shell language.

Hth A

On Thu, Aug 4, 2016 at 11:11 AM, Michael Morehouse <notifications@github.com

wrote:

Seems to me there's basically six desirable outcomes, which are to resolve the env and:

1) create an interactive child shell process in the existing console window

2) create an interactive child shell process in a new console window

3) create an interactive shell process in a new process tree in the existing console window

4) create an interactive shell process in a new process tree in a new console window

5) create a non-interactive version of 1

6) create a non-interactive version of 3

7) do not create a new shell at all, but spit back the appropriate shell code to create one (i.e. allow something like "rez env --shell=bash --pipe python | bash" ... this being most useful to someone like me who is trying to wrap Rez in an artist-facing contextual configuration manager.

Basically on Windows you'll ultimately need /C if your intent is to have the subshell run a command and exit back to an unmodified parent shell, and /K for any interactive work.

M

On Aug 4, 2016, 08:34 -0700, allan johns notifications@github.com, wrote:

No, a shell is actually created, otherwise the only effect on the env would be environment variables. You can do that in the API - see ResolvedContext.execute_command() - but that isn't what's used when you run "rez-env pkgs -- command".

As an aside though it could be useful to expose this functionality in rez-env.

A

On Thu, Aug 4, 2016 at 12:31 AM, Thorsten Kaufmann < notifications@github.com

wrote:

But isn't that treated differently already? I thought when providing a command to run there is no actual shell created, but ResolvedContext directly applies to the python environment and runs the command which is the reason for certain shell features not working with double-dash commands (like alias).

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/nerdvegas/rez/issues/336#issuecomment-237475265, or mute the thread https://github.com/notifications/unsubscribe-auth/ ABjqSlaBcfzEl70O4IugA7Fa1N6E2cUtks5qcZVLgaJpZM4JbYFf .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub ( https://github.com/nerdvegas/rez/issues/336#issuecomment-237590880), or mute the thread (https://github.com/notifications/unsubscribe- auth/AAnEp5RMkmgD-1lGnFTzwyvPjFWjgciFks5qcgZogaJpZM4JbYFf).

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/nerdvegas/rez/issues/336#issuecomment-237636236, or mute the thread https://github.com/notifications/unsubscribe-auth/ABjqShWWWqb_sD3SKY7jkXxMTthsAKNCks5qcitcgaJpZM4JbYFf .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub (https://github.com/nerdvegas/rez/issues/336#issuecomment-237645439), or mute the thread (https://github.com/notifications/unsubscribe-auth/AAnEp8W_PMUrhtUzgyUwf-Z2SPszj2BQks5qcjK8gaJpZM4JbYFf).

[skral]

Sorry, I didn't see the issue earlier. This was working for me but I probably missed some use cases when testing it.

[yawpitch]

Hey Allan, I'm flying a bit blind on some Windows fixes ... can you give me an idea of precisely what is intended to happen (in terms of processes launched and terminal windows opened) on Linux in the following situations:

1) rez env python -- python -c "import sys; sys.exit()" 2) rez env --detached python --python -c "import sys; sys.exit()" 3) rez env --new-session python --python -c "import sys; sys.exit()" 4) rez env --detached --new-session python --python -c "import sys; sys.exit()"

I'm pretty sure the Windows behavior is very wrong in some subtle ways (like CTRL+C not killing child processes when it should, etc), but I'm not sure what the bar is we're aiming for.

Also when a new terminal is launched in Linux do you do anything to set the title?

Thanks,

M

[yawpitch]

Ok, here's the summary of my findings ... YES, the /C should be a /K, but the issues with the Windows shell go deeper than that.

TL;DR version, when launching a command via rez (ie rez env blah -- command args) on Windows there is no way to terminate the command properly (ie via KeyboardInterrupt), except by either force closing the launching terminal or killing the process via the OS, and all sorts of ways in which processes can continue to run in odd ways. I'd like to come as close as possible to replicating the "correct" behavior on Linux, but I don't know exactly how the processes in Linux react to, for instance, KeyboardInterrupt or the normal signals.

Overall findings below.

1) rez env python -- python test_rez.py
    launches the following processes:
        A: rez.exe (initial launch process)
        B: python.exe (subprocess launched by the rez.exe bootloader, uses rez's python copy)
        C: cmd.exe (subprocess launched by B; runs the temp rez-shell.bat to set up configured cmd, waits for D to exit) *
        D: python process launches (process launched in rez configured cmd, actually runs test_rez.py)
    attempting to kill via keyboard interrupt (CTRL+C):
        1) is not possible for process A
        2) is not possible for process B
        3) is not possible for process C
        4) is not possible for process D (keyboard input is ignored; if process D is killed then input is eventually echo'd in the launching shell)
    attempting to kill via Task Manager "end process":
        1) killing process A:
            i) kills process B immediately
            ii) user control drops back to launching shell
                - running "exit" on the shell kills the launching shell process, but the terminal remains open until C and D are killed
            iii) process C is orphaned and runs until killed 
                - killing C does not kill D
            iv) process D is orphaned and runs until killed
                - killing D kills C (and returns full control to the launching terminal)
        2) killing process B behaves exactly as per killing process A
        3) killing process C:
            i) kills process A immediately
            ii) kills process A immediately
            iii) user control drops back to launching shell
                - running "exit" on the shell kills the launching shell process, but the terminal remains open until D is killed
            iv. process D is orphaned and runs until killed
                - killing D returns full control the launching terminal
        4) killing process D:
            i) kills process A immediately
            ii) kills process B immediately
            iii) kills process C immediately
            iv) user control drops back to launching shell

2) rez env --detached python -- python test_rez.py
    launches the following processes:
        A) rez.exe (initial launch process)
        B) python.exe (subprocess launched by the rez.exe bootloader, uses rez's python copy)
        C) cmd.exe (subprocess launched by B; runs CMD.exe /Q /K to launch a nested CMD.exe, waits for D to exit)
        D) cmd.exe (subprocess launched by C; runs the temp rez-shell.bat to set up configured cmd, waits for E to exit) *
        E) python process launches (process launched in rez configured cmd, actually runs test_rez.py)
    attempting to kill via keyboard interrupt (CTRL+C):
        1) is not possible for process A
        2) is not possible for process B
        3) is not possible for process C
        4) is not possible for process D
        5) is not possible for process E (keyboard input is ignored; if process E is killed then input is eventually echo'd in the launching shell)
    attempting to kill via Task Manager "end process":
        1) killing process A:
            i) kills process B immediately
            ii) user control drops back to launching shell
                - running "exit" on the shell kills the launching shell process, but the terminal remains open until C, D, and E are killed
            iii) process C is orphaned and runs until killed 
                - killing C does not kill D or E (and further killing E will no longer kill D)
            iv) process D is orphaned and runs until killed 
                - killing D does not kill C or E
            v) process E is orphaned and runs until killed
                - killing E kills D, but leaves C orphaned; returns partial control to the launching terminal (terminal remains open until C is killed)
        2) killing process B behaves exactly as per killing process A
        3) killing process C:
            i) kills process A immediately
            ii) kills process B immediately
            iii) kills process C immediately
            iv) user control drops back to launching shell
                - running "exit" on the shell kills the launching shell process, but the terminal remains open until D, and E are killed
            v) process D is orphaned and runs until killed 
                - killing D does not kill E
            vi) process E is orphaned and runs until killed
                - killing E kills D, returns full control to the launching terminal
        4) killing process D:
            i) user control drops back the C subshell
                - running "exit" on the C subshell kills the A, B, and C processes and returns user control to the launching shell, however full control is lost until E is killed
            ii) process A is orphaned and runs until killed 
                - killing A kills B, but does not kill C or E (both of which can be killed without killing the other)
            iii) process B is orphaned and runs until killed 
                - killing B kills A, but does not kill C or E (both of which can be killed without killing the other)
            iv) process C is orphaned and runs until killed 
                - killing C kills A and B, but does not kill E
            v) process E is orphaned and runs until killed
                - killing E kills D, but does not kill A, B, or C; returns full control to the C terminal, which can then exit and kill A and B, which returns full control to the launching terminal
        5) killing process E:
            i) kills process D immediately
            ii) user control drops back the C subshell
                - running "exit" on the C subshell kills the A, B, and C processes and returns user control to the launching shell

3) rez env --new-session python -- python test_rez.py
    launches the following processes:
        A) rez.exe (initial launch process)
        B) python.exe (subprocess launched by the rez.exe bootloader, uses rez's python copy)
        C) cmd.exe (new terminal window launched by B; runs the temp rez-shell.bat to set up configured cmd, waits for D to exit) *
        D) python process launches (process launched in rez configured cmd, actually runs test_rez.py)
    attempting to kill via keyboard interrupt (CTRL+C):
        1) is not possible for process A
        2) is not possible for process B (keyboard input in the original terminal is ignored; if process E is killed then input is eventually echo'd in the launching shell)
        3) is not possible for process C
        4) is not possible for process D (keyboard input is ignored; if process E is killed then input is lost when C closes)
    attempting to kill via Task Manager "end process":
        1) killing process A:
            i) kills process B immediately
            ii) user control drops back to launching shell
                - running "exit" on the shell kills the launching shell process immediately, but the C terminal remains open until D is killed
            iii) process C is orphaned and runs until killed 
                - killing C does not kill D
            iv) process D is orphaned and runs until killed 
                - killing D kills process C (closing the new terminal window) and restores control to the launching terminal
        2) killing process B behaves exactly as per killing process A
        3) killing process C:
            i) kills process A immediately
            ii) kills process B immediately
            iii) in the original terminal returns user control back to launching shell
                - running "exit" on the shell kills the launching shell process immediately, but the C terminal remains open until D is killed
            iv) process D is orphaned and runs until killed
                - the new terminal window remains open, but unresponsive to input
                - killing D immediately closes the new terminal window
        4) killing process D:
            i) kills process A immediately
            ii) kills process B immediately
            ii) kills process C immediately
            iii) the new terminal window closes
            iv) in the original terminal returns full user control back to launching shell

4) rez env --detached --new-session python -- python test_rez.py
    launches the following processes:
        A) rez.exe (initial launch process)
        B) python.exe (subprocess launched by the rez.exe bootloader, uses rez's python copy)
        C) cmd.exe (new terminal window launched by B; runs CMD.exe /Q /K to launch a nested CMD.exe, waits for D to exit)
        D) cmd.exe (subprocess launched by C; runs the temp rez-shell.bat to set up configured cmd, waits for E to exit) *
        E) python process launches (process launched in rez configured cmd, actually runs test_rez.py)
    attempting to kill via keyboard interrupt (CTRL+C):
        1) is not possible for process A
        2) is not possible for process B (keyboard input in the original terminal is ignored; if process E is killed then input is eventually echo'd in the launching shell)
        3) is not possible for process C
        4) is not possible for process D
        5) is not possible for process E (keyboard input in the new terminal is ignored; if process E is killed then input is eventually echo'd in the C shell)
    attempting to kill via Task Manager "end process":
        1) killing process A:
            i) kills process B immediately
            ii) user control drops back to launching shell
                - running "exit" on the shell kills the launching shell process immediately, but the C terminal remains open until D and E are killed
            iii) process C is orphaned and runs until killed 
                - killing C does not kill D or E
                - further killing D does not kill E, but killing E does kill D
            iv) process D is orphaned and runs until killed 
                - killing D does not kill C or E
                - further killing E does not kill C, but does return control to the C process in the new terminal window
            v) process E is orphaned and runs until killed 
                - killing E kills process D and restores control to the C process in the new terminal window
        2) killing process B behaves exactly as per killing process A
        3) killing process C:
            i) kills process A immediately
            ii) kills process B immediately
            iii) in the original terminal returns user control back to launching shell
                - running "exit" on the shell kills the launching shell process immediately, but the C terminal remains open until D is killed
            iv) process D is orphaned and runs until killed
                - killing D does not kill E
            v) process E is orphaned and runs until killed 
                - killing E kills process D and immediately closes the new terminal window
        4) killing process D:
            i) user control drops back the C new terminal
                - running "exit" on the C subshell kills the A, B, and C processes and returns full control to the launching shell, but the C terminal remains open until E is killed
            ii) process A is orphaned and runs until killed 
                - killing A kills B, but does not kill C or E (both of which can be killed without killing the other)
            iii) process B is orphaned and runs until killed 
                - killing B kills A, but does not kill C or E (both of which can be killed without killing the other)
            iv) process C is orphaned and runs until killed 
                - killing C kills A and B, but does not kill E
                - in the original window full control is returned to the launching shell, but the C terminal remains open until E is killed
            v) process E is orphaned and runs until killed
                - killing E does not kill A, B, or C; returns full control to the C terminal, which can then exit and kill A and B, which returns full control to the launching terminal
        5) killing process E:
            i) kills process D immediately
            ii) user control drops back the C subshell
                - running "exit" on the C subshell kills the A, B, and C processes and returns user control to the launching shell

[instinct-vfx]

Woah, now that is a thorough investigation and write-up! Thanks a lot Michael! I am wondering if we can do some of that better and also wonder if going for powershell as an alternative shell plugin may give us a bit more control. We are moving parts of our shell script stuff to powershell anyways here and still want to look into that but have not yet found the time.

Cheers, Thorsten

[yawpitch]

Personally I find powershell, like pretty much anything that's ever come out of Microsoft, to be a brutally infuriating half-assed attempt to rip off something good, package it up in extra verbosity, find some godforsaken stupid reason to connect it to .net, and then act like they've improved the world ... but it's just been open sourced and is available on all platforms, it might be worth developing a plugin for.

That said the problem here isn't so much with Window's shell language, it's with the way processes are launched on Windows period, and the fact that I'm pretty sure we're not setting things up right in the stream of subprocess calls that Rez is using ... since powershell (for questionably material backwards compatibility reasons) inherits all of the same problems before it even launches I'm not sure an alternative plugin would help.

[yawpitch]

BTW, if you want to see what I mean with pshell ... take a look here and check out the "power" equivalent of mkdir and rm.

[yawpitch]

@nerdvegas : it's a bit convoluted, but based on the docs on rez/utils/platform_.Platform and given what's going on in rez/resolved_context.ResolvedContext.spawn_shell, it appears that:

1. the original intent of the --detached flag of rez env is to cause a wholly new terminal window to be launched, and that new window will then be configured with the resolved environment
2. the original intent of the --new-session flag of rez env appears to be to cause the resolved environment to open in the calling terminal window, but inside a new process group (I'm not sure of the value of this on Windows).
3. from that, I gather that the original intent of combining the --detached and --new-session flags would be to launch a new terminal window in a process that would live on if the original terminal were killed.

Is that all correct?

If so the current CMD plugin is not following those conventions; the --detached flag is launching a subshell in a subshell, end result being a process that's still in the same terminal window, albeit at one step removed from the calling shell. The --new-session is actually launching a new terminal window, though in such a manner that it leaves the original terminal locked and effectively useless.

Really wish I had a readily accessible Linux box to check against. Regardless I'm about to make a pull request that at least reverts to the "/K" behavior that this Issue is related to, but I do think the Windows CMD plugin's going to need some more work to be ready for production use.

[bfloch]

I had pretty much the same issues and made the same change. I remember that I tried power shell but it would not cure the issue in my attempts. Don't quote me on this, though - I usually make windows work and hope to never have to boot it up again :)

Do you guys think this is related to cmd or a windows issue? I would be willing to deploy an alternative shell if it fixes the rez issues we are having. There are a couple to choose from: http://stackoverflow.com/questions/17349022/alternative-windows-shells-besides-cmd-exe

[yawpitch]

I'll be honest, I think it very well may have as much to do with Python's subprocess and what it does (and does not) do with calls to Window's underlying CreateProcess (and CreateProcessAsUser, CreateProcessWithLogonW, and CreateProcessWithTokenW) function(s) as it does with anything else.

If you look closely at the Python source it's using just a limited subset of the flags that Windows will accept, which is why you cannot pass it all the Windows StartupInfo flags, etc. And Rez is passing a subset of that subset.

Unfortunately properly launching a new process in Windows is not trivial, and doing it in a manner where you expect it to have entirely similar behavior to Linux is, at best, Hard. I am quite certain Rez isn't doing it in the "correct" manner, but without a clearer understanding of exactly what behavior we're trying to create with each of the various flag permutations, I also don't know precisely what to do to fix it.

I don't think you'll find any other terminal emulator on Windows working better for you ... they're all, in the end, using one or the other of the CreateProcess functions, with all the inherent limitations (ie no forking) that are part of the underlying OS.

[instinct-vfx]

I agree on pretty much everything. I am pretty sure that the windows part is doing quite a lot of things in non-ideal ways but i am also unsure about the "desired" behaviour. I reverted the change to /K here as it seemed to behave more consistent for us and the wrappers/batch files etc. we have in place. I am pretty confident that the IT guys here could give a lot of input on how to properly achieve certain behavior (and Michael also seems to be very knowledgable there). So once Alan is back i wonder if it would be a good idea to take your "analytical" Matrix above, put it into the docs to document the different expected behaviors so we can start working on the windows side of things.

I regards to powershell i do agree basically, but there are quite some things you can do that you can not do in cmd. And while it may not be needed, it may help in some places.

Also some behavior really gave us a headache here when working towards a productive wrapper setup with a fully centralized install. e.g. that the cmd plugin pulls PATH from the registry which makes it impossible to hack up things in wrappers and pre-built shells. (Besides the implementation that uses subprocess and result parsing to get the registry value instead of _winreg).

We'd really love to contribute to this effort here!

[yawpitch]

We* would like to contribute as well, we're using Rez inside a contextual configuration tool we've written to even discover what bloody package (never mind dependencies) are required on which production, and we're stuck on Windows (for worse or even worse), and we've got it working fine for direct command execution, but the launching of new persistent configured shells is really in need of improvement.

Personally I've got a lot of interest in bootstrapping something like xonsh , as that would make a pretty much ideal shell that's already cross-platform (it's a wee bit opinionated, but the kid who wrote it seems to be one of those terrifying geniuses you always hope aren't stroking cats) and almost perfect for VFX work. But, honestly, it might make more sense to write a Rez "xontrib" (a xonsh plugin) that uses Rez for the configuration resolution and hands off all the process-management portion to something that's a bit deeper in that area.

* well I say we, but the fact is I'm leaving in a month and will do everything possible to never look at a Windows CMD.exe again in this lifetime.

[instinct-vfx]

Just took a look at xonsh. Seems really interesting. Do you have any experience with it? As it just taps on top of cmd or other replacement shells it still faces all of the underlying issues though, or does it circumvent these with its own calls?

Cheers, Thorsten

[yawpitch]

Well, xonsh actually is a shell ... CMD.exe has always been a bit weird -- well, I say a bit, but basically we're talking something created by a manifestly evil human being -- in that it's both shell and a terminal emulator. But yes it'll always run into the underlying issues of the OS itself (ie true no process forking, environment variables being superseded by that monstrosity of a shitshow database they call the registry) and have to either a) accept them (process forks), or b) work around them (ie bypass the registry entirely).

I do think that xonsh looks pretty promising, though a) it's slow (shells, like operating systems, fundamentally shouldn't be written in something like Python), b) it's pretty opinionated, and c) it's young. There are other alternative, cross-platform shells out there as well ... and somewhere along the line someone will bootstrap one with sequential file tools and a SAP solver like Rez, and that'll be all she wrote for VFX. Well, until someone figures out that your renders are NOT your assets, that your assets are actually your artist's session file + the compute environment + the dependency files, and comes up with a good way of making, storing, and indexing Nuke- and Maya- encasing full-compute containers.

So, short answer, yes it will still have some serious issues, the big advantage would be not under-the-hood but outside it ... you'd have one common shell environment for VFX work on all platforms.

[alexxbb]

Hey. Trying Rez for the first time and being not able to get resolved subshell on windows found this discussion. Is there anything could be done to overcome this problem?

[yawpitch]

Try the Pixomondo/rez master branch (the one in this pull request). That seems to be giving equivalent behavior on Windows as in Linux, after much experimentation.

On Nov 6, 2016, 09:55 -0800, alexxbb notifications@github.com, wrote:

Hey. Trying Rez for the first time and being not able to get resolved subshell on windows found this discussion. Is there anything could be done to overcome this problem?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub (https://github.com/nerdvegas/rez/issues/336#issuecomment-258697729), or mute the thread (https://github.com/notifications/unsubscribe-auth/AAnEpyFgkXhbDi-I5odArzglPbBuUxboks5q7hSWgaJpZM4JbYFf).

[alexxbb]

Will give it a try, thanks!

[mgcollie]

Can confirm that the Pixomondo/rez branch seems to be giving equivalent behavior on Windows.

[nerdvegas]

Hey all,

So while others are presumably looking into a fix (!), I will prioritize the following work:

• Update to env var management, so that original pre-rez values are kept intact;
• Add a --source option to rez-env.

The second point is to provide a rez-env mode that just prints the env setup code (ie bash/cmd/other) to stdout. This would give users more options as to how they want to go about creating a subshell, if at all - you could source this code in order to put the current shell into a resolved env, for example.

A

On Sun, Feb 19, 2017 at 12:39 PM, mgcollie notifications@github.com wrote:

Can confirm that the Pixomondo/rez branch seems to be giving equivalent behavior on Windows.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nerdvegas/rez/issues/336#issuecomment-280888856, or mute the thread https://github.com/notifications/unsubscribe-auth/ABjqSsIfjUqSH7T8lgB5J-lXT8sKoMqvks5rd51YgaJpZM4JbYFf .

[KelSolaar]

Hi,

I'm making a serious dive in Rez and came across that issue (among a few others), it was disconcerting to say the least as I was expecting rez-env python to load the resolved environment in the current shell.

What I'm doing right now is the following: rez-env python -- cmd which does reuse the current shell (but loose history).

OT: Would be fantastic to expand / update the documentation: I'm seeing myself doing a lot of introspection to understand why some things that do work on Mac don't on Windows, or the behaviour is different / unexpected :)

Cheers,

Thomas

[nerdvegas]

Hey Thomas,

None of the rez commands modify the current shell - even running a command as above runs in a subshell. This is by design. Happy to take a PR on the docs if they aren't obvious enough (note that the main docs are at the repo rez.wiki.git).

I would not be adverse to adding an --eval/other flag to rez-env which would just print the script that you'd need to source to update the current shell. Then you could just provide your own command (alias/bash function) to wrap that. There are a couple of things to note however.. rez stores the current context in a tmpdir which gets cleaned up on shell exit, but since there'd be no child shell here, we'd have to have configurable behavior (probably on cli) to determine whether you still want the tmpdir to be created (and its up to you to clean it up), or if you want to forgo the tempdir, at the expense of less info on the current context being stored.

Having an --eval flag would also help on Windows since that would skip the complexities of creating the subshell, and leave it to the user to solve as they see fit. We're looking into this as we speak.

Thx A

On Tue, Feb 21, 2017 at 3:29 PM, Thomas Mansencal notifications@github.com wrote:

Hi,

I'm making a serious dive in Rez and came across that issue (among a few others), it was disconcerting to say the least as I was expecting rez-env python to load the resolved environment in the current shell.

What I'm doing right now is the following: rez-env python -- cmd which does reuse the current shell (but loose history).

OT: Would be fantastic to expand / update the documentation: I'm seeing myself doing a lot of introspection to understand why some things that do work on Mac don't on Windows, or the behaviour is different / unexpected :)

Cheers,

Thomas

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nerdvegas/rez/issues/336#issuecomment-281242665, or mute the thread https://github.com/notifications/unsubscribe-auth/ABjqSmrBgVe_OJEgbO0keNXrEBvM1vOKks5remhAgaJpZM4JbYFf .

[KelSolaar]

None of the rez commands modify the current shell - even running a command as above runs in a subshell. This is by design.

Dully noted :)

I would not be adverse to adding an --eval/other flag to rez-env which would just print the script that you'd need to source to update the current shell. Then you could just provide your own command (alias/bash function) to wrap that.

That would be very useful, I'm working on Windows these days and it makes simple things interesting to say the least :)

I'm still trying to figure out how best to handle some user expectations, they like the comfort of being able to launch Maya by double clicking a file in the explorer, etc... :D

We're looking into this as we speak.

Excellent!

Bests,

Thomas

[yawpitch]

You're going to find it VERY hard to give people a "normal" Windows experience in a contextual environment like VFX, where that shortcut they're used to points to an entirely different version of Maya or say the same version with a entirely different set of plug-ins. What you ultimately need is the "context manager" that's mentioned in the Rez documents but that Rez doesn't provide itself... you then have to provide shortcuts on the desktop of the workstation image you're giving to artists and have those shortcuts actually call the context manager -- which for instance would provide a GUI where the user selects the context they're working in -- and then that actually runs rez to get the package configuration resolve and prepare the shell environment through which you'd actually launch Maya.

Due to fundamental limitations of CMD this is really hard; Powershell is better, but using it this way is also getting into an esoteric space (since Powershell ultimately still can't give you process forking) and so is trying to do it all via Cygwin or other "bash-light" options. There's just an enormous number of little places where something that "just works" in Linux either cannot be done, or cannot be done and not lose say history or tab completion or the gods know what else, in Windows.

The Shotgun Desktop model tries to provide that context manager AND a desktop style launcher GUI, so really there's just a single access point and you don't have to worry about curating desktop shortcuts for people, but that comes with the caveat of a) a lot of vendor lockin and b) the extremely simplistic context model that Shotgun employs. Plus, in the end, all it really does is sets up the application launched to know about Shotgun integration, it doesn't seem to give you much of a hook into providing context for other Shotgun-adjacent-but-still-external problems (say asset management or farm submission).

Honestly I'd say worrying about getting the Rez-resolved env to effect the current CMD shell process (you'd have to invoke "call" in your current shell on a .bat file containing the Rez env commands, ensuring that it didn't use setlocal/endolocal and didn't itself use further "call" or "start" invocations) is the least of the problem. The work we did at Pixomondo before I left got rez env calls on Windows most of the way there -- though we never did get a truly precise accounting of exactly what each possible invocation of "rez env" was supposed to result in on Linux and thus what the expected outcome on Windows should be -- but that's fine tuning compared to figuring out the user experience you want your artists to have and the shim in the middle, they currently doesn't exist, to deliver that to them using Rez.

Two bits from halfway round the world.

M

On Feb 22, 2017, 02:32 +0530, Thomas Mansencal notifications@github.com, wrote:

None of the rez commands modify the current shell - even running a command as above runs in a subshell. This is by design.

Dully noted :)

I would not be adverse to adding an --eval/other flag to rez-env which would just print the script that you'd need to source to update the current shell. Then you could just provide your own command (alias/bash function) to wrap that.

That would be very useful, I'm working on Windows these days and it makes simple things interesting to say the least :)

I'm still trying to figure out how best to handle some user expectations, they like the comfort of being able to launch Maya by double clicking a file in the explorer, etc... :D

We're looking into this as we speak.

Excellent!

Bests,

Thomas

— You are receiving this because you commented. Reply to this email directly, view it on GitHub (https://github.com/nerdvegas/rez/issues/336#issuecomment-281479179), or mute the thread (https://github.com/notifications/unsubscribe-auth/AAnEp9oPOOcDQH01ffEeCFRP4hgI4RG6ks5re1DhgaJpZM4JbYFf).

[KelSolaar]

@michaelmovies : From initial tests rez-gui is pretty much that context resolver, it is too complicated for an artist but I can see something leveraging the contexts it generates or a similar tool in hands of artists.

There is still the question of what to do when somebody double-click a random maya scene in a project directory, we have some thoughts about handling that: the file association is made toward a dedicated wrapper that could then figure out which context to apply depending the location of the file in the directory tree.

Shotgun Desktop is interesting, thanks for mentioning it!

By the way, are you the former LBG / Weta Michael?

[yawpitch]

Former Weta (Prodeng) yes, and assume you mean Little Baby Jesus (LBJ) where I was at time far too talkative? If so, yep, me.

Ok, so you've worked at Weta, so we've got a similar basis: rez-gui really isn't the context resolver, exactly ... at least not in the way oz was. It might be enough for a small facility with a simple project hierarchy, but it doesn't really solve the long term problem, a problem that is made much worse on Windows.

The problem is basically this: what context am I working in? The context determines what "gross" packages are required, and the gross packages determine (via dependency linking) what additional "fine" packages are required. You then need to run a SAT solver to determine what is the actual minimal combination of packages that satisfy those requirements (if one is even available).

Now a context can be thought of as a vertical hierarchy, and the simplest one we all think of in VFX is SHOW>SEQ>SHOT, but we all very quickly end up tripping over the fact that there's also SHOW>ASSET_TYPE>ASSET_CLASS and even a hell of a lot of contexts that aren't even in a SHOW to begin with. This is the great trap of VFX pipelining.

In actual truth a context really is a vertical hierarchy with horizontal overrides at each vertical level, and these horizontal overrides have an order of application that is relevant.

So a in a given SHOW each department may have its own package setup. Each task type within that department might need to further refine it. And each artist might even want individual run-time overrides ... as far as I can see Rez's GUI basically handles the vertical hierarchy (by walking a list of file paths) and it handles user level overrides (on individual Rez calls in the CLI that use a context), but it really doesn't handle the other ideas. Nor, frankly, should it.

I think Rez is very correctly aimed at the package resolution domain, and I am quite certain (having worked at way too many of these places now) that context management MUST be a database. It might start as a "disk db" of xml files ... anyone who tries yaml or worse json files first will run into so much with the heaps of trouble eventually (schemas and a tree DOM are exactly what is required here), but in the mid to long run this is exactly what a server sitting somewhere with a defined API is for ... context resolution needs to be fast AND it needs to be versioned, package resolution can be slow and done on the artist's machine's time (though I'd still make the SAT solver a service as well if I could get everyone to stop fearing http).

Your double click example is an interesting tangent (storing context breadcrumbs in the file path is trap 2 that VFX companies inevitably fall into), but it's also best resolved via a database, albeit an asset management one ... on file creation you stamp the context metadata into the asset management db, someone double clicks the file and you use the association to run a hash of the file, deriving the key (say c4id for argument's sake, but md5 or sha256 would be fine), then ask the asset db for the contextual metadata for that key, which you then feed to the context manager API and on to the SAT solver. No asset DB? Ugh, then fall back on breadcrumbs in the path or a sidecar file.

Ultimately my best advice is think of it in stages and use the right tool for each stage; context begets package needs, packages needs begets sat solve, sat solve gives us a resolved answer to the question of a specific question in a specific context until the context itself changes again, which means we might want to store that somewhere and re-use it, which gets you thinking about your context as time-based assets of the company, and round and round the loop she goes.

I swear I've been thinking of this stuff for way too long. Don't even get me started on how something like a Docker Hub gets you re-useable "compiled" runtime environments that could be used in the future to run a scene file in precisely the same -- but unlike with a sat solve from installed packages effectively immutable -- environment that it was initially created in.

Where you at that you've suddenly got this problem on Winblows?

M

On Feb 23, 2017, 10:31 +0530, Thomas Mansencal notifications@github.com, wrote:

@michaelmovies (https://github.com/michaelmovies) : From initial tests rez-gui is pretty much that context resolver, it is too complicated for an artist but I can see something leveraging the contexts it generates or a similar tool in hands of artists.

There is still the question of what to do when somebody double-click a random maya scene in a project directory, we have some thoughts about handling that: the file association is made toward a dedicated wrapper that could then figure out which context to apply depending the location of the file in the directory tree.

Shotgun Desktop is interesting, thanks for mentioning it!

By the way, are you the former LBG / Weta Michael?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub (https://github.com/nerdvegas/rez/issues/336#issuecomment-281895180), or mute the thread (https://github.com/notifications/unsubscribe-auth/AAnEpwidT_TfAl4LAj61H7aydAlsxvzeks5rfRKRgaJpZM4JbYFf).

[yawpitch]

Btw, that hash discussion tangent there assumes you're doing "version on save" where you're putting committed changes in an immutable, read-only area, and artists load from that area to begin work on the next version... as that's the only way file contents remain a viable key into the asset db.

On Feb 23, 2017, 10:31 +0530, Thomas Mansencal notifications@github.com, wrote:

@michaelmovies (https://github.com/michaelmovies) : From initial tests rez-gui is pretty much that context resolver, it is too complicated for an artist but I can see something leveraging the contexts it generates or a similar tool in hands of artists.

There is still the question of what to do when somebody double-click a random maya scene in a project directory, we have some thoughts about handling that: the file association is made toward a dedicated wrapper that could then figure out which context to apply depending the location of the file in the directory tree.

Shotgun Desktop is interesting, thanks for mentioning it!

By the way, are you the former LBG / Weta Michael?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub (https://github.com/nerdvegas/rez/issues/336#issuecomment-281895180), or mute the thread (https://github.com/notifications/unsubscribe-auth/AAnEpwidT_TfAl4LAj61H7aydAlsxvzeks5rfRKRgaJpZM4JbYFf).

[jasoncscott]

Interesting ideas, but I think I fundamentally disagree with the need to do it (not arguing with the execution). I feel like in a production environment, if you're storing the context per file, and then re-opening that file using that exact context, you're ignoring the production needs of updating the suite of tools that the user should be using to open that file. Keeping track of it for historical reasons is beneficial, but I would want to force all my users by default to use the controlled environment I provide (whether "I" = DFX, pipeline supervisor, etc.), not the one they used at the time they saved the file.

Outside of that, we (on Linux) just don't even provide most of the applications to open files from the desktop environment, requiring users to run a few shell commands first that get us into a ballpark of contexts. We only have a few wrappers that do that, and yes, they parse the path, but I totally agree that's a dangerous route to go down (its development predates me here).

Jason Scott

On Thu, Feb 23, 2017 at 2:18 AM, Michael Morehouse <notifications@github.com

wrote:

Btw, that hash discussion tangent there assumes you're doing "version on save" where you're putting committed changes in an immutable, read-only area, and artists load from that area to begin work on the next version... as that's the only way file contents remain a viable key into the asset db.

On Feb 23, 2017, 10:31 +0530, Thomas Mansencal notifications@github.com, wrote:

@michaelmovies (https://github.com/michaelmovies) : From initial tests rez-gui is pretty much that context resolver, it is too complicated for an artist but I can see something leveraging the contexts it generates or a similar tool in hands of artists.

There is still the question of what to do when somebody double-click a random maya scene in a project directory, we have some thoughts about handling that: the file association is made toward a dedicated wrapper that could then figure out which context to apply depending the location of the file in the directory tree.

Shotgun Desktop is interesting, thanks for mentioning it!

By the way, are you the former LBG / Weta Michael?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub ( https://github.com/nerdvegas/rez/issues/336#issuecomment-281895180), or mute the thread (https://github.com/notifications/unsubscribe- auth/AAnEpwidT_TfAl4LAj61H7aydAlsxvzeks5rfRKRgaJpZM4JbYFf).

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/nerdvegas/rez/issues/336#issuecomment-281952647, or mute the thread https://github.com/notifications/unsubscribe-auth/AMjo9w4rAanlV4DCJpzUGzDoyQ9U7d7mks5rfVz6gaJpZM4JbYFf .

[yawpitch]

I'm not suggesting that you force the user to open only in the previous version of the context; if the current "official" context is compatible, use the -- presumably but no necessarily higher -- versions in that, if the current is incompatible, warn the user of the discrepancies and or environment breaking issues. The context saved in the asset db / sidecar file should always be the actual complete context used at file "publish" (i.e. it should allow you to recreate the exact environment and fully resolved package set that was used to create the file, with all dependencies, manual overrides, etc accounted for). The problem with parsing the file path is you can only get vertical hierarchy encoded into that (otherwise you'd need query parameter, like in a URL), so without the sidecar or a db you're, at best, taking a guess as to the context (sans department, task, user, ... overrides) and maybe, if you're lucky, using the file creation time to do a Rez solve that at least doesn't include packages installed after that point.

My goal is always to empower the artists, never to hem them in ... what I do want to do is remove from their thinking things that are really just terrible habits we learned before computers could talk to each other, like mnemonic directory structures in a world where (if you're on Windows) you STILL can't have a reliable $PATH of > 2048 characters. An artist in a well-designed pipeline doesn't even consider where to save a file, their context determines that. They don't even consider whether or not to put it in a database, their pipeline ubiquitously does that for them behind the scenes... artists are more expensive than machines and their job is to create art, not directory structures or well formed file paths, both of which are things machines are ridiculously good at. So ultimately anything I suggest is from too many years of watching artist's lives and talent be wasted on automatable esoterica. My problem is that our industry has voluntarily NOT kept up with the rest of the world, generally out of a misguided fear of networking and databases, and that it's time we woke up and caught up; in a world where Dropbox is seamlessly versioning every file as you and/or others edit it collaboratively in real time, we should have this problem solved.

Anyway, don't want to digress too far, especially on a Github issue that I can't really contribute to at the moment. I chimed in because Windows is a very hard nut to crack in terms of giving people the contextually pipelined experience, due to a) a very ingrained model of "click on the damned thing to open in the appropriate program" that doesn't actually line up with how someone working on different projects with different needs from the same underlying tools actually works, b) a historically "what sort of idiot needs a terminal" approach from Redmonds, and c) fundamental limitations with CMD (many of which aren't resolved by either Powershell or Cygwin, though they can help make the UX prettier, and some of which cannot be resolved -- I see you, process forking) and that getting stuck in a child shell and having to exit rather than updating the running shell is perhaps a red herring you might not want to chase too far.

And, sidebar, yes I MASSIVELY prefer what has become the "Linux model", and my mantra for years has been teach the artists the command line, because a) they're not stupid and b) it will make them MUCH more powerful (hence higher paid). Unfortunately there are a fair few business (and IT) types in the smaller shops who can't wrap their head around not paying for an OS.

My four or so bita,

M

On Feb 23, 2017, 18:02 +0530, Jason Scott notifications@github.com, wrote:

Interesting ideas, but I think I fundamentally disagree with the need to do it (not arguing with the execution). I feel like in a production environment, if you're storing the context per file, and then re-opening that file using that exact context, you're ignoring the production needs of updating the suite of tools that the user should be using to open that file. Keeping track of it for historical reasons is beneficial, but I would want to force all my users by default to use the controlled environment I provide (whether "I" = DFX, pipeline supervisor, etc.), not the one they used at the time they saved the file.

Outside of that, we (on Linux) just don't even provide most of the applications to open files from the desktop environment, requiring users to run a few shell commands first that get us into a ballpark of contexts. We only have a few wrappers that do that, and yes, they parse the path, but I totally agree that's a dangerous route to go down (its development predates me here).

Jason Scott

On Thu, Feb 23, 2017 at 2:18 AM, Michael Morehouse <notifications@github.com

wrote:

Btw, that hash discussion tangent there assumes you're doing "version on save" where you're putting committed changes in an immutable, read-only area, and artists load from that area to begin work on the next version... as that's the only way file contents remain a viable key into the asset db.

On Feb 23, 2017, 10:31 +0530, Thomas Mansencal notifications@github.com, wrote:

@michaelmovies (https://github.com/michaelmovies) : From initial tests rez-gui is pretty much that context resolver, it is too complicated for an artist but I can see something leveraging the contexts it generates or a similar tool in hands of artists.

There is still the question of what to do when somebody double-click a random maya scene in a project directory, we have some thoughts about handling that: the file association is made toward a dedicated wrapper that could then figure out which context to apply depending the location of the file in the directory tree.

Shotgun Desktop is interesting, thanks for mentioning it!

By the way, are you the former LBG / Weta Michael?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub ( https://github.com/nerdvegas/rez/issues/336#issuecomment-281895180), or mute the thread (https://github.com/notifications/unsubscribe- auth/AAnEpwidT_TfAl4LAj61H7aydAlsxvzeks5rfRKRgaJpZM4JbYFf).

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/nerdvegas/rez/issues/336#issuecomment-281952647, or mute the thread https://github.com/notifications/unsubscribe-auth/AMjo9w4rAanlV4DCJpzUGzDoyQ9U7d7mks5rfVz6gaJpZM4JbYFf .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub (https://github.com/nerdvegas/rez/issues/336#issuecomment-281979956), or mute the thread (https://github.com/notifications/unsubscribe-auth/AAnEp79yoPtJ-QL-_CFY82ro3-FsSIo4ks5rfXxOgaJpZM4JbYFf).

[nerdvegas]

Hey all,

So it may help to give a bit of an overview of how we manage production environments at Method, and talk to some of the environment immutability that Michael talks about.

First, Michael is right - this is not rez's domain, and that is by design. Rez doesn't know about shows or shots, or about a production hierarchy - it only knows how to configure environments and build packages. That isn't to say that rez wouldn't benefit a lot from being used in conjunction with another project that does do these things... unfortunately my own endeavors there have only ever been closed source.

We have an approach at Method that gives us anything between an unchanging, immutable env, all the way to an env where a tool launch gets the latest and greatest of everything, depending on how it's configured.

We have 'profiles', which are yaml-based configs, that live on the filesystem in a job/seq/shot/task hierarchy (profiles further down are merged/overridden with those above). One profile defines the env that a tool or group of tools run in, so typically you'll have one per major DCC, at least (in practice you have profiles for your own major apps as well).

All the tools available after 'setshotting' into the system, are actually wrappers that are aware of these profiles. When the tool is launched, the resolve occurs and the env is dynamically created, then the target tool is started within that shell. This would be pretty expensive (with every tool doing a resolve) except that the rez caching system comes into play here, and typically has a 98-99% hit rate. So new resolves actually don't occur often.

The profiles define what package versions/ranges we want, but that still leaves us exposed to getting the newest versions within the requested version ranges ("foo-1" might get "foo-1.3.0" one moment, and "foo-1.4.0" the next). This doesn't help if we want more stability than that. This is where "package locks" come in. A package lock is a timestamp (also stored into a config file) that basically says "don't use package versions released after this time, unless the resolve demands it." I'll skip the details of how that actually works but suffice to say it is this feature that gives us both stability and the ability to update to a newer package version explicitly (in a profile) if need be.

All these config files (profiles, locks) are also version controlled. So we also have the ability to roll any environment back to exactly its state at some point in the past.

All of the above would indeed work even better if turned into a service and db-backed. We didn't go the db route initially though, because you basically get source control for free from git if you do things file-based. I could see how a duel git+db approach might work well though. Rez as a service is a separate but intriguing topic - certainly a performance benefit is to be had going down that route.

Hth A

On Fri, Feb 24, 2017 at 2:07 AM, Michael Morehouse <notifications@github.com

wrote:

I'm not suggesting that you force the user to open only in the previous version of the context; if the current "official" context is compatible, use the -- presumably but no necessarily higher -- versions in that, if the current is incompatible, warn the user of the discrepancies and or environment breaking issues. The context saved in the asset db / sidecar file should always be the actual complete context used at file "publish" (i.e. it should allow you to recreate the exact environment and fully resolved package set that was used to create the file, with all dependencies, manual overrides, etc accounted for). The problem with parsing the file path is you can only get vertical hierarchy encoded into that (otherwise you'd need query parameter, like in a URL), so without the sidecar or a db you're, at best, taking a guess as to the context (sans department, task, user, ... overrides) and maybe, if you're lucky, using the file creation time to do a Rez solve that at least doesn't include packages installed after that point.

My goal is always to empower the artists, never to hem them in ... what I do want to do is remove from their thinking things that are really just terrible habits we learned before computers could talk to each other, like mnemonic directory structures in a world where (if you're on Windows) you STILL can't have a reliable $PATH of > 2048 characters. An artist in a well-designed pipeline doesn't even consider where to save a file, their context determines that. They don't even consider whether or not to put it in a database, their pipeline ubiquitously does that for them behind the scenes... artists are more expensive than machines and their job is to create art, not directory structures or well formed file paths, both of which are things machines are ridiculously good at. So ultimately anything I suggest is from too many years of watching artist's lives and talent be wasted on automatable esoterica. My problem is that our industry has voluntarily NOT kept up with the rest of the world, generally out of a misguided fear of networking and databases, and that it's time we woke up and caught up; in a world where Dropbox is seamlessly versioning every file as you and/or others edit it collaboratively in real time, we should have this problem solved.

Anyway, don't want to digress too far, especially on a Github issue that I can't really contribute to at the moment. I chimed in because Windows is a very hard nut to crack in terms of giving people the contextually pipelined experience, due to a) a very ingrained model of "click on the damned thing to open in the appropriate program" that doesn't actually line up with how someone working on different projects with different needs from the same underlying tools actually works, b) a historically "what sort of idiot needs a terminal" approach from Redmonds, and c) fundamental limitations with CMD (many of which aren't resolved by either Powershell or Cygwin, though they can help make the UX prettier, and some of which cannot be resolved -- I see you, process forking) and that getting stuck in a child shell and having to exit rather than updating the running shell is perhaps a red herring you might not want to chase too far.

And, sidebar, yes I MASSIVELY prefer what has become the "Linux model", and my mantra for years has been teach the artists the command line, because a) they're not stupid and b) it will make them MUCH more powerful (hence higher paid). Unfortunately there are a fair few business (and IT) types in the smaller shops who can't wrap their head around not paying for an OS.

My four or so bita,

M

On Feb 23, 2017, 18:02 +0530, Jason Scott notifications@github.com, wrote:

Interesting ideas, but I think I fundamentally disagree with the need to do it (not arguing with the execution). I feel like in a production environment, if you're storing the context per file, and then re-opening that file using that exact context, you're ignoring the production needs of updating the suite of tools that the user should be using to open that file. Keeping track of it for historical reasons is beneficial, but I would want to force all my users by default to use the controlled environment I provide (whether "I" = DFX, pipeline supervisor, etc.), not the one they used at the time they saved the file.

Outside of that, we (on Linux) just don't even provide most of the applications to open files from the desktop environment, requiring users to run a few shell commands first that get us into a ballpark of contexts. We only have a few wrappers that do that, and yes, they parse the path, but I totally agree that's a dangerous route to go down (its development predates me here).

Jason Scott

On Thu, Feb 23, 2017 at 2:18 AM, Michael Morehouse < notifications@github.com

wrote:

Btw, that hash discussion tangent there assumes you're doing "version on save" where you're putting committed changes in an immutable, read-only area, and artists load from that area to begin work on the next version... as that's the only way file contents remain a viable key into the asset db.

On Feb 23, 2017, 10:31 +0530, Thomas Mansencal < notifications@github.com>, wrote:

@michaelmovies (https://github.com/michaelmovies) : From initial tests rez-gui is pretty much that context resolver, it is too complicated for an artist but I can see something leveraging the contexts it generates or a similar tool in hands of artists.

There is still the question of what to do when somebody double-click a random maya scene in a project directory, we have some thoughts about handling that: the file association is made toward a dedicated wrapper that could then figure out which context to apply depending the location of the file in the directory tree.

Shotgun Desktop is interesting, thanks for mentioning it!

By the way, are you the former LBG / Weta Michael?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub ( https://github.com/nerdvegas/rez/issues/336#issuecomment-281895180), or mute the thread (https://github.com/notifications/unsubscribe- auth/AAnEpwidT_TfAl4LAj61H7aydAlsxvzeks5rfRKRgaJpZM4JbYFf).

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/nerdvegas/rez/issues/336#issuecomment-281952647, or mute the thread https://github.com/notifications/unsubscribe-auth/ AMjo9w4rAanlV4DCJpzUGzDoyQ9U7d7mks5rfVz6gaJpZM4JbYFf .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub ( https://github.com/nerdvegas/rez/issues/336#issuecomment-281979956), or mute the thread (https://github.com/notifications/unsubscribe- auth/AAnEp79yoPtJ-QL-_CFY82ro3-FsSIo4ks5rfXxOgaJpZM4JbYFf).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nerdvegas/rez/issues/336#issuecomment-282017521, or mute the thread https://github.com/notifications/unsubscribe-auth/ABjqSukZy_RIf-gZ0Etmfm-BfyCR61k3ks5rfaDGgaJpZM4JbYFf .

[yawpitch]

This isn't a dig at Method, or any shop, as I know as well as anyone that this is a core problem of VFX and the workloads that are placed on all of the companies big and small within it, but that isn't "managing" context, that's managing not to always be in a broken context.

That roll up of yaml profiles is pretty much what's been developed, separately, at every facility I've worked at... the industry really does like spinning its wheels on the same problem. Larger companies have tended towards XML, but the idea is fundamentally the same. The problems with it range from the general to the specific:

1) you're immediately buying into the idea that putting extra load on file reads is a good idea ... every time an artist (or a pipeline process, or a farm process) does a "set context" you have to do the roll up again, which means walking a disk hierarchy, stat for file existence, read file...

2) you're again limited to a context that is effectively vertical only ... lower contextual levels override higher ones -- i.e. if you're in this shot, use maya 2014 + [plugins] -- but there's no concept of horizontal overrides -- i.e. if you're in this shot, use maya 2014 + [plugins] unless you're in FX, in which case use maya 2016 + [plugins] + [fx plugins]).

3) on top of your grinding on file reads, you're also limited by file interpretation and validation: xml is fast across all OS in python, thanks to the c extension, but is complex and most programmers and especially anytime editing it manually hate it, while yaml, in Python, is EXTREMELY slow without libyaml (which fails to compile out of the box on Windows, and seems to be an abandoned project that hasn't been updated in years) ... also neither YAML or JSON has a scheme mechanism, which makes writing anything more than a linter that validates these files after the fact really hard or at least very error prone.

4) anyone who can edit any of those files directly can break anything from a whole shot to a whole company if they introduce a syntax error. Or forget that "true" and "True" in YAML are entirely different primitive types, or a large number of other esoteric rules. And what even is the right thing to do if someone chmods a directory recusrvidsly? Does what was your time-locked context yesterday just lose all that granularity silently? If so, what was the point of the time locking again?

5) context ends up getting split over a large number of files (i.e. per DCC app and per internal app, as you describe, but also site wide, show wide, etc common values) which makes rollup quite complex. You can't just use Chainmap in Python, because that doesn't work with nested dicts and lists, and how do you resolve (if at all) someone setting a particular key's value as an Integer at the facility level but a float (or a boolean, or a string) at lower level?

6) writing and editing contexts becomes VERY hard as your company's complexity grows... if you've got 13 simultaneously running massive feature film projects, each one offset from the others by potentially 2-3 entire generations of DCC app releases, how does someone get a high level overview of the entire context, drill down and make a change, and then commit that change out into production? More importantly, how do they know that a high level change they're thinking of making will break half those productions? Or at least require them to be locked off? Does that mean a notification mechanism to allow other stakeholders to opt their own production into or out of that change? Now you need an entire testing and delivery mechanism for context DATA, nevermind just context code... and anyone reading this knows just how abysmally bad and under-skilled we are at testing, nevermind CI and CD in the VFX world.

7) though git does give you version control over context -- congratulations, you're a seriously large step ahead from quite a few places I've been at or known about -- git is a TERRIBLE delivery mechanism for keeping multiple sites in sync. You either need to run a "git reset --HARD && git pull" on every configuration file at every depth in the contextual hierarchy before ever reading it, or you need to keep all your sites informed of the exact hash they should have locally for each repo or your one, no doubt unwieldy, monolith repo.

I can go on and on, but basically what I'm saying is that context file rollup is like generation 2 of VFX configuration management, one step above a single static configuration, and this is basically exactly the problem that the rest of the world solves with a database and a well-formed API. Your ability to scale should not be locked to file IO performance, nor should you be forced to walk down the path of ad hoc caching of files with notification invalidation by file change watching daemon processes, especially when the solutions to the problem, broadly, already exist and have been battle-tested for a decade or more at scales unimaginable to VFX.

Now, if only I had some time to sit down and write the manifesto and a reference implementation.

I think Rez is great, especially for a small project addressing an esoteric problem of VFX and managed, really by one guy... the SAT solver works pretty well, and it's got some good ideas in it. Sure, it's lacking deep documentation, the code in places is fairly opaque (my gods would I not want to run it all through pylint) and the Windows support is really just sort of there -- because of course no one is actually testing it in production simultaneously on the entire possible OS and shell matrix -- but the bones are pretty solid and all of that could be fixed right quick if a couple of companies dedicated a bit of time and resources to cleaning it all up, BUT as Allan backed me up on it's a package resolver back-end that's looking for a context resolver front-end to tell it what to do, and THAT project is really the missing link for any VFX startup or going concern thinking of using Rez to answer the package question.

Which augers towards those scary, scary words: cloud, microservices, and on-demand.

M

On Feb 24, 2017, 02:12 +0530, allan johns notifications@github.com, wrote:

Hey all,

So it may help to give a bit of an overview of how we manage production environments at Method, and talk to some of the environment immutability that Michael talks about.

First, Michael is right - this is not rez's domain, and that is by design. Rez doesn't know about shows or shots, or about a production hierarchy - it only knows how to configure environments and build packages. That isn't to say that rez wouldn't benefit a lot from being used in conjunction with another project that does do these things... unfortunately my own endeavors there have only ever been closed source.

We have an approach at Method that gives us anything between an unchanging, immutable env, all the way to an env where a tool launch gets the latest and greatest of everything, depending on how it's configured.

We have 'profiles', which are yaml-based configs, that live on the filesystem in a job/seq/shot/task hierarchy (profiles further down are merged/overridden with those above). One profile defines the env that a tool or group of tools run in, so typically you'll have one per major DCC, at least (in practice you have profiles for your own major apps as well).

All the tools available after 'setshotting' into the system, are actually wrappers that are aware of these profiles. When the tool is launched, the resolve occurs and the env is dynamically created, then the target tool is started within that shell. This would be pretty expensive (with every tool doing a resolve) except that the rez caching system comes into play here, and typically has a 98-99% hit rate. So new resolves actually don't occur often.

The profiles define what package versions/ranges we want, but that still leaves us exposed to getting the newest versions within the requested version ranges ("foo-1" might get "foo-1.3.0" one moment, and "foo-1.4.0" the next). This doesn't help if we want more stability than that. This is where "package locks" come in. A package lock is a timestamp (also stored into a config file) that basically says "don't use package versions released after this time, unless the resolve demands it." I'll skip the details of how that actually works but suffice to say it is this feature that gives us both stability and the ability to update to a newer package version explicitly (in a profile) if need be.

All these config files (profiles, locks) are also version controlled. So we also have the ability to roll any environment back to exactly its state at some point in the past.

All of the above would indeed work even better if turned into a service and db-backed. We didn't go the db route initially though, because you basically get source control for free from git if you do things file-based. I could see how a duel git+db approach might work well though. Rez as a service is a separate but intriguing topic - certainly a performance benefit is to be had going down that route.

Hth A

On Fri, Feb 24, 2017 at 2:07 AM, Michael Morehouse <notifications@github.com

wrote:

I'm not suggesting that you force the user to open only in the previous version of the context; if the current "official" context is compatible, use the -- presumably but no necessarily higher -- versions in that, if the current is incompatible, warn the user of the discrepancies and or environment breaking issues. The context saved in the asset db / sidecar file should always be the actual complete context used at file "publish" (i.e. it should allow you to recreate the exact environment and fully resolved package set that was used to create the file, with all dependencies, manual overrides, etc accounted for). The problem with parsing the file path is you can only get vertical hierarchy encoded into that (otherwise you'd need query parameter, like in a URL), so without the sidecar or a db you're, at best, taking a guess as to the context (sans department, task, user, ... overrides) and maybe, if you're lucky, using the file creation time to do a Rez solve that at least doesn't include packages installed after that point.

My goal is always to empower the artists, never to hem them in ... what I do want to do is remove from their thinking things that are really just terrible habits we learned before computers could talk to each other, like mnemonic directory structures in a world where (if you're on Windows) you STILL can't have a reliable $PATH of > 2048 characters. An artist in a well-designed pipeline doesn't even consider where to save a file, their context determines that. They don't even consider whether or not to put it in a database, their pipeline ubiquitously does that for them behind the scenes... artists are more expensive than machines and their job is to create art, not directory structures or well formed file paths, both of which are things machines are ridiculously good at. So ultimately anything I suggest is from too many years of watching artist's lives and talent be wasted on automatable esoterica. My problem is that our industry has voluntarily NOT kept up with the rest of the world, generally out of a misguided fear of networking and databases, and that it's time we woke up and caught up; in a world where Dropbox is seamlessly versioning every file as you and/or others edit it collaboratively in real time, we should have this problem solved.

Anyway, don't want to digress too far, especially on a Github issue that I can't really contribute to at the moment. I chimed in because Windows is a very hard nut to crack in terms of giving people the contextually pipelined experience, due to a) a very ingrained model of "click on the damned thing to open in the appropriate program" that doesn't actually line up with how someone working on different projects with different needs from the same underlying tools actually works, b) a historically "what sort of idiot needs a terminal" approach from Redmonds, and c) fundamental limitations with CMD (many of which aren't resolved by either Powershell or Cygwin, though they can help make the UX prettier, and some of which cannot be resolved -- I see you, process forking) and that getting stuck in a child shell and having to exit rather than updating the running shell is perhaps a red herring you might not want to chase too far.

And, sidebar, yes I MASSIVELY prefer what has become the "Linux model", and my mantra for years has been teach the artists the command line, because a) they're not stupid and b) it will make them MUCH more powerful (hence higher paid). Unfortunately there are a fair few business (and IT) types in the smaller shops who can't wrap their head around not paying for an OS.

My four or so bita,

M

On Feb 23, 2017, 18:02 +0530, Jason Scott notifications@github.com, wrote:

Interesting ideas, but I think I fundamentally disagree with the need to do it (not arguing with the execution). I feel like in a production environment, if you're storing the context per file, and then re-opening that file using that exact context, you're ignoring the production needs of updating the suite of tools that the user should be using to open that file. Keeping track of it for historical reasons is beneficial, but I would want to force all my users by default to use the controlled environment I provide (whether "I" = DFX, pipeline supervisor, etc.), not the one they used at the time they saved the file.

Outside of that, we (on Linux) just don't even provide most of the applications to open files from the desktop environment, requiring users to run a few shell commands first that get us into a ballpark of contexts. We only have a few wrappers that do that, and yes, they parse the path, but I totally agree that's a dangerous route to go down (its development predates me here).

Jason Scott

On Thu, Feb 23, 2017 at 2:18 AM, Michael Morehouse < notifications@github.com

wrote:

Btw, that hash discussion tangent there assumes you're doing "version on save" where you're putting committed changes in an immutable, read-only area, and artists load from that area to begin work on the next version... as that's the only way file contents remain a viable key into the asset db.

On Feb 23, 2017, 10:31 +0530, Thomas Mansencal < notifications@github.com>, wrote:

@michaelmovies (https://github.com/michaelmovies) : From initial tests rez-gui is pretty much that context resolver, it is too complicated for an artist but I can see something leveraging the contexts it generates or a similar tool in hands of artists.

There is still the question of what to do when somebody double-click a random maya scene in a project directory, we have some thoughts about handling that: the file association is made toward a dedicated wrapper that could then figure out which context to apply depending the location of the file in the directory tree.

Shotgun Desktop is interesting, thanks for mentioning it!

By the way, are you the former LBG / Weta Michael?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub ( https://github.com/nerdvegas/rez/issues/336#issuecomment-281895180), or mute the thread (https://github.com/notifications/unsubscribe- auth/AAnEpwidT_TfAl4LAj61H7aydAlsxvzeks5rfRKRgaJpZM4JbYFf).

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/nerdvegas/rez/issues/336#issuecomment-281952647, or mute the thread https://github.com/notifications/unsubscribe-auth/ AMjo9w4rAanlV4DCJpzUGzDoyQ9U7d7mks5rfVz6gaJpZM4JbYFf .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub ( https://github.com/nerdvegas/rez/issues/336#issuecomment-281979956), or mute the thread (https://github.com/notifications/unsubscribe- auth/AAnEp79yoPtJ-QL-_CFY82ro3-FsSIo4ks5rfXxOgaJpZM4JbYFf).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nerdvegas/rez/issues/336#issuecomment-282017521, or mute the thread https://github.com/notifications/unsubscribe-auth/ABjqSukZy_RIf-gZ0Etmfm-BfyCR61k3ks5rfaDGgaJpZM4JbYFf .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub (https://github.com/nerdvegas/rez/issues/336#issuecomment-282114728), or mute the thread (https://github.com/notifications/unsubscribe-auth/AAnEpzm6vFeBKwM-zhfwnli_NGF1BJCFks5rfe8ngaJpZM4JbYFf).

[nerdvegas]

Well I do agree with some of your points, however some of them we've also addressed already in our current implementation. Specifically:

1) File system load hasn't been appreciable so far. And we haven't yet even put a caching mechanism in place, which would reduce any tool load to a handful of file stats (and reads on the odd occasion a config is updated). In practice this doesn't seem to add up to much.

2) This isn't strictly true for us. We also have the concept of 'views', which choose particular profiles to make visible. So while compositors will see one set of profiles, FX artists will see another.

3) Config load time isn't huge for us even given the (admittedly terrible) YAML load times. Again this could be cached and we'd drop 95% of whatever load we currently have. As for schema validation, given the sheer number of configurations we have, I don't really see anyone taking the time to author said schemas, even if we had the ability to enforce them.

4) Our config files are controlled by a system that manages the backing git repo, and syncs changes across all sites. We're planning on adding syntax checking to this service as well. Thus we don't have the problems you describe here.

5) Our system doesn't allow a type mismatched override in any of our config files. Yes a lot of info is split across many files but I'm not sure what specific downside that brings.

6) We definitely lack the ability to have an overview of our profiles (and where they're getting overridden). Even if we had that info easily accessible though I'm not yet clear on how we'd utilise it. Specifically though, in terms of how someone tests a change... we have user-level config overrides. So a change can be inserted anywhere in our hierarchy, and is only visible to the user testing it. In terms of not knowing the implications of changing a facility-level config: here we put the onus on the show. They have the ability to block themselves from upstream changes using another kind of lock, called a "profile lock" - if they don't have one, then it's assumed they were fine with getting the latest updates.

7) We have a separate system that manages these concerns. All config files are source controlled and all changes are synced across sites, but we don't require a single monolithic repo. What we have seems to work fairly well so far.

I still agree with you though that the system as a whole would benefit in multiple ways from being a service; and also that this production management part is the missing piece of what could otherwise be a pretty cohesive solution to package management in VFX. Unfortunately that's not how the cards have fallen so far, it turned out that I never did get the chance to implement the production management end as an open source project.

In terms of rez itself, of course it could be improved with more resources put in; it's simply become a huge project for a single person to keep updated (there are a string of releases coming this week though). But just to clear up a few things - it's not a SAT solver (I did implement one in the past though, the results were interesting); and there is a studio out there using rez extensively across all platforms (although admittedly not all the shells). Windows does seem problematic for several users atm, but I rely on third party contributions completely wrt Windows so I'm leaning on external PRs to solve those problems.

A

On Fri, Feb 24, 2017 at 12:14 PM, Michael Morehouse < notifications@github.com> wrote:

This isn't a dig at Method, or any shop, as I know as well as anyone that this is a core problem of VFX and the workloads that are placed on all of the companies big and small within it, but that isn't "managing" context, that's managing not to always be in a broken context.

That roll up of yaml profiles is pretty much what's been developed, separately, at every facility I've worked at... the industry really does like spinning its wheels on the same problem. Larger companies have tended towards XML, but the idea is fundamentally the same. The problems with it range from the general to the specific:

1) you're immediately buying into the idea that putting extra load on file reads is a good idea ... every time an artist (or a pipeline process, or a farm process) does a "set context" you have to do the roll up again, which means walking a disk hierarchy, stat for file existence, read file...

2) you're again limited to a context that is effectively vertical only ... lower contextual levels override higher ones -- i.e. if you're in this shot, use maya 2014 + [plugins] -- but there's no concept of horizontal overrides -- i.e. if you're in this shot, use maya 2014 + [plugins] unless you're in FX, in which case use maya 2016 + [plugins] + [fx plugins]).

3) on top of your grinding on file reads, you're also limited by file interpretation and validation: xml is fast across all OS in python, thanks to the c extension, but is complex and most programmers and especially anytime editing it manually hate it, while yaml, in Python, is EXTREMELY slow without libyaml (which fails to compile out of the box on Windows, and seems to be an abandoned project that hasn't been updated in years) ... also neither YAML or JSON has a scheme mechanism, which makes writing anything more than a linter that validates these files after the fact really hard or at least very error prone.

4) anyone who can edit any of those files directly can break anything from a whole shot to a whole company if they introduce a syntax error. Or forget that "true" and "True" in YAML are entirely different primitive types, or a large number of other esoteric rules. And what even is the right thing to do if someone chmods a directory recusrvidsly? Does what was your time-locked context yesterday just lose all that granularity silently? If so, what was the point of the time locking again?

5) context ends up getting split over a large number of files (i.e. per DCC app and per internal app, as you describe, but also site wide, show wide, etc common values) which makes rollup quite complex. You can't just use Chainmap in Python, because that doesn't work with nested dicts and lists, and how do you resolve (if at all) someone setting a particular key's value as an Integer at the facility level but a float (or a boolean, or a string) at lower level?

6) writing and editing contexts becomes VERY hard as your company's complexity grows... if you've got 13 simultaneously running massive feature film projects, each one offset from the others by potentially 2-3 entire generations of DCC app releases, how does someone get a high level overview of the entire context, drill down and make a change, and then commit that change out into production? More importantly, how do they know that a high level change they're thinking of making will break half those productions? Or at least require them to be locked off? Does that mean a notification mechanism to allow other stakeholders to opt their own production into or out of that change? Now you need an entire testing and delivery mechanism for context DATA, nevermind just context code... and anyone reading this knows just how abysmally bad and under-skilled we are at testing, nevermind CI and CD in the VFX world.

7) though git does give you version control over context -- congratulations, you're a seriously large step ahead from quite a few places I've been at or known about -- git is a TERRIBLE delivery mechanism for keeping multiple sites in sync. You either need to run a "git reset --HARD && git pull" on every configuration file at every depth in the contextual hierarchy before ever reading it, or you need to keep all your sites informed of the exact hash they should have locally for each repo or your one, no doubt unwieldy, monolith repo.

I can go on and on, but basically what I'm saying is that context file rollup is like generation 2 of VFX configuration management, one step above a single static configuration, and this is basically exactly the problem that the rest of the world solves with a database and a well-formed API. Your ability to scale should not be locked to file IO performance, nor should you be forced to walk down the path of ad hoc caching of files with notification invalidation by file change watching daemon processes, especially when the solutions to the problem, broadly, already exist and have been battle-tested for a decade or more at scales unimaginable to VFX.

Now, if only I had some time to sit down and write the manifesto and a reference implementation.

I think Rez is great, especially for a small project addressing an esoteric problem of VFX and managed, really by one guy... the SAT solver works pretty well, and it's got some good ideas in it. Sure, it's lacking deep documentation, the code in places is fairly opaque (my gods would I not want to run it all through pylint) and the Windows support is really just sort of there -- because of course no one is actually testing it in production simultaneously on the entire possible OS and shell matrix -- but the bones are pretty solid and all of that could be fixed right quick if a couple of companies dedicated a bit of time and resources to cleaning it all up, BUT as Allan backed me up on it's a package resolver back-end that's looking for a context resolver front-end to tell it what to do, and THAT project is really the missing link for any VFX startup or going concern thinking of using Rez to answer the package question.

Which augers towards those scary, scary words: cloud, microservices, and on-demand.

M

On Feb 24, 2017, 02:12 +0530, allan johns notifications@github.com, wrote:

Hey all,

So it may help to give a bit of an overview of how we manage production environments at Method, and talk to some of the environment immutability that Michael talks about.

First, Michael is right - this is not rez's domain, and that is by design. Rez doesn't know about shows or shots, or about a production hierarchy - it only knows how to configure environments and build packages. That isn't to say that rez wouldn't benefit a lot from being used in conjunction with another project that does do these things... unfortunately my own endeavors there have only ever been closed source.

We have an approach at Method that gives us anything between an unchanging, immutable env, all the way to an env where a tool launch gets the latest and greatest of everything, depending on how it's configured.

We have 'profiles', which are yaml-based configs, that live on the filesystem in a job/seq/shot/task hierarchy (profiles further down are merged/overridden with those above). One profile defines the env that a tool or group of tools run in, so typically you'll have one per major DCC, at least (in practice you have profiles for your own major apps as well).

All the tools available after 'setshotting' into the system, are actually wrappers that are aware of these profiles. When the tool is launched, the resolve occurs and the env is dynamically created, then the target tool is started within that shell. This would be pretty expensive (with every tool doing a resolve) except that the rez caching system comes into play here, and typically has a 98-99% hit rate. So new resolves actually don't occur often.

The profiles define what package versions/ranges we want, but that still leaves us exposed to getting the newest versions within the requested version ranges ("foo-1" might get "foo-1.3.0" one moment, and "foo-1.4.0" the next). This doesn't help if we want more stability than that. This is where "package locks" come in. A package lock is a timestamp (also stored into a config file) that basically says "don't use package versions released after this time, unless the resolve demands it." I'll skip the details of how that actually works but suffice to say it is this feature that gives us both stability and the ability to update to a newer package version explicitly (in a profile) if need be.

All these config files (profiles, locks) are also version controlled. So we also have the ability to roll any environment back to exactly its state at some point in the past.

All of the above would indeed work even better if turned into a service and db-backed. We didn't go the db route initially though, because you basically get source control for free from git if you do things file-based. I could see how a duel git+db approach might work well though. Rez as a service is a separate but intriguing topic - certainly a performance benefit is to be had going down that route.

Hth A

On Fri, Feb 24, 2017 at 2:07 AM, Michael Morehouse < notifications@github.com

wrote:

I'm not suggesting that you force the user to open only in the previous version of the context; if the current "official" context is compatible, use the -- presumably but no necessarily higher -- versions in that, if the current is incompatible, warn the user of the discrepancies and or environment breaking issues. The context saved in the asset db / sidecar file should always be the actual complete context used at file "publish" (i.e. it should allow you to recreate the exact environment and fully resolved package set that was used to create the file, with all dependencies, manual overrides, etc accounted for). The problem with parsing the file path is you can only get vertical hierarchy encoded into that (otherwise you'd need query parameter, like in a URL), so without the sidecar or a db you're, at best, taking a guess as to the context (sans department, task, user, ... overrides) and maybe, if you're lucky, using the file creation time to do a Rez solve that at least doesn't include packages installed after that point.

My goal is always to empower the artists, never to hem them in ... what I do want to do is remove from their thinking things that are really just terrible habits we learned before computers could talk to each other, like mnemonic directory structures in a world where (if you're on Windows) you STILL can't have a reliable $PATH of > 2048 characters. An artist in a well-designed pipeline doesn't even consider where to save a file, their context determines that. They don't even consider whether or not to put it in a database, their pipeline ubiquitously does that for them behind the scenes... artists are more expensive than machines and their job is to create art, not directory structures or well formed file paths, both of which are things machines are ridiculously good at. So ultimately anything I suggest is from too many years of watching artist's lives and talent be wasted on automatable esoterica. My problem is that our industry has voluntarily NOT kept up with the rest of the world, generally out of a misguided fear of networking and databases, and that it's time we woke up and caught up; in a world where Dropbox is seamlessly versioning every file as you and/or others edit it collaboratively in real time, we should have this problem solved.

Anyway, don't want to digress too far, especially on a Github issue that I can't really contribute to at the moment. I chimed in because Windows is a very hard nut to crack in terms of giving people the contextually pipelined experience, due to a) a very ingrained model of "click on the damned thing to open in the appropriate program" that doesn't actually line up with how someone working on different projects with different needs from the same underlying tools actually works, b) a historically "what sort of idiot needs a terminal" approach from Redmonds, and c) fundamental limitations with CMD (many of which aren't resolved by either Powershell or Cygwin, though they can help make the UX prettier, and some of which cannot be resolved -- I see you, process forking) and that getting stuck in a child shell and having to exit rather than updating the running shell is perhaps a red herring you might not want to chase too far.

And, sidebar, yes I MASSIVELY prefer what has become the "Linux model", and my mantra for years has been teach the artists the command line, because a) they're not stupid and b) it will make them MUCH more powerful (hence higher paid). Unfortunately there are a fair few business (and IT) types in the smaller shops who can't wrap their head around not paying for an OS.

My four or so bita,

M

On Feb 23, 2017, 18:02 +0530, Jason Scott notifications@github.com, wrote:

Interesting ideas, but I think I fundamentally disagree with the need to do it (not arguing with the execution). I feel like in a production environment, if you're storing the context per file, and then re-opening that file using that exact context, you're ignoring the production needs of updating the suite of tools that the user should be using to open that file. Keeping track of it for historical reasons is beneficial, but I would want to force all my users by default to use the controlled environment I provide (whether "I" = DFX, pipeline supervisor, etc.), not the one they used at the time they saved the file.

Outside of that, we (on Linux) just don't even provide most of the applications to open files from the desktop environment, requiring users to run a few shell commands first that get us into a ballpark of contexts. We only have a few wrappers that do that, and yes, they parse the path, but I totally agree that's a dangerous route to go down (its development predates me here).

Jason Scott

On Thu, Feb 23, 2017 at 2:18 AM, Michael Morehouse < notifications@github.com

wrote:

Btw, that hash discussion tangent there assumes you're doing "version on save" where you're putting committed changes in an immutable, read-only area, and artists load from that area to begin work on the next version... as that's the only way file contents remain a viable key into the asset db.

On Feb 23, 2017, 10:31 +0530, Thomas Mansencal < notifications@github.com>, wrote:

@michaelmovies (https://github.com/michaelmovies) : From initial tests rez-gui is pretty much that context resolver, it is too complicated for an artist but I can see something leveraging the contexts it generates or a similar tool in hands of artists.

There is still the question of what to do when somebody double-click a random maya scene in a project directory, we have some thoughts about handling that: the file association is made toward a dedicated wrapper that could then figure out which context to apply depending the location of the file in the directory tree.

Shotgun Desktop is interesting, thanks for mentioning it!

By the way, are you the former LBG / Weta Michael?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub ( https://github.com/nerdvegas/rez/issues/336#issuecomment-281895180 ), or mute the thread (https://github.com/notifications/unsubscribe- auth/AAnEpwidT_TfAl4LAj61H7aydAlsxvzeks5rfRKRgaJpZM4JbYFf).

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/nerdvegas/rez/issues/336#issuecomment- 281952647, or mute the thread https://github.com/notifications/unsubscribe-auth/ AMjo9w4rAanlV4DCJpzUGzDoyQ9U7d7mks5rfVz6gaJpZM4JbYFf .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub ( https://github.com/nerdvegas/rez/issues/336#issuecomment-281979956), or mute the thread (https://github.com/notifications/unsubscribe- auth/AAnEp79yoPtJ-QL-_CFY82ro3-FsSIo4ks5rfXxOgaJpZM4JbYFf).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nerdvegas/rez/issues/336#issuecomment-282017521, or mute the thread https://github.com/notifications/unsubscribe- auth/ABjqSukZy_RIf-gZ0Etmfm-BfyCR61k3ks5rfaDGgaJpZM4JbYFf .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub ( https://github.com/nerdvegas/rez/issues/336#issuecomment-282114728), or mute the thread (https://github.com/notifications/unsubscribe- auth/AAnEpzm6vFeBKwM-zhfwnli_NGF1BJCFks5rfe8ngaJpZM4JbYFf).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nerdvegas/rez/issues/336#issuecomment-282174116, or mute the thread https://github.com/notifications/unsubscribe-auth/ABjqSsCaMyPZB3O8adR8XLUeAUoApigaks5rfi7dgaJpZM4JbYFf .

[yawpitch]

Yeah, I'm aware of the (many) workarounds that let files fill the gap; I've certainly seen bespoke solutions to config rollup working -- to various degrees -- at 5 different facilities personally, and have discussed in depth the solutions used by a number of others, but what I described above are some of the fundamental limitations at scale of using a "disk database", all of which are answered by thinking of the solution to the problem as a service that answers questions using an actual database.

For instance I've personally seen 1, 3, and 5 (config file load and parse) have a dramatic impact, in aggregate, on a very large farm. Admittedly it took seeing the problem at Weta-scale for it to really click just how much impact repeated and absolutely unnecessary file IO over a massive scale distributed file system could have -- tangentially, ever seen RV take over five minutes to load due solely to the size of the PYTHONPATH? I have -- but it absolutely has a substantial impact, when considered on the system at large, on the bottom line. The question isn't should we avoid it as an optimization step, it's why incur that cost at all, especially when it's also soaking up the bandwidth budget of other much higher priority business processes?

I've also seen 4 and 7 (using a VCS to keep config files in sync over multiple sites that are geographically disparate) bite facilities in the ass really, really hard. By copying the files in multiple locations you're creating more than one base truth... when it works, not too much of an issue, when something goes wrong (ie someone somewhere has write permissions they shouldn't, but much more subtly when some cron job is working at different times in different jurisdictions), it can be incredibly hard to unwind, and the more you scale the more it necessarily goes wrong. A single source of truth with a single audit log is always preferable; git improved things from SVN -- with its unending capacity for sqlite3 screwups and disk locks -- and if you've set up a scripted system of cascade sync it works, but it's still a problem that neither VCS or file systems were designed to solve.

As for Rez, a) didn't know it wasn't a SAT solver under the hood, assumed I was because it's solving the same problem that SAT solvers are typically used for and, at a glance, seemed to be doing it the same way, and b) totally understand the problems and relate to the lack of resources, also never had time to write the service based arch for the context resolver I'm describing, hope to have that this year. I think Rez is a great project and ticks off a lot of boxes, and you did a fine job in the docs of stating what it is and what it isn't. I personally think it'd also be a very good fit to being implemented either as a microservice-based or even a largely serverless application, but those are just observations of a way you might turn it into a project that could one day earn it's own keep. I admit it'd be a pretty mighty effort, I just think it might make it a lot easier to further develop with what will likely remain limited resources and time.

M

On Feb 24, 2017, 10:26 +0530, allan johns notifications@github.com, wrote:

Well I do agree with some of your points, however some of them we've also addressed already in our current implementation. Specifically:

1) File system load hasn't been appreciable so far. And we haven't yet even put a caching mechanism in place, which would reduce any tool load to a handful of file stats (and reads on the odd occasion a config is updated). In practice this doesn't seem to add up to much.

2) This isn't strictly true for us. We also have the concept of 'views', which choose particular profiles to make visible. So while compositors will see one set of profiles, FX artists will see another.

3) Config load time isn't huge for us even given the (admittedly terrible) YAML load times. Again this could be cached and we'd drop 95% of whatever load we currently have. As for schema validation, given the sheer number of configurations we have, I don't really see anyone taking the time to author said schemas, even if we had the ability to enforce them.

4) Our config files are controlled by a system that manages the backing git repo, and syncs changes across all sites. We're planning on adding syntax checking to this service as well. Thus we don't have the problems you describe here.

5) Our system doesn't allow a type mismatched override in any of our config files. Yes a lot of info is split across many files but I'm not sure what specific downside that brings.

6) We definitely lack the ability to have an overview of our profiles (and where they're getting overridden). Even if we had that info easily accessible though I'm not yet clear on how we'd utilise it. Specifically though, in terms of how someone tests a change... we have user-level config overrides. So a change can be inserted anywhere in our hierarchy, and is only visible to the user testing it. In terms of not knowing the implications of changing a facility-level config: here we put the onus on the show. They have the ability to block themselves from upstream changes using another kind of lock, called a "profile lock" - if they don't have one, then it's assumed they were fine with getting the latest updates.

7) We have a separate system that manages these concerns. All config files are source controlled and all changes are synced across sites, but we don't require a single monolithic repo. What we have seems to work fairly well so far.

I still agree with you though that the system as a whole would benefit in multiple ways from being a service; and also that this production management part is the missing piece of what could otherwise be a pretty cohesive solution to package management in VFX. Unfortunately that's not how the cards have fallen so far, it turned out that I never did get the chance to implement the production management end as an open source project.

In terms of rez itself, of course it could be improved with more resources put in; it's simply become a huge project for a single person to keep updated (there are a string of releases coming this week though). But just to clear up a few things - it's not a SAT solver (I did implement one in the past though, the results were interesting); and there is a studio out there using rez extensively across all platforms (although admittedly not all the shells). Windows does seem problematic for several users atm, but I rely on third party contributions completely wrt Windows so I'm leaning on external PRs to solve those problems.

A

On Fri, Feb 24, 2017 at 12:14 PM, Michael Morehouse < notifications@github.com> wrote:

This isn't a dig at Method, or any shop, as I know as well as anyone that this is a core problem of VFX and the workloads that are placed on all of the companies big and small within it, but that isn't "managing" context, that's managing not to always be in a broken context.

That roll up of yaml profiles is pretty much what's been developed, separately, at every facility I've worked at... the industry really does like spinning its wheels on the same problem. Larger companies have tended towards XML, but the idea is fundamentally the same. The problems with it range from the general to the specific:

1) you're immediately buying into the idea that putting extra load on file reads is a good idea ... every time an artist (or a pipeline process, or a farm process) does a "set context" you have to do the roll up again, which means walking a disk hierarchy, stat for file existence, read file...

2) you're again limited to a context that is effectively vertical only ... lower contextual levels override higher ones -- i.e. if you're in this shot, use maya 2014 + [plugins] -- but there's no concept of horizontal overrides -- i.e. if you're in this shot, use maya 2014 + [plugins] unless you're in FX, in which case use maya 2016 + [plugins] + [fx plugins]).

3) on top of your grinding on file reads, you're also limited by file interpretation and validation: xml is fast across all OS in python, thanks to the c extension, but is complex and most programmers and especially anytime editing it manually hate it, while yaml, in Python, is EXTREMELY slow without libyaml (which fails to compile out of the box on Windows, and seems to be an abandoned project that hasn't been updated in years) ... also neither YAML or JSON has a scheme mechanism, which makes writing anything more than a linter that validates these files after the fact really hard or at least very error prone.

4) anyone who can edit any of those files directly can break anything from a whole shot to a whole company if they introduce a syntax error. Or forget that "true" and "True" in YAML are entirely different primitive types, or a large number of other esoteric rules. And what even is the right thing to do if someone chmods a directory recusrvidsly? Does what was your time-locked context yesterday just lose all that granularity silently? If so, what was the point of the time locking again?

5) context ends up getting split over a large number of files (i.e. per DCC app and per internal app, as you describe, but also site wide, show wide, etc common values) which makes rollup quite complex. You can't just use Chainmap in Python, because that doesn't work with nested dicts and lists, and how do you resolve (if at all) someone setting a particular key's value as an Integer at the facility level but a float (or a boolean, or a string) at lower level?

6) writing and editing contexts becomes VERY hard as your company's complexity grows... if you've got 13 simultaneously running massive feature film projects, each one offset from the others by potentially 2-3 entire generations of DCC app releases, how does someone get a high level overview of the entire context, drill down and make a change, and then commit that change out into production? More importantly, how do they know that a high level change they're thinking of making will break half those productions? Or at least require them to be locked off? Does that mean a notification mechanism to allow other stakeholders to opt their own production into or out of that change? Now you need an entire testing and delivery mechanism for context DATA, nevermind just context code... and anyone reading this knows just how abysmally bad and under-skilled we are at testing, nevermind CI and CD in the VFX world.

7) though git does give you version control over context -- congratulations, you're a seriously large step ahead from quite a few places I've been at or known about -- git is a TERRIBLE delivery mechanism for keeping multiple sites in sync. You either need to run a "git reset --HARD && git pull" on every configuration file at every depth in the contextual hierarchy before ever reading it, or you need to keep all your sites informed of the exact hash they should have locally for each repo or your one, no doubt unwieldy, monolith repo.

I can go on and on, but basically what I'm saying is that context file rollup is like generation 2 of VFX configuration management, one step above a single static configuration, and this is basically exactly the problem that the rest of the world solves with a database and a well-formed API. Your ability to scale should not be locked to file IO performance, nor should you be forced to walk down the path of ad hoc caching of files with notification invalidation by file change watching daemon processes, especially when the solutions to the problem, broadly, already exist and have been battle-tested for a decade or more at scales unimaginable to VFX.

Now, if only I had some time to sit down and write the manifesto and a reference implementation.

I think Rez is great, especially for a small project addressing an esoteric problem of VFX and managed, really by one guy... the SAT solver works pretty well, and it's got some good ideas in it. Sure, it's lacking deep documentation, the code in places is fairly opaque (my gods would I not want to run it all through pylint) and the Windows support is really just sort of there -- because of course no one is actually testing it in production simultaneously on the entire possible OS and shell matrix -- but the bones are pretty solid and all of that could be fixed right quick if a couple of companies dedicated a bit of time and resources to cleaning it all up, BUT as Allan backed me up on it's a package resolver back-end that's looking for a context resolver front-end to tell it what to do, and THAT project is really the missing link for any VFX startup or going concern thinking of using Rez to answer the package question.

Which augers towards those scary, scary words: cloud, microservices, and on-demand.

M

On Feb 24, 2017, 02:12 +0530, allan johns notifications@github.com, wrote:

Hey all,

So it may help to give a bit of an overview of how we manage production environments at Method, and talk to some of the environment immutability that Michael talks about.

First, Michael is right - this is not rez's domain, and that is by design. Rez doesn't know about shows or shots, or about a production hierarchy - it only knows how to configure environments and build packages. That isn't to say that rez wouldn't benefit a lot from being used in conjunction with another project that does do these things... unfortunately my own endeavors there have only ever been closed source.

We have an approach at Method that gives us anything between an unchanging, immutable env, all the way to an env where a tool launch gets the latest and greatest of everything, depending on how it's configured.

We have 'profiles', which are yaml-based configs, that live on the filesystem in a job/seq/shot/task hierarchy (profiles further down are merged/overridden with those above). One profile defines the env that a tool or group of tools run in, so typically you'll have one per major DCC, at least (in practice you have profiles for your own major apps as well).

All the tools available after 'setshotting' into the system, are actually wrappers that are aware of these profiles. When the tool is launched, the resolve occurs and the env is dynamically created, then the target tool is started within that shell. This would be pretty expensive (with every tool doing a resolve) except that the rez caching system comes into play here, and typically has a 98-99% hit rate. So new resolves actually don't occur often.

The profiles define what package versions/ranges we want, but that still leaves us exposed to getting the newest versions within the requested version ranges ("foo-1" might get "foo-1.3.0" one moment, and "foo-1.4.0" the next). This doesn't help if we want more stability than that. This is where "package locks" come in. A package lock is a timestamp (also stored into a config file) that basically says "don't use package versions released after this time, unless the resolve demands it." I'll skip the details of how that actually works but suffice to say it is this feature that gives us both stability and the ability to update to a newer package version explicitly (in a profile) if need be.

All these config files (profiles, locks) are also version controlled. So we also have the ability to roll any environment back to exactly its state at some point in the past.

All of the above would indeed work even better if turned into a service and db-backed. We didn't go the db route initially though, because you basically get source control for free from git if you do things file-based. I could see how a duel git+db approach might work well though. Rez as a service is a separate but intriguing topic - certainly a performance benefit is to be had going down that route.

Hth A

On Fri, Feb 24, 2017 at 2:07 AM, Michael Morehouse < notifications@github.com

wrote:

I'm not suggesting that you force the user to open only in the previous version of the context; if the current "official" context is compatible, use the -- presumably but no necessarily higher -- versions in that, if the current is incompatible, warn the user of the discrepancies and or environment breaking issues. The context saved in the asset db / sidecar file should always be the actual complete context used at file "publish" (i.e. it should allow you to recreate the exact environment and fully resolved package set that was used to create the file, with all dependencies, manual overrides, etc accounted for). The problem with parsing the file path is you can only get vertical hierarchy encoded into that (otherwise you'd need query parameter, like in a URL), so without the sidecar or a db you're, at best, taking a guess as to the context (sans department, task, user, ... overrides) and maybe, if you're lucky, using the file creation time to do a Rez solve that at least doesn't include packages installed after that point.

My goal is always to empower the artists, never to hem them in ... what I do want to do is remove from their thinking things that are really just terrible habits we learned before computers could talk to each other, like mnemonic directory structures in a world where (if you're on Windows) you STILL can't have a reliable $PATH of > 2048 characters. An artist in a well-designed pipeline doesn't even consider where to save a file, their context determines that. They don't even consider whether or not to put it in a database, their pipeline ubiquitously does that for them behind the scenes... artists are more expensive than machines and their job is to create art, not directory structures or well formed file paths, both of which are things machines are ridiculously good at. So ultimately anything I suggest is from too many years of watching artist's lives and talent be wasted on automatable esoterica. My problem is that our industry has voluntarily NOT kept up with the rest of the world, generally out of a misguided fear of networking and databases, and that it's time we woke up and caught up; in a world where Dropbox is seamlessly versioning every file as you and/or others edit it collaboratively in real time, we should have this problem solved.

Anyway, don't want to digress too far, especially on a Github issue that I can't really contribute to at the moment. I chimed in because Windows is a very hard nut to crack in terms of giving people the contextually pipelined experience, due to a) a very ingrained model of "click on the damned thing to open in the appropriate program" that doesn't actually line up with how someone working on different projects with different needs from the same underlying tools actually works, b) a historically "what sort of idiot needs a terminal" approach from Redmonds, and c) fundamental limitations with CMD (many of which aren't resolved by either Powershell or Cygwin, though they can help make the UX prettier, and some of which cannot be resolved -- I see you, process forking) and that getting stuck in a child shell and having to exit rather than updating the running shell is perhaps a red herring you might not want to chase too far.

And, sidebar, yes I MASSIVELY prefer what has become the "Linux model", and my mantra for years has been teach the artists the command line, because a) they're not stupid and b) it will make them MUCH more powerful (hence higher paid). Unfortunately there are a fair few business (and IT) types in the smaller shops who can't wrap their head around not paying for an OS.

My four or so bita,

M

On Feb 23, 2017, 18:02 +0530, Jason Scott notifications@github.com, wrote:

Interesting ideas, but I think I fundamentally disagree with the need to do it (not arguing with the execution). I feel like in a production environment, if you're storing the context per file, and then re-opening that file using that exact context, you're ignoring the production needs of updating the suite of tools that the user should be using to open that file. Keeping track of it for historical reasons is beneficial, but I would want to force all my users by default to use the controlled environment I provide (whether "I" = DFX, pipeline supervisor, etc.), not the one they used at the time they saved the file.

Outside of that, we (on Linux) just don't even provide most of the applications to open files from the desktop environment, requiring users to run a few shell commands first that get us into a ballpark of contexts. We only have a few wrappers that do that, and yes, they parse the path, but I totally agree that's a dangerous route to go down (its development predates me here).

Jason Scott

On Thu, Feb 23, 2017 at 2:18 AM, Michael Morehouse < notifications@github.com

wrote:

Btw, that hash discussion tangent there assumes you're doing "version on save" where you're putting committed changes in an immutable, read-only area, and artists load from that area to begin work on the next version... as that's the only way file contents remain a viable key into the asset db.

On Feb 23, 2017, 10:31 +0530, Thomas Mansencal < notifications@github.com>, wrote:

@michaelmovies (https://github.com/michaelmovies) : From initial tests rez-gui is pretty much that context resolver, it is too complicated for an artist but I can see something leveraging the contexts it generates or a similar tool in hands of artists.

There is still the question of what to do when somebody double-click a random maya scene in a project directory, we have some thoughts about handling that: the file association is made toward a dedicated wrapper that could then figure out which context to apply depending the location of the file in the directory tree.

Shotgun Desktop is interesting, thanks for mentioning it!

By the way, are you the former LBG / Weta Michael?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub ( https://github.com/nerdvegas/rez/issues/336#issuecomment-281895180 ), or mute the thread (https://github.com/notifications/unsubscribe- auth/AAnEpwidT_TfAl4LAj61H7aydAlsxvzeks5rfRKRgaJpZM4JbYFf).

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <https://github.com/nerdvegas/rez/issues/336#issuecomment- 281952647 (tel:281952647)>, or mute the thread https://github.com/notifications/unsubscribe-auth/ AMjo9w4rAanlV4DCJpzUGzDoyQ9U7d7mks5rfVz6gaJpZM4JbYFf .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub ( https://github.com/nerdvegas/rez/issues/336#issuecomment-281979956), or mute the thread (https://github.com/notifications/unsubscribe- auth/AAnEp79yoPtJ-QL-_CFY82ro3-FsSIo4ks5rfXxOgaJpZM4JbYFf).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nerdvegas/rez/issues/336#issuecomment-282017521, or mute the thread https://github.com/notifications/unsubscribe- auth/ABjqSukZy_RIf-gZ0Etmfm-BfyCR61k3ks5rfaDGgaJpZM4JbYFf .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub ( https://github.com/nerdvegas/rez/issues/336#issuecomment-282114728), or mute the thread (https://github.com/notifications/unsubscribe- auth/AAnEpzm6vFeBKwM-zhfwnli_NGF1BJCFks5rfe8ngaJpZM4JbYFf).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nerdvegas/rez/issues/336#issuecomment-282174116, or mute the thread https://github.com/notifications/unsubscribe-auth/ABjqSsCaMyPZB3O8adR8XLUeAUoApigaks5rfi7dgaJpZM4JbYFf .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub (https://github.com/nerdvegas/rez/issues/336#issuecomment-282204316), or mute the thread (https://github.com/notifications/unsubscribe-auth/AAnEp9tyUIy3-4yTpBppr7tgaewuu9Iaks5rfmMGgaJpZM4JbYFf).

[nerdvegas]

WRT git backed config files, we actually have just the one git server. So yeah it's not the fastest, that's certainly a downside, but we don't have issues with ambiguous source of truth.

Some other interesting points.

RE long PYTHONPATH, yep I've seen this - ie I've seem that being the mail culprit in taking down a studio's NFS. IMO this is one of the biggest downsides to rez, and I've spoken about a potential solution previously. I think it would be great to allow any resolve access to a directory, where packages could nominate to softlink/hardlink/cp their python source for example; these "context filesystems" would be cached away so the same subsequent resolve could reuse an existing one. Now of course you could go a totally different tack and write an import hook that pulls from a db, but my understanding atm is that this is problematic in python-2 for various reasons (unsure if this is improved in 3).

RE Rez not a SAT solver. When I implemented a SAT-based rez solver I discovered that SAT was much faster at finding ALL the potential solves, and especially faster at finding if there are NO potential solves; but by and large rez's solver was faster at finding the "best" solve. As well as that you get the resolve graph too - so there didn't seem to be sense in moving to SAT.

RE Rez implementation - I'd love to move to a microservice-based model. I think it would need to be a service, to get the performance benefits I have in mind. But really I think I'd need a couple of months of dedicated time to pull that one off.

Cheers A

On Sat, Feb 25, 2017 at 2:43 AM, Michael Morehouse <notifications@github.com

wrote:

Yeah, I'm aware of the (many) workarounds that let files fill the gap; I've certainly seen bespoke solutions to config rollup working -- to various degrees -- at 5 different facilities personally, and have discussed in depth the solutions used by a number of others, but what I described above are some of the fundamental limitations at scale of using a "disk database", all of which are answered by thinking of the solution to the problem as a service that answers questions using an actual database.

For instance I've personally seen 1, 3, and 5 (config file load and parse) have a dramatic impact, in aggregate, on a very large farm. Admittedly it took seeing the problem at Weta-scale for it to really click just how much impact repeated and absolutely unnecessary file IO over a massive scale distributed file system could have -- tangentially, ever seen RV take over five minutes to load due solely to the size of the PYTHONPATH? I have -- but it absolutely has a substantial impact, when considered on the system at large, on the bottom line. The question isn't should we avoid it as an optimization step, it's why incur that cost at all, especially when it's also soaking up the bandwidth budget of other much higher priority business processes?

I've also seen 4 and 7 (using a VCS to keep config files in sync over multiple sites that are geographically disparate) bite facilities in the ass really, really hard. By copying the files in multiple locations you're creating more than one base truth... when it works, not too much of an issue, when something goes wrong (ie someone somewhere has write permissions they shouldn't, but much more subtly when some cron job is working at different times in different jurisdictions), it can be incredibly hard to unwind, and the more you scale the more it necessarily goes wrong. A single source of truth with a single audit log is always preferable; git improved things from SVN -- with its unending capacity for sqlite3 screwups and disk locks -- and if you've set up a scripted system of cascade sync it works, but it's still a problem that neither VCS or file systems were designed to solve.

As for Rez, a) didn't know it wasn't a SAT solver under the hood, assumed I was because it's solving the same problem that SAT solvers are typically used for and, at a glance, seemed to be doing it the same way, and b) totally understand the problems and relate to the lack of resources, also never had time to write the service based arch for the context resolver I'm describing, hope to have that this year. I think Rez is a great project and ticks off a lot of boxes, and you did a fine job in the docs of stating what it is and what it isn't. I personally think it'd also be a very good fit to being implemented either as a microservice-based or even a largely serverless application, but those are just observations of a way you might turn it into a project that could one day earn it's own keep. I admit it'd be a pretty mighty effort, I just think it might make it a lot easier to further develop with what will likely remain limited resources and time.

M

On Feb 24, 2017, 10:26 +0530, allan johns notifications@github.com, wrote:

Well I do agree with some of your points, however some of them we've also addressed already in our current implementation. Specifically:

1) File system load hasn't been appreciable so far. And we haven't yet even put a caching mechanism in place, which would reduce any tool load to a handful of file stats (and reads on the odd occasion a config is updated). In practice this doesn't seem to add up to much.

2) This isn't strictly true for us. We also have the concept of 'views', which choose particular profiles to make visible. So while compositors will see one set of profiles, FX artists will see another.

3) Config load time isn't huge for us even given the (admittedly terrible) YAML load times. Again this could be cached and we'd drop 95% of whatever load we currently have. As for schema validation, given the sheer number of configurations we have, I don't really see anyone taking the time to author said schemas, even if we had the ability to enforce them.

4) Our config files are controlled by a system that manages the backing git repo, and syncs changes across all sites. We're planning on adding syntax checking to this service as well. Thus we don't have the problems you describe here.

5) Our system doesn't allow a type mismatched override in any of our config files. Yes a lot of info is split across many files but I'm not sure what specific downside that brings.

6) We definitely lack the ability to have an overview of our profiles (and where they're getting overridden). Even if we had that info easily accessible though I'm not yet clear on how we'd utilise it. Specifically though, in terms of how someone tests a change... we have user-level config overrides. So a change can be inserted anywhere in our hierarchy, and is only visible to the user testing it. In terms of not knowing the implications of changing a facility-level config: here we put the onus on the show. They have the ability to block themselves from upstream changes using another kind of lock, called a "profile lock" - if they don't have one, then it's assumed they were fine with getting the latest updates.

7) We have a separate system that manages these concerns. All config files are source controlled and all changes are synced across sites, but we don't require a single monolithic repo. What we have seems to work fairly well so far.

I still agree with you though that the system as a whole would benefit in multiple ways from being a service; and also that this production management part is the missing piece of what could otherwise be a pretty cohesive solution to package management in VFX. Unfortunately that's not how the cards have fallen so far, it turned out that I never did get the chance to implement the production management end as an open source project.

In terms of rez itself, of course it could be improved with more resources put in; it's simply become a huge project for a single person to keep updated (there are a string of releases coming this week though). But just to clear up a few things - it's not a SAT solver (I did implement one in the past though, the results were interesting); and there is a studio out there using rez extensively across all platforms (although admittedly not all the shells). Windows does seem problematic for several users atm, but I rely on third party contributions completely wrt Windows so I'm leaning on external PRs to solve those problems.

A

On Fri, Feb 24, 2017 at 12:14 PM, Michael Morehouse < notifications@github.com> wrote:

This isn't a dig at Method, or any shop, as I know as well as anyone that this is a core problem of VFX and the workloads that are placed on all of the companies big and small within it, but that isn't "managing" context, that's managing not to always be in a broken context.

That roll up of yaml profiles is pretty much what's been developed, separately, at every facility I've worked at... the industry really does like spinning its wheels on the same problem. Larger companies have tended towards XML, but the idea is fundamentally the same. The problems with it range from the general to the specific:

1) you're immediately buying into the idea that putting extra load on file reads is a good idea ... every time an artist (or a pipeline process, or a farm process) does a "set context" you have to do the roll up again, which means walking a disk hierarchy, stat for file existence, read file...

2) you're again limited to a context that is effectively vertical only ... lower contextual levels override higher ones -- i.e. if you're in this shot, use maya 2014 + [plugins] -- but there's no concept of horizontal overrides -- i.e. if you're in this shot, use maya 2014 + [plugins] unless you're in FX, in which case use maya 2016 + [plugins] + [fx plugins]).

3) on top of your grinding on file reads, you're also limited by file interpretation and validation: xml is fast across all OS in python, thanks to the c extension, but is complex and most programmers and especially anytime editing it manually hate it, while yaml, in Python, is EXTREMELY slow without libyaml (which fails to compile out of the box on Windows, and seems to be an abandoned project that hasn't been updated in years) ... also neither YAML or JSON has a scheme mechanism, which makes writing anything more than a linter that validates these files after the fact really hard or at least very error prone.

4) anyone who can edit any of those files directly can break anything from a whole shot to a whole company if they introduce a syntax error. Or forget that "true" and "True" in YAML are entirely different primitive types, or a large number of other esoteric rules. And what even is the right thing to do if someone chmods a directory recusrvidsly? Does what was your time-locked context yesterday just lose all that granularity silently? If so, what was the point of the time locking again?

5) context ends up getting split over a large number of files (i.e. per DCC app and per internal app, as you describe, but also site wide, show wide, etc common values) which makes rollup quite complex. You can't just use Chainmap in Python, because that doesn't work with nested dicts and lists, and how do you resolve (if at all) someone setting a particular key's value as an Integer at the facility level but a float (or a boolean, or a string) at lower level?

6) writing and editing contexts becomes VERY hard as your company's complexity grows... if you've got 13 simultaneously running massive feature film projects, each one offset from the others by potentially 2-3 entire generations of DCC app releases, how does someone get a high level overview of the entire context, drill down and make a change, and then commit that change out into production? More importantly, how do they know that a high level change they're thinking of making will break half those productions? Or at least require them to be locked off? Does that mean a notification mechanism to allow other stakeholders to opt their own production into or out of that change? Now you need an entire testing and delivery mechanism for context DATA, nevermind just context code... and anyone reading this knows just how abysmally bad and under-skilled we are at testing, nevermind CI and CD in the VFX world.

7) though git does give you version control over context -- congratulations, you're a seriously large step ahead from quite a few places I've been at or known about -- git is a TERRIBLE delivery mechanism for keeping multiple sites in sync. You either need to run a "git reset --HARD && git pull" on every configuration file at every depth in the contextual hierarchy before ever reading it, or you need to keep all your sites informed of the exact hash they should have locally for each repo or your one, no doubt unwieldy, monolith repo.

I can go on and on, but basically what I'm saying is that context file rollup is like generation 2 of VFX configuration management, one step above a single static configuration, and this is basically exactly the problem that the rest of the world solves with a database and a well-formed API. Your ability to scale should not be locked to file IO performance, nor should you be forced to walk down the path of ad hoc caching of files with notification invalidation by file change watching daemon processes, especially when the solutions to the problem, broadly, already exist and have been battle-tested for a decade or more at scales unimaginable to VFX.

Now, if only I had some time to sit down and write the manifesto and a reference implementation.

I think Rez is great, especially for a small project addressing an esoteric problem of VFX and managed, really by one guy... the SAT solver works pretty well, and it's got some good ideas in it. Sure, it's lacking deep documentation, the code in places is fairly opaque (my gods would I not want to run it all through pylint) and the Windows support is really just sort of there -- because of course no one is actually testing it in production simultaneously on the entire possible OS and shell matrix -- but the bones are pretty solid and all of that could be fixed right quick if a couple of companies dedicated a bit of time and resources to cleaning it all up, BUT as Allan backed me up on it's a package resolver back-end that's looking for a context resolver front-end to tell it what to do, and THAT project is really the missing link for any VFX startup or going concern thinking of using Rez to answer the package question.

Which augers towards those scary, scary words: cloud, microservices, and on-demand.

M

On Feb 24, 2017, 02:12 +0530, allan johns notifications@github.com, wrote:

Hey all,

So it may help to give a bit of an overview of how we manage production environments at Method, and talk to some of the environment immutability that Michael talks about.

First, Michael is right - this is not rez's domain, and that is by design. Rez doesn't know about shows or shots, or about a production hierarchy - it only knows how to configure environments and build packages. That isn't to say that rez wouldn't benefit a lot from being used in conjunction with another project that does do these things... unfortunately my own endeavors there have only ever been closed source.

We have an approach at Method that gives us anything between an unchanging, immutable env, all the way to an env where a tool launch gets the latest and greatest of everything, depending on how it's configured.

We have 'profiles', which are yaml-based configs, that live on the filesystem in a job/seq/shot/task hierarchy (profiles further down are merged/overridden with those above). One profile defines the env that a tool or group of tools run in, so typically you'll have one per major DCC, at least (in practice you have profiles for your own major apps as well).

All the tools available after 'setshotting' into the system, are actually wrappers that are aware of these profiles. When the tool is launched, the resolve occurs and the env is dynamically created, then the target tool is started within that shell. This would be pretty expensive (with every tool doing a resolve) except that the rez caching system comes into play here, and typically has a 98-99% hit rate. So new resolves actually don't occur often.

The profiles define what package versions/ranges we want, but that still leaves us exposed to getting the newest versions within the requested version ranges ("foo-1" might get "foo-1.3.0" one moment, and "foo-1.4.0" the next). This doesn't help if we want more stability than that. This is where "package locks" come in. A package lock is a timestamp (also stored into a config file) that basically says "don't use package versions released after this time, unless the resolve demands it." I'll skip the details of how that actually works but suffice to say it is this feature that gives us both stability and the ability to update to a newer package version explicitly (in a profile) if need be.

All these config files (profiles, locks) are also version controlled. So we also have the ability to roll any environment back to exactly its state at some point in the past.

All of the above would indeed work even better if turned into a service and db-backed. We didn't go the db route initially though, because you basically get source control for free from git if you do things file-based. I could see how a duel git+db approach might work well though. Rez as a service is a separate but intriguing topic - certainly a performance benefit is to be had going down that route.

Hth A

On Fri, Feb 24, 2017 at 2:07 AM, Michael Morehouse < notifications@github.com

wrote:

I'm not suggesting that you force the user to open only in the previous version of the context; if the current "official" context is compatible, use the -- presumably but no necessarily higher -- versions in that, if the current is incompatible, warn the user of the discrepancies and or environment breaking issues. The context saved in the asset db / sidecar file should always be the actual complete context used at file "publish" (i.e. it should allow you to recreate the exact environment and fully resolved package set that was used to create the file, with all dependencies, manual overrides, etc accounted for). The problem with parsing the file path is you can only get vertical hierarchy encoded into that (otherwise you'd need query parameter, like in a URL), so without the sidecar or a db you're, at best, taking a guess as to the context (sans department, task, user, ... overrides) and maybe, if you're lucky, using the file creation time to do a Rez solve that at least doesn't include packages installed after that point.

My goal is always to empower the artists, never to hem them in ... what I do want to do is remove from their thinking things that are really just terrible habits we learned before computers could talk to each other, like mnemonic directory structures in a world where (if you're on Windows) you STILL can't have a reliable $PATH of > 2048 characters. An artist in a well-designed pipeline doesn't even consider where to save a file, their context determines that. They don't even consider whether or not to put it in a database, their pipeline ubiquitously does that for them behind the scenes... artists are more expensive than machines and their job is to create art, not directory structures or well formed file paths, both of which are things machines are ridiculously good at. So ultimately anything I suggest is from too many years of watching artist's lives and talent be wasted on automatable esoterica. My problem is that our industry has voluntarily NOT kept up with the rest of the world, generally out of a misguided fear of networking and databases, and that it's time we woke up and caught up; in a world where Dropbox is seamlessly versioning every file as you and/or others edit it collaboratively in real time, we should have this problem solved.

Anyway, don't want to digress too far, especially on a Github issue that I can't really contribute to at the moment. I chimed in because Windows is a very hard nut to crack in terms of giving people the contextually pipelined experience, due to a) a very ingrained model of "click on the damned thing to open in the appropriate program" that doesn't actually line up with how someone working on different projects with different needs from the same underlying tools actually works, b) a historically "what sort of idiot needs a terminal" approach from Redmonds, and c) fundamental limitations with CMD (many of which aren't resolved by either Powershell or Cygwin, though they can help make the UX prettier, and some of which cannot be resolved -- I see you, process forking) and that getting stuck in a child shell and having to exit rather than updating the running shell is perhaps a red herring you might not want to chase too far.

And, sidebar, yes I MASSIVELY prefer what has become the "Linux model", and my mantra for years has been teach the artists the command line, because a) they're not stupid and b) it will make them MUCH more powerful (hence higher paid). Unfortunately there are a fair few business (and IT) types in the smaller shops who can't wrap their head around not paying for an OS.

My four or so bita,

M

On Feb 23, 2017, 18:02 +0530, Jason Scott < notifications@github.com>, wrote:

Interesting ideas, but I think I fundamentally disagree with the need to do it (not arguing with the execution). I feel like in a production environment, if you're storing the context per file, and then re-opening that file using that exact context, you're ignoring the production needs of updating the suite of tools that the user should be using to open that file. Keeping track of it for historical reasons is beneficial, but I would want to force all my users by default to use the controlled environment I provide (whether "I" = DFX, pipeline supervisor, etc.), not the one they used at the time they saved the file.

Outside of that, we (on Linux) just don't even provide most of the applications to open files from the desktop environment, requiring users to run a few shell commands first that get us into a ballpark of contexts. We only have a few wrappers that do that, and yes, they parse the path, but I totally agree that's a dangerous route to go down (its development predates me here).

Jason Scott

On Thu, Feb 23, 2017 at 2:18 AM, Michael Morehouse < notifications@github.com

wrote:

Btw, that hash discussion tangent there assumes you're doing "version on save" where you're putting committed changes in an immutable, read-only area, and artists load from that area to begin work on the next version... as that's the only way file contents remain a viable key into the asset db.

On Feb 23, 2017, 10:31 +0530, Thomas Mansencal < notifications@github.com>, wrote:

@michaelmovies (https://github.com/michaelmovies) : From initial tests rez-gui is pretty much that context resolver, it is too complicated for an artist but I can see something leveraging the contexts it generates or a similar tool in hands of artists.

There is still the question of what to do when somebody double-click a random maya scene in a project directory, we have some thoughts about handling that: the file association is made toward a dedicated wrapper that could then figure out which context to apply depending the location of the file in the directory tree.

Shotgun Desktop is interesting, thanks for mentioning it!

By the way, are you the former LBG / Weta Michael?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub ( https://github.com/nerdvegas/rez/issues/336#issuecomment- 281895180 ), or mute the thread (https://github.com/notifications/unsubscribe- auth/AAnEpwidT_TfAl4LAj61H7aydAlsxvzeks5rfRKRgaJpZM4JbYFf).

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <https://github.com/nerdvegas/rez/issues/336#issuecomment- 281952647 (tel:281952647)>, or mute the thread https://github.com/notifications/unsubscribe-auth/ AMjo9w4rAanlV4DCJpzUGzDoyQ9U7d7mks5rfVz6gaJpZM4JbYFf .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub ( https://github.com/nerdvegas/rez/issues/336#issuecomment-281979956 ), or mute the thread (https://github.com/notifications/unsubscribe- auth/AAnEp79yoPtJ-QL-_CFY82ro3-FsSIo4ks5rfXxOgaJpZM4JbYFf).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nerdvegas/rez/issues/336#issuecomment- 282017521, or mute the thread https://github.com/notifications/unsubscribe- auth/ABjqSukZy_RIf-gZ0Etmfm-BfyCR61k3ks5rfaDGgaJpZM4JbYFf .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub ( https://github.com/nerdvegas/rez/issues/336#issuecomment-282114728), or mute the thread (https://github.com/notifications/unsubscribe- auth/AAnEpzm6vFeBKwM-zhfwnli_NGF1BJCFks5rfe8ngaJpZM4JbYFf).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nerdvegas/rez/issues/336#issuecomment-282174116, or mute the thread https://github.com/notifications/unsubscribe-auth/ ABjqSsCaMyPZB3O8adR8XLUeAUoApigaks5rfi7dgaJpZM4JbYFf .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub ( https://github.com/nerdvegas/rez/issues/336#issuecomment-282204316), or mute the thread (https://github.com/notifications/unsubscribe- auth/AAnEp9tyUIy3-4yTpBppr7tgaewuu9Iaks5rfmMGgaJpZM4JbYFf).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nerdvegas/rez/issues/336#issuecomment-282323361, or mute the thread https://github.com/notifications/unsubscribe-auth/ABjqSrig7wJVsa2hj0sTJPKciBJrdANXks5rfvqqgaJpZM4JbYFf .

[instinct-vfx]

Just wanted to chime in and say that i'd be interested in contributing if anyone plans on working towards a service based model (e.g. make the solver a service and have rez-env hand off the resolve to the service and only create the actual shell locally as a first step). That would really help us in deploying things more easily here.

Other than that thanks a lot for the insights and discussions. We are evaluating how to handle context management and we are pretty open to do anything as we do not have a legacy system in place that would be replaced but are starting pretty much from scratch.

I have rolled around the idea of a central configuration management that also handles things like sparsely overridden tool settings (think site, production unit, project, seq, shot but also allow for the multi-dimensional aspects Michael mentioned) and extends to managing context since quite a while and would happily discuss that if anyone is interested. If there is interest in discussing thins or a more service based approach it would make sense to move that to a thread in the google groups rather than an issue here i'd say.

Cheers, Thorsten