tests have s subtle execution order bug

[theodox]

Looking at the tests in my most recent pytcharm install, it appears that the monkeypatching in test_Nested is failing. It used to work, but now when I run the tests I can see that before the patch is applied most of mGui is already loaded. So, replacing cmds.whatever in this test has no effect because the loaded modules already have a reference to the proper maya cmds version -- at least, that's what I think is happening.

[bob-white]

That is super weird. Just did a quick local run and the whole test suite passed. Though I'm just running them manually from sublime, not from pycharm.

I wonder if its some oddity with the pycharm test runner, any idea how well it isolates tests between runs? Because if it just runs them all alphabetically I can see how running test_api first would mess up the mocking in test_nested

[theodox]

That’s definitely what happens, I think it’s because the mgui tests are being found and run in somebody else’s runners

Sent from my iPhone

On Jul 10, 2018, at 8:16 PM, bob-white notifications@github.com wrote:

That is super weird. Just did a quick local run and the whole test suite passed. Though I'm just running them manually from sublime, not from pycharm.

I wonder if its some oddity with the pycharm test runner, any idea how well it isolates tests between runs? Because if it just runs them all alphabetically I can see how running test_api first would mess up the mocking in test_nested

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[bob-white]

Realized I never hit comment before actually pushing a new branch. But the basic idea in there is that we remove mGui from sys.modules before applying the mocks. That way when we re-import mGui, it will be relying on the altered namespace.

Not sure if its really the best fix, but I use this method when doing GUI tests in maya, as it provides a functional if hacky reload for the whole module.

[theodox]

Does it fix the ‘you can’t reload metaclass hierarchies’ problem?

[bob-white]

Yeah, that's actually why I started using it for testing in Maya. I couldn't ever get the proper module order for reload to not be awful. So I just went with, kill it and start over.

There are some issues if you wipe out the module while a window is open, because it loses a bunch of the weakrefs, but better than restarting Maya.

[theodox]

yeah, I think we can call reloading with open UI an 'advanced feature' we don't want to support.

[theodox]

Thinking this over I'm wondering if the right thing to do is to a full-blown conversion to mocks. I did a kind of half-assed mock for some tests but its inconsistently applied, I think we could make the testing situation a lot clearer if we went ahead and did it everywhere

[bob-white]

Yeah you're probably right. You want to just use mock? It would be the default if/when we can ever upgrade our Maya code to py3. Or do you think there is a benefit from us continuing to roll our own?

Depending how deep we need to go for this, it might be worth checking with some of the other Maya projects, someone might have a base we can build off of.

[theodox]

This is from something else I'm playing with... but it works, and I think is showcases a way forward:

import sys
import mock
from types import ModuleType

_maya = ModuleType('maya')
_cmds = ModuleType('cmds')
_api = ModuleType('api')
_openMaya= ModuleType('OpenMaya')

class MVector(list):
    pass

class MMatrix(list):
    pass
_openMaya.MVector = MVector
_openMaya.MMatrix = MMatrix

_maya.cmds = mock.MagicMock()

sys.modules['maya'] = _maya
sys.modules['maya.cmds'] = _cmds
sys.modules['maya.api'] = _api
sys.modules['maya.api.OpenMaya'] = _openMaya

import maya.cmds as cmds

import sys
sys.path.append(r"D:\prj\ulMaya\mayaScene")
import proxy_replacement as pr

ls = cmds.ls
ls.side_effect = [('pCube1', 'transform')]
result =  pr.proxy('pCube1')
ls.side_effect = [('pCube1.translate', 'double3')]
ga = cmds.getAttr
ga.side_effect = [[(1,2,3)]]
print result.translate

[theodox]

incidentally it looks like maya 2019 (the beta) is breaking all the forms classes -- seems to have trouble finding children of a formLayout :(