CP3->CP4: Change in use of `add_module_for_tst`

[votti]

Hi CP team,

Congratulations to your CP4 release!

I am porting my custom modules to Cellprofiler 4 and run into an undocumented issue when upgrading my tests:

Previously this worked:

import cellprofiler_core.preferences as cppref
import cellprofiler_core.pipeline as cpp
from cellprofiler_core.utilities.core import modules as cpmodules

cppref.set_headless()

from plugins import mymodule

cpmodules.add_module_for_tst(mymodule.MyModule)

data = r"""STRING CONTAING A PIPELINE ONLY CONTAING MY MODULE"""
pipeline = cpp.Pipeline()
pipeline.load(io.StringIO(data))
assert len(pipeline.modules()) == 1

Using add_module_for_tst to inject the custom module.

However this would result in the error:

ERROR    root:_pipeline.py:529 Failed to load pipeline
Traceback (most recent call last):
  File "/home/vitoz/Git/ImcPluginsCP/.nox/tests-3-8/lib/python3.8/site-packages/cellprofiler_core/pipeline/_pipeline.py", line 519, in setup_modules
    module = self.setup_module(
  File "/home/vitoz/Git/ImcPluginsCP/.nox/tests-3-8/lib/python3.8/site-packages/cellprofiler_core/pipeline/_pipeline.py", line 545, in setup_module
    module = self.instantiate_module(module_name)
  File "/home/vitoz/Git/ImcPluginsCP/.nox/tests-3-8/lib/python3.8/site-packages/cellprofiler_core/pipeline/_pipeline.py", line 186, in instantiate_module
    return instantiate_module(module_name)
  File "/home/vitoz/Git/ImcPluginsCP/.nox/tests-3-8/lib/python3.8/site-packages/cellprofiler_core/utilities/core/modules/__init__.py", line 180, in instantiate_module
    module = get_module_class(module_name)()
  File "/home/vitoz/Git/ImcPluginsCP/.nox/tests-3-8/lib/python3.8/site-packages/cellprofiler_core/utilities/core/modules/__init__.py", line 175, in get_module_class
    raise ValueError("Could not find the %s module" % module_class)
ValueError: Could not find the MyModule module

After a quite exhausting debugging session, I found that the issue was that during pipeline loading somehow the added module were overwritten if the all_modules dictionary was not already pre-populated with fill_modules beforehand:

import cellprofiler_core.preferences as cppref
import cellprofiler_core.pipeline as cpp
from cellprofiler_core.utilities.core import modules as cpmodules

cppref.set_headless()

from plugins import mymodule

cpmodules.fill_modules()
cpmodules.add_module_for_tst(mymodule.MyModule)

data = r"""CellProfiler Pipeline: http://www.cellprofiler.org
    Version:3
    DateRevision:20130522170932
    ModuleCount:1
    HasImagePlaneDetails:False

    MyModule:[module_num:1|svn_version:\'Unknown\'|variable_revision_number:2|show_window:False|notes:\x5B\x5D|batch_state:array(\x5B\x5D, dtype=uint8)|enabled:True]
    Select the input image:InputImage
    Name the output image:OutputImage
    """
pipeline = cpp.Pipeline()
pipeline.load(io.StringIO(data))
assert len(pipeline.modules()) == 1

I am not sure if this can be considered a bug or just an undocumented 'feature' :)
Also I am not sure if my solution is really the correct one - it just was the first thing that was working.

But I guess I should not be the only author of custom modules that runs into this eventually.

If you want I could add a test in cellprofiler_core for add_module_for_tst modelling the addition of an example dummy custom module. Do you think this would be worth the effort?