mypy runs forever with colour-science

[bersbersbers]

Bug Report

mypy never finishes with colour-science==0.4.0.

To Reproduce

cd /tmp
rm .python-version
pyenv virtualenv mypy
pyenv local mypy

pip install mypy "colour-science<0.4"
mypy $PYENV_ROOT/versions/mypy/lib/python3.10/site-packages/colour/constants/__init__.py 
# finishes in just over 2 minutes

pip install mypy "colour-science>=0.4.0"
mypy $PYENV_ROOT/versions/mypy/lib/python3.10/site-packages/colour/constants/__init__.py 
# does not finish within 10 minutes

Expected Behavior

Finish within ~2 minutes

Actual Behavior

Does not finish

Your Environment

• Mypy version used: mypy 0.931
• Mypy command-line flags: none
• Mypy configuration options from mypy.ini (and other config files): none
• Python version used: 3.9.10
• Operating system and version: OpenSUSE Leap 15.2

[KelSolaar]

Cross-posting here if helpful:

I have just checked on my end, without any mypy cache and not using the daemon:

(colour-49B8_mty-py3.9) Eris:colour kelsolaar$ time mypy --show-error-codes --warn-unused-ignores --warn-redundant-casts --install-types --non-interactive -p colour
colour/io/image.py:119: error: Unused "type: ignore" comment
colour/io/image.py:119: error: Module has no attribute "float128"; maybe "float32" or "float16"?  [attr-defined]
colour/io/image.py:135: error: Unused "type: ignore" comment
colour/io/image.py:135: error: Module has no attribute "float128"; maybe "float32" or "float16"?  [attr-defined]
Found 4 errors in 1 file (checked 532 source files)

real    2m30.199s
user    2m29.421s
sys     0m2.268s

===============================================================================
*                                                                             *
*   Interpreter :                                                             *
*       python : 3.9.9 (main, Nov 21 2021, 03:16:13)                          *
*                [Clang 13.0.0 (clang-1300.0.29.3)]                           *
*                                                                             *
*   colour-science.org :                                                      *
*       colour : v0.3.16-750-g8aab3f78a                                       *
*                                                                             *
*   Runtime :                                                                 *
*       imageio : 2.16.0                                                      *
*       matplotlib : 3.4.3                                                    *
*       networkx : 2.6.3                                                      *
*       numpy : 1.22.2                                                        *
*       pandas : 1.4.1                                                        *
*       pygraphviz : 1.9                                                      *
*       scipy : 1.8.0                                                         *
*       sklearn : 1.0.2                                                       *
*       tqdm : 4.62.3                                                         *
*       trimesh : 3.10.0                                                      *
*                                                                             *
===============================================================================

PS: Those 4 errors are expected on my platform, I'm running with a MacBook M1.

[bersbersbers]

It may be that cleaning the mypy cache has solved this for me.

[bersbersbers]

This issue keeps coming back in various forms. While reporting #12442, I think I confirmed that mypy hangs during cache creation, and with -v, I have seen it hangs at this line:

LOG:  Found 392 SCCs; largest has 302 nodes
[...]
LOG:  Processing SCC of size 302 (colour.utilities.data_structures colour.utilities.metrics colour.utilities.verbose colour.utilities.common colour.algebra.common colour.utilities colour.utilities.array colour.utilities.deprecation colour.models.rgb.transfer_functions.viper_log colour.models.rgb.transfer_functions.pivoted_log colour.models.rgb.transfer_functions.panalog colour.models.rgb.transfer_functions.log colour.models.rgb.transfer_functions.linear colour.models.rgb.transfer_functions.itur_bt_1886 colour.models.rgb.transfer_functions.gopro colour.models.rgb.transfer_functions.filmlight_tlog colour.models.rgb.transfer_functions.exponent colour.models.rgb.transfer_functions.dji_dlog colour.models.rgb.transfer_functions.dicom_gsdf colour.models.rgb.transfer_functions.davinci_intermediate colour.models.rgb.transfer_functions.cineon colour.models.rgb.transfer_functions.blackmagic_design colour.models.rgb.transfer_functions.arri_alexa_log_c colour.models.rgb.transfer_functions.aces colour.models.rgb.transfer_functions.common colour.colorimetry.datasets.light_sources.chromaticity_coordinates colour.colorimetry.datasets.illuminants.tristimulus_values colour.colorimetry.datasets.illuminants.hunterlab colour.colorimetry.datasets.illuminants.chromaticity_coordinates colour.continuous.abstract colour.volume.datasets.optimal_colour_stimuli colour.notation.datasets.munsell colour.models.rgb.prismatic colour.models.rgb.cmyk colour.models.rgb.cylindrical colour.io.luts.sequence colour.blindness.datasets.machado2010 colour.biochemistry.michaelis_menten colour.algebra.coordinates.transformations colour.adaptation.datasets.cat colour.temperature.robertson1968 colour.temperature.krystek1985 colour.temperature.kang2002 colour.contrast.barten1999 colour.models.cie_ucs colour.io.ocio colour.io.image colour.geometry.primitives colour.difference.stress colour.difference.huang2015 colour.algebra.random colour.algebra.interpolation colour.algebra.geometry colour.colorimetry.yellowness colour.colorimetry.whiteness colour.volume.datasets colour.notation.datasets colour.blindness.datasets colour.biochemistry colour.algebra.coordinates colour.adaptation.datasets colour.contrast colour.algebra.extrapolation colour.algebra colour.models.rgb.transfer_functions.srgb colour.models.rgb.transfer_functions.smpte_240m colour.models.rgb.transfer_functions.rimm_romm_rgb colour.models.rgb.transfer_functions.itur_bt_2020 colour.models.rgb.transfer_functions.itur_bt_601 colour.models.rgb.transfer_functions.filmic_pro colour.models.rgb.transfer_functions.dcdm colour.models.rgb.transfer_functions.gamma colour.models.rgb.ycocg colour.models.rgb.hanbury2003 colour.models.rgb.transfer_functions.st_2084 colour.io.luts.operator colour.io.luts.lut colour.continuous.multi_signals colour.continuous.signal colour.continuous colour.characterisation.correction colour.models.oklab colour.models.ipt colour.models.igpgtg colour.models.cam02_ucs colour.models.common colour.geometry.section colour.difference.delta_e colour.appearance.llab colour.appearance.hke colour.appearance.atd95 colour.appearance.hunt colour.colorimetry.luminance colour.colorimetry.lightness colour.adaptation.zhai2018 colour.adaptation.cie1994 colour.adaptation.cmccat2000 colour.adaptation.fairchild1990 colour.adaptation.vonkries colour.adaptation colour.models.rgb.transfer_functions.sony_slog colour.models.rgb.transfer_functions.red_log colour.models.rgb.transfer_functions.nikon_nlog colour.models.rgb.transfer_functions.fujifilm_flog colour.models.rgb.transfer_functions.panasonic_vlog colour.models.rgb.transfer_functions.itur_bt_2100 colour.models.rgb.transfer_functions.itur_bt_709 colour.models.rgb.transfer_functions.canon_log colour.models.rgb.transfer_functions.arib_std_b67 colour.models.rgb.transfer_functions colour.models.cam16_ucs colour.io.luts.cinespace_csp colour.io.luts.sony_spimtx colour.io.luts.sony_spi3d colour.io.luts.sony_spi1d colour.io.luts.resolve_cube colour.io.luts.iridas_cube colour.io.luts colour.difference.cam02_ucs colour.appearance.rlab colour.colorimetry.spectrum colour.geometry.vertices colour.geometry colour.colorimetry.datasets.light_sources.sds colour.colorimetry.datasets.illuminants.sds colour.recovery.datasets.smits1999 colour.models.rgb.ycbcr colour.models.jzazbz colour.models.icacb colour.difference.cam16_ucs colour.colorimetry.datasets.light_sources colour.models.rgb.datasets.xtreme_rgb colour.models.rgb.datasets.panasonic_v_gamut colour.models.rgb.datasets.srgb colour.models.rgb.datasets.sony colour.models.rgb.datasets.ntsc colour.models.rgb.datasets.smpte_c colour.models.rgb.datasets.smpte_240m colour.models.rgb.datasets.sharp colour.models.rgb.datasets.russell_rgb colour.models.rgb.datasets.rimm_romm_rgb colour.models.rgb.datasets.red colour.models.rgb.datasets.pal_secam colour.models.rgb.datasets.nikon_n_gamut colour.models.rgb.datasets.max_rgb colour.models.rgb.datasets.itur_bt_2020 colour.models.rgb.datasets.itur_bt_709 colour.models.rgb.datasets.itur_bt_470 colour.models.rgb.datasets.gopro colour.models.rgb.datasets.filmlight_egamut colour.models.rgb.datasets.fujifilm_f_gamut colour.models.rgb.datasets.ekta_space_ps5 colour.models.rgb.datasets.eci_rgb_v2 colour.models.rgb.datasets.dji_dgamut colour.models.rgb.datasets.don_rgb_4 colour.models.rgb.datasets.p3_d65 colour.models.rgb.datasets.display_p3 colour.models.rgb.datasets.dci_p3 colour.models.rgb.datasets.dcdm_xyz colour.models.rgb.datasets.davinci_wide_gamut colour.models.rgb.datasets.color_match_rgb colour.models.rgb.datasets.canon_cinema_gamut colour.models.rgb.datasets.cie_rgb colour.models.rgb.datasets.blackmagic_design colour.models.rgb.datasets.beta_rgb colour.models.rgb.datasets.best_rgb colour.models.rgb.datasets.arri_alexa_wide_gamut colour.models.rgb.datasets.apple_rgb colour.models.rgb.datasets.adobe_wide_gamut_rgb colour.models.rgb.datasets.adobe_rgb_1998 colour.models.rgb.datasets.aces colour.colorimetry.datasets.illuminants.sds_d_illuminant_series colour.models.rgb.ictcp colour.models.rgb.common colour.models.rgb.datasets colour.models.rgb.rgb_colourspace colour.models.rgb.derivation colour.models.datasets.pointer_gamut colour.colorimetry.datasets.lefs colour.colorimetry.datasets.illuminants colour.colorimetry.datasets.cmfs colour.models.rgb colour.models.datasets colour.models.prolab colour.models.osa_ucs colour.models.hdr_ipt colour.models.hunter_rdab colour.models.hunter_lab colour.models.hdr_cie_lab colour.models.din99 colour.models.cie_uvw colour.models.cie_luv colour.models.cie_lab colour.models.cie_xyy colour.colorimetry.transformations colour.colorimetry.photometry colour.colorimetry.dominant colour.colorimetry.lefs colour.colorimetry.illuminants colour.colorimetry.correction colour.colorimetry.uniformity colour.colorimetry.tristimulus_values colour.colorimetry.generation colour.colorimetry.datasets colour.colorimetry.cmfs colour.colorimetry.blackbody colour.models colour.colorimetry colour.characterisation.datasets.lenses.sds colour.characterisation.datasets.filters.sds colour.characterisation.datasets.colour_checkers.sds colour.characterisation.datasets.colour_checkers.chromaticity_coordinates colour.plotting.datasets.astm_g_173 colour.recovery.datasets.otsu2018 colour.recovery.datasets.mallett2019 colour.quality.datasets.vs colour.quality.datasets.tcs colour.characterisation.displays colour.characterisation.cameras colour.temperature.ohno2013 colour.temperature.mccamy1992 colour.temperature.hernandez1999 colour.temperature.cie_d colour.recovery.meng2015 colour.quality.ssi colour.notation.hexadecimal colour.phenomena.rayleigh colour.io.xrite colour.io.tm2714 colour.io.tabular colour.difference.din99 colour.appearance.nayatani95 colour.appearance.ciecam02 colour.volume.rgb colour.volume.spectrum colour.volume.pointer_gamut colour.volume.macadam_limits colour.volume colour.corresponding.prediction colour.corresponding colour.characterisation.datasets.lenses colour.characterisation.datasets.filters colour.characterisation.datasets.colour_checkers colour.plotting.datasets colour.recovery.datasets colour.quality.datasets colour.appearance.zcam colour.appearance.kim2009 colour.appearance.cam16 colour.temperature colour.notation.munsell colour.notation colour.phenomena colour.io.uprtek_sekonic colour.io colour.difference colour.recovery.jakob2019 colour.quality.cqs colour.quality.cri colour.characterisation.datasets.displays.lcd.primaries colour.characterisation.datasets.displays.crt.primaries colour.characterisation.datasets.cameras.dslr.sensitivities colour.characterisation.datasets.displays.lcd colour.characterisation.datasets.displays.crt colour.characterisation.datasets.cameras.dslr colour.characterisation.datasets.displays colour.characterisation.datasets.cameras colour.characterisation.datasets.aces_it colour.characterisation.aces_it colour.characterisation.datasets colour.characterisation colour.appearance colour.plotting.common colour.quality.cfi2017 colour.recovery.smits1999 colour.recovery.otsu2018 colour.recovery.mallett2019 colour.recovery colour.blindness.machado2009 colour.blindness colour.quality.tm3018 colour.quality colour.graph.conversion colour.graph colour.plotting.graph colour colour.plotting.tm3018.components colour.plotting.tm3018.report colour.plotting.volume colour.plotting.tm3018 colour.plotting.temperature colour.plotting.section colour.plotting.quality colour.plotting.phenomena colour.plotting.notation colour.plotting.models colour.plotting.corresponding colour.plotting.diagrams colour.plotting.characterisation colour.plotting.colorimetry colour.plotting.blindness colour.plotting) as inherently stale with stale deps (colour.algebra.regression colour.constants colour.corresponding.datasets colour.models.datasets.macadam_ellipses colour.notation.datasets.munsell.all colour.utilities.documentation colour.volume.mesh)

I have no idea what an SCC is and whether one with size 302 is good or not, but it's by far the biggest reported. Maybe it helps someone to understand what is going on.

[bersbersbers]

I ran the following code on two systems across three installations (Windows 10 Python 3.10, Linux Python 3.9 and 3.10):

time mypy --show-error-codes --warn-unused-ignores --warn-redundant-casts --install-types --non-interactive -p colour

Windows finished in close to 9 minutes (Core i7-9750H), Linux in 22 and 23 minutes (Core i9-7900X). All color-science 0.4.1, mypy 0.942.

So it's not "forever", but still very long (and when using colour-science in my package, mypy run times can exceed an hour.)

[bersbersbers]

@KelSolaar what do you think would be good way to typecheck the whole package for comparison purposes?

The following does not work with 0.3.16:

$ mypy -p colour
Can't find package 'colour'

I also tried this, which does not work, either:

$ mypy -c "import colour"
<string>:1: error: Skipping analyzing "colour": module is installed, but missing library stubs or py.typed marker
<string>:1: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
Found 1 error in 1 file (checked 1 source file)

(but this explains why I saw such drastic runtime increases with colour-science==0.4.1: mypy maybe was never doing much type checking before, so 0.3.1 is not a useful baseline for comparison at all.)

What else can I provide to find out what makes mypy spend dozens of minutes on colour-science? Is https://github.com/python/mypy/issues/12408#issuecomment-1080734519 helpful? (Do these files contain any private information such as IPs, host names etc.?) Anything else?

[KelSolaar]

The released Colour 0.3.16 does not have any typing support at all, it was only added to 0.4.x. I have personally not experienced any long type checking times locally but they do vary on CI though: https://github.com/colour-science/colour/actions/workflows/continuous-integration-static-type-checking.yml

[bersbersbers]

they do vary on CI though: https://github.com/colour-science/colour/actions/workflows/continuous-integration-static-type-checking.yml

Are you sure? I checked couple of jobs that took between 12min and 1h34min, and the actual type checking always takes some 11-15 minutes - and that includes the pip install part:

So it doesn't look that bad on CI IMO, or am I missing some more mypy-specific timing?

[bersbersbers]

It hurts me to have to say that, but a useful workaround in pyproject.toml is

[[tool.mypy.overrides]]
module = ["colour.*"]
follow_imports = "skip"

I guess this skips type checking for colour which is a pity, but it's better than having no type checking at all.

[JukkaL]

I profiled a mypy run targeting colour and it seems likely that most of the time is spent on operations involving large union types. There are mypy operations on union types that are O(n**2), which has been known to cause slow type checking. It's probably not too difficult to make this better. We already have some related optimizations, but clearly they don't seem to help here.

[JukkaL]

The latest GitHub master seems to be marginally faster with colour-science than 0.942 (around 20%). I did some prototyping, and caching the results of make_simplified_union if there are over, say, 4 items in the input might speed up mypy -p colour by as much as 3x (there is a lot of uncertainty about this, though).

Caching would be somewhat non-trivial to implement properly, instead of just prototyping. Some of our current __hash__ implementations in mypy.types probably aren't quite right. Also, we'd need to be careful to avoid growing the cache without an upper bound.

[bersbersbers]

time mypy --show-error-codes --warn-unused-ignores --warn-redundant-casts --install-types --non-interactive -p colour

Windows finished in close to 9 minutes (Core i7-9750H), Linux in 22 and 23 minutes (Core i9-7900X). All color-science 0.4.1, mypy 0.942.

With mypy 0.950, Windows is down to less than 5 minutes, and Linux is down to 8 minutes. This is much better already!

[hauntsaninja]

Thanks for reporting! mypy 0.960 made several further improvements to perf. We have #12526 as an issue to track improvements to union simplification, so closing this out

[hauntsaninja]

I added colour-science to mypy_primer today, so regressions on the codebase for pyright or mypy will now be flagged.

I noticed that colour-science has switched to pyright, but latest mypy runs about 4x faster than pyright does on colour-science, so wondering if I'm missing something.

Edit: nice, looks like posting this got Eric to make pyright faster! https://github.com/microsoft/pyright/pull/9115 :-)