pip, cythonize nphusl on macos with clang ?

[denis-bz]

Tad, can you help me install nphusl with Cython on macos with clang ?

pip install nphusl

Collecting nphusl
  Using cached nphusl-1.4.1.tar.gz (63 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: numpy in /opt/local/py3/lib/python3.9/site-packages (from nphusl) (1.22.3)
Building wheels for collected packages: nphusl
  Building wheel for nphusl (setup.py): started
  Building wheel for nphusl (setup.py): finished with status 'error'
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [19 lines of output]
      /opt/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/dist.py:260: UserWarning: 'licence' distribution option is deprecated; use 'license'
        warnings.warn(msg)
      running bdist_wheel
      running build
      running build_py
      creating build
      ...
      /usr/bin/clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -g -O3 -Wno-format-security -I/opt/local/py3/lib/python3.9/site-packages/numpy/core/include -arch x86_64 -I/opt/local/py3/lib/python3.9/site-packages/numpy/core/include -I/Library/Frameworks/Python.framework/Versions/3.9/include/python3.9 -c nphusl/_nphusl_cython.c -o build/temp.macosx-10.9-universal2-3.9/nphusl/_nphusl_cython.o -fopenmp -O3 -ffast-math
      clang: error: unsupported option '-fopenmp'
      error: command '/usr/bin/clang' failed with exit code 1

Ok, comment out -fopenmp and -ftree-vectorizer-verbose=2 in setup.py then

python3 setup.py build_ext --cythonize

# from: python3 setup.py build_ext --cythonize
# run: 11 Mar 2022 13:16  in ~bz/py/color/husl/husl-numpy-master  Denis-iMac  mac 10.14.6 i386 

.../husl-numpy-master/nphusl/nphusl.py:66: UserWarning: No Cython extension module: cannot import name '_cython_opt' from partially initialized module 'nphusl' (most likely due to a circular import) (/opt/local/py/_plot/husl-numpy-master/nphusl/__init__.py)
  warnings.warn("No Cython extension module: {}".format(e))
.../husl-numpy-master/nphusl/nphusl.py:71: UserWarning: No SIMD extension module: cannot import name '_simd_opt' from partially initialized module 'nphusl' (most likely due to a circular import) (/opt/local/py/_plot/husl-numpy-master/nphusl/__init__.py)
  warnings.warn("No SIMD extension module: {}".format(e))
Application options:
{'CYTHONIZE': True,
 'INTERPOLATE_CHROMA': False,
 'NO_CHROMA_LUT': False,
 'NO_CYTHON_EXT': False,
 'NO_HUE_ATAN2_APPROX': False,
 'NO_LIGHT_LUT': False,
 'NO_SIMD_EXT': False}
running build_ext
building 'nphusl._simd_opt' extension
/usr/bin/clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -g -O3 -Wno-format-security -I/opt/local/py3/lib/python3.9/site-packages/numpy/core/include -arch x86_64 -Inphusl -Inphusl/ -I/opt/local/py3/lib/python3.9/site-packages/numpy/core/include -I/Library/Frameworks/Python.framework/Versions/3.9/include/python3.9 -c nphusl/_chroma_lookup.c -o build/temp.macosx-10.9-universal2-3.9/nphusl/_chroma_lookup.o -ftree-vectorize -std=c99 -mtune=native -O3 -ffast-math -DUSE_LIGHT_LUT -DUSE_CHROMA_LUT -DUSE_HUE_ATAN2_APPROX
clang: error: no such file or directory: 'nphusl/_chroma_lookup.c'
clang: error: no input files
error: command '/usr/bin/clang' failed with exit code 1

Versions: Python 3.9.9 cython 0.29.28 Apple clang 11.0.0 pip 22.0.4

Bytheway there's another husl in https://github.com/mwaskom/seaborn/blob/master/seaborn/external/husl.py any comments ?

Thanks, cheers -- denis

[TadLeonard]

Hey Denis,

Thanks for writing this. I haven't looked at this project in a long time and it was fun getting it working again.

I wrote this package with Linux in mind, but these days I'm using a macbook. Just like you, I'm finding it's not really Mac friendly. In a separate project I've had a difficult time getting OpenMP to work well enough with Clang. It's definitely possible to get those -fopenmp compile args working on your mac, but it takes a lot more work than it should.

You can get this project working mostly in its current state by omitting some compile args and turning off some options with the flags setup.py provides. I made some changes to help you get started in a separate branch.

Here's a list of things I think are going wrong for you to keep my thoughts organized:

• Like you already explained, there are some compile arguments that are (in your system's current state) incompatible with clang
• I left some old cruft in the setup.py file that doesn't really belong anymore. The error you're getting is for a missing lookup table C file of some kind, but I moved on from that at some point when I found SIMD-optimized (or OpenMP-optimized, I can't remember) to be a lot faster than hitting a lookup table for those particular values. So that's too bad. I could probably restore that but I think the project really just needs to be cleaned up. Luckily you can bypass that by providing the project-specific setup arg --no-chroma-lut.
• If you start from a pretty simple MacOS setup like I did in testing this, you might find a few other things need to be turned off. I disabled the atan2 approximation, for example, and the lightness value lookup table. The final compile command I landed on looks like this: python setup.py build_ext --cythonize --no-chroma-lut --no-hue-atan2-approx --no-light-lut --inplace. You can omit --inplace depending on how you're trying to install things.
• Just to convince myself that things were working I got the example code working. You'll see in the branch a few small changes to get example.py to work as it should (with imageio instead of imread, which isn't in dev-requirements.txt.

When I run the tests with these changes I get two failures for two tests which are frankly not well written and aren't a huge concern. So the behavior of the package is fine with these changes. HOWEVER, disabling these optimizations and especially not compiling with OpenMP is a shame. It's a lot slower than it should be, as I can only generate 5 FPS in my "melonize" example given a ~1080p input image where I'd usually expect something closer to 24 FPS. I'd revisit this project to make the build process more portable and to improve performance on more platforms if I thought people were out there using it, but I think this project fits a really specific niche that makes that unlikely.

Bytheway there's another husl in https://github.com/mwaskom/seaborn/blob/master/seaborn/external/husl.py any comments ?

That code comes from another old project which I am familiar with. I worked with it to validate this project in its early stages. The goals of these two projects are pretty different. nphusl is supposed to work with images, whereas I get the sense that most people are using the HuSL color space to automatically generate color palettes and things like that. Basically, when I tried to use the existing HuSL libraries to do image processing with this fancy color space (i.e. the stuff in the README) I found that they were much too slow. It's not really the fault of those projects it's more that they weren't designed to convert millions of color values back and forth between the HuSL and RGB color spaces. Also this project sacrifices a (small) amount of precision to make things go faster.

Hope that helps!

[denis-bz]

Tad, that does the trick, thanks ! (lovely clean code, I have a lot to learn)

A general? problem is that max_chroma( L, h ) looks way too high, colorspacious( [L, max_chroma( L, h ), h], "CIELCh", "sRGB1" ) can have rgb < -5 but roundtrip LCh -> sRGB1 -> LCh works ??

17mar: deletd yesterday's silly plot

NOT an nphusl issue, but it seems that nphusl lch and colorspacious LCh JCh -> 3 different chromas ? Just running the corners of the RGB cube gives

rgb: (8, 3)
[[  0   0   0   0 255 255 255 255]
 [  0   0 255 255   0   0 255 255]
 [  0 255   0 255   0 255   0 255]]

{ nphusl__rgb_to_lch  max 179 at [1 0 0]
[[  0  32  88  91  53  60  97 100]
 [  0 130 135  72 179 137 107   0]
 [  0 266 128 192  12 308  86  20]]
}

{ cspace_rgb_LCh  max 133 at [0 0 1]
[[  0  32  88  91  53  60  97 100]
 [  0 133 119  50 104 115  96   0]
 [  0 306 136 196  40 328 103 302]]
}

{ cspace_rgb_JCh  max 111 at [1 0 0]
[[  0  21  80  84  47  54  96 100]
 [  0  90 106  59 111  97  80   3]
 [  0 258 137 198  32 330 106 211]]
}

sure confusing for unwary people (me) who expect them to be the same.

"Standard" definition of Chroma ? https://xkcd.com/927/