C-language compiled enhancement trial throws ImportError on running selftest.py

[michiel5342]

What did you do?

I was trying to implement an imho trivial enhancement of the ModeFilter, which I would find useful for my use-case: despeckling of geography map scans. The enhancement consists of adding a second int arg: numpix, which decides how many same pixel values exist in the size box. This is hard coded as 2 in the existing filter, and could be altered up to 8 to prevent the filter obliterating map details. A value of 8 should only remove isolated pixels.

What did you expect to happen?

I expected a working and testable ModeFilter and Imaging library

What actually happened?

My code compiled all-right, but on running selftest.py I got an Import error saying the extension library did not have a PILLOW_VERSION attribute, and so the Pillow library cannot be imported.

What are your OS, Python and Pillow versions?

• OS: Ubuntu 18.04.5 LTS
• Python: 3.6.9 (default, Jan 26 2021, 15:33:00) [GCC 8.4.0]
• Pillow: 8.3.0.dev0

Diff output of changes:

--- _imaging.c  2021-05-23 16:18:38.000000000 +0200
+++ ../src/_imaging.c   2021-05-24 17:34:29.635831165 +0200
@@ -64,6 +64,7 @@
  * 2004-10-04 fl   Added modefilter
  * 2005-10-02 fl   Added access proxy
  * 2006-06-18 fl   Always draw last point in polyline
+ * 2021-05-24 mr   Added numpix arg to ModeFilter
  *
  * Copyright (c) 1997-2006 by Secret Labs AB
  * Copyright (c) 1995-2006 by Fredrik Lundh
@@ -1312,12 +1313,14 @@
 #ifdef WITH_MODEFILTER
 static PyObject *
 _modefilter(ImagingObject *self, PyObject *args) {
+    /* 2021-05-24 mr: added numpix arg */
     int size;
-    if (!PyArg_ParseTuple(args, "i", &size)) {
+    int numpix;
+    if (!PyArg_ParseTuple(args, "ii", &size, &numpix)) {
         return NULL;
     }

-    return PyImagingNew(ImagingModeFilter(self->image, size));
+    return PyImagingNew(ImagingModeFilter(self->image, size, numpix));
 }
 #endif

--- libimaging/Imaging.h    2021-05-23 16:18:38.000000000 +0200
+++ ../src/libImaging/Imaging.h 2021-05-24 17:22:46.751129166 +0200
@@ -331,8 +331,9 @@
 ImagingGetProjection(Imaging im, UINT8 *xproj, UINT8 *yproj);
 extern ImagingHistogram
 ImagingGetHistogram(Imaging im, Imaging mask, void *extrema);
+/* 2021-05-24 mr: added numpix arg */
 extern Imaging
-ImagingModeFilter(Imaging im, int size);
+ImagingModeFilter(Imaging im, int size, int numpix);
 extern Imaging
 ImagingNegative(Imaging im);
 extern Imaging

--- libimaging/ModeFilter.c 2021-05-23 16:18:38.000000000 +0200
+++ ../src/libImaging/ModeFilter.c  2021-05-24 17:20:26.070173338 +0200
@@ -7,6 +7,7 @@
  * history:
  * 2002-06-08 fl    Created (based on code from IFUNC95)
  * 2004-10-05 fl    Rewritten; use a simpler brute-force algorithm
+ * 2021-05-24 mr    Added numpix arg to fine tune same pixel nums
  *
  * Copyright (c) Secret Labs AB 2002-2004.  All rights reserved.
  *
@@ -16,7 +17,7 @@
 #include "Imaging.h"

 Imaging
-ImagingModeFilter(Imaging im, int size) {
+ImagingModeFilter(Imaging im, int size, int numpix) {
     Imaging imOut;
     int x, y, i;
     int xx, yy;
@@ -66,8 +67,8 @@
                     maxpixel = (UINT8)i;
                 }
             }
-
-            if (maxcount > 2) {
+            /* parametrize numpix instead of hard coded 2 */
+            if (maxcount > numpix) {
                 out[x] = maxpixel;
             } else {
                 out[x] = IMAGING_PIXEL_L(im, x, y);

--- PIL/ImageFilter.py  2021-05-23 16:18:38.000000000 +0200
+++ ../src/PIL/ImageFilter.py   2021-05-24 17:29:24.138492673 +0200
@@ -8,6 +8,7 @@
 # 1995-11-27 fl   Created
 # 2002-06-08 fl   Added rank and mode filters
 # 2003-09-15 fl   Fixed rank calculation in rank filter; added expand call
+# 2021-05-24 mr   Added numpix argument to ModeFilter
 #
 # Copyright (c) 1997-2003 by Secret Labs AB.
 # Copyright (c) 1995-2002 by Fredrik Lundh.
@@ -137,15 +138,17 @@
     pixel value occurs more than twice, the original pixel value is preserved.

     :param size: The kernel size, in pixels.
+    :param numpix: The number of same pixel values.
     """

     name = "Mode"

-    def __init__(self, size=3):
+    def __init__(self, size=3, numpix=2):
         self.size = size
+        self.numpix = numpix

     def filter(self, image):
-        return image.modefilter(self.size)
+        return image.modefilter((self.size, self.numpix))

 class GaussianBlur(MultibandFilter):

[wiredfool]

You need to build and install the entire package — make install should do it if you’re in a virtualenv, otherwise some combination of python setup.py build and however you do your installs.

There’s a version tag in the c extension added by setup that must match the python code, or it won’t load. This prevents people from running with all forms of mismatched python and c extensions.

[michiel5342]

I studied setup.py and its options. I then tried to run "python3 setup.py build develop --user" and, hey, "python3 selftest.py" succeeds. Here the result of running selftest.py:

michiel@odin:~/Source/python/Pillow-master$ python3 selftest.py 
--------------------------------------------------------------------
Pillow 8.3.0.dev0
Python 3.6.9 (default, Jan 26 2021, 15:33:00)
       [GCC 8.4.0]
--------------------------------------------------------------------
Python modules loaded from /home/michiel/Source/python/Pillow-master/src/PIL
Binary modules loaded from /home/michiel/Source/python/Pillow-master/src/PIL
--------------------------------------------------------------------
--- PIL CORE support ok, compiled for 8.3.0.dev0
--- TKINTER support ok, loaded 8.6
--- FREETYPE2 support ok, loaded 2.8.1
--- LITTLECMS2 support ok, loaded 2.9
--- WEBP support ok, loaded 0.6.1
--- WEBP Transparency support ok
--- WEBPMUX support ok
--- WEBP Animation support ok
--- JPEG support ok, compiled for libjpeg-turbo 1.5.2
--- OPENJPEG (JPEG2000) support ok, loaded 2.2.0
--- ZLIB (PNG/ZIP) support ok, loaded 1.2.11
--- LIBTIFF support ok, loaded 4.0.9
*** RAQM (Bidirectional Text) support not installed
*** LIBIMAGEQUANT (Quantization method) support not installed
--- XCB (X protocol) support ok
--------------------------------------------------------------------
Running selftest:
--- 58 tests passed.

Nevertheless, after running "python3 setup.py install --user", which finished OK, running "python3 selftest.py" still gave the error.

After some hopeless fiddling I studied how selftest.py was using the built library, which was from the src/PIL project directory. That contained the built extension libs, which selftest.py found OK. So I decided to do an awful kludge, which I use to "install" my own projects: I copied the whole PIL directory, with python files and *.so's into my home local python lib, and look at that, my enhancement is running and testable ! I still have no idea, why setup does not succeed in installing. Anyhow, "works for me" so I am closing this issue. Please feel free to use the enhancement if you find it useful. My first testing results are very promising.

[wiredfool]

Develop is a misfeature for complicated projects, because it pollutes the paths and pulls in items which may or may not be part of the actual installed package.

I think you can make a bdist_wheel (to get a binary wheel) and then install that as a —user or otherwise.