scandir 1.9.0 is broken on openindiana

[ThomasWaldmann]

$ vagrant ssh openindiana64
The Illumos Project     SunOS 5.11      illumos-a942f1f5f0      August 2018

(borg-env) vagrant@openindiana:/vagrant/borg$ python
Python 3.4.7 (default, Feb 11 2018, 23:05:03) 
[GCC 6.4.0] on sunos5
Type "help", "copyright", "credits" or "license" for more information.

>>> from scandir import *
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/vagrant/borg/borg-env/lib/python3.4/site-packages/scandir.py", line 591, in <module>
    DirEntry = PosixDirEntry
NameError: name 'PosixDirEntry' is not defined

(borg-env) vagrant@openindiana:/vagrant/borg$ pip freeze|grep scandir
scandir==1.9.0

(borg-env) vagrant@openindiana:/vagrant/borg$ pip install 'scandir<1.9.0'
...

(borg-env) vagrant@openindiana:/vagrant/borg$ python
Python 3.4.7 (default, Feb 11 2018, 23:05:03) 
[GCC 6.4.0] on sunos5
Type "help", "copyright", "credits" or "license" for more information.
>>> from scandir import *
(no traceback)

[benhoyt]

Hmm, interesting. I'm not sure how this broke with 1.9.0 -- it looks like this problem has been there for a long time. It's caused because sys.platform is sunos5 on this OS, and PosixDirEntry is defined if ctypes is not None and have_dirent_d_type, but used if ctypes is not None without the extra clause.

However, can you please tell me what is the version of scandir installed when it works, i.e., the scandir<1.9.0 in your case?

[ThomasWaldmann]

It installed 1.8.0.

[ThomasWaldmann]

I found this while doing borgbackup testing and resurrecting the openindiana vagrant box now as the vagrant change that broke oi boxes was found. I'll try to reproduce this with a clean virtualenv in a minute...

[ThomasWaldmann]

In a clean python 3.4.7 virtual env:

started at 1.5.0, upgrading scandir via pip install scandir==V for V in 1.6.0, 1.7.0, 1.8.0, 1.9.0.

it definitely does not traceback with <= 1.8.0, but tracebacks after upgrading to 1.9.0.

[ThomasWaldmann]

OK, guess I have the reason why it crashes:

• if the c extension is built / available, it does not crash
• so there is a different behaviour for [not] building the c ext on openindiana since 1.9.0 when the package is normally installed from pypi
• if the c extension is not available, it crashes due to the bad 2nd if-condition that does not match the 1st one. this bug was there since long, but it did not get noticed as building the c ext usually worked in < 1.9.0. fixed by #111.

[ThomasWaldmann]

OK, guess I have why the C ext building was broken:

Because vagrant did not support openindiana for a while and was just recently fixed, my openindiana box was a bit outdated. Some stuff (e.g. current Cython, maybe also scandir?) now wanted gcc-6 which somehow was not there (yet). So my first tries to build that stuff failed when it tried to compile C stuff - but I guess pip cached a wheel for scandir 1.9.0 made at that time (and thus: without _scandir C extension).

Later, i fixed by updating the system and also installing gcc-6.

But, as pip had a cached wheel for scandir 1.9.0, it continued using this and thus missed the C extension.

I killed ~/.cache/pip later and now a freshly pip-installed scandir 1.9.0 does have the C extension and thus does not trigger the exception any more.

So seems like this rather was a caching issue than a build problem rooted in scandir setup code.

But at least we found a bug, so please merge #111.

[benhoyt]

Excellent, thanks. I just merged #111.