pandas.libs not copied when pandas package is listed

[Rob-Hulley]

I have upgraded to python 3.11.5 and running pynsist 2.8 and pandas 2.1.0 pip install pandas now create two folders pandas pandas.libs (has the windows dll file msvcp140-59fdf63e48138046aebeb6ddb5b4e960.dll)

pandas.libs is not copied across. if I add pandas.libs to the package section I get a key error for pandas.

I used this workaround to get it copied:

Other files and folders that should be installed

files = lib C:\Program Files\Python311\Lib\site-packages\pandas.libs > $INSTDIR/pkgs

Is there any better way to do this?

Thanks

Rob

pandas pandas-2.1.0.dist-info pandas.libs

[Rob-Hulley]

I now have the same issue with numpy after updating to version 1.26.0 Older version had the .libs as a subfolder. The RECORD file is different for the two versions. Have pandas and numpy installer made a mistake, or is this the new way forward.

eg: 1.25.2 numpy-1.25.2.dist-info/top_level.txt,sha256=4J9lbBMLnAiyxatxh8iRKV5Entd_6-oqbO7pzJjMsPw,6 numpy/.libs/libopenblas64__v0.3.23-246-g3d31191b-gcc_10_3_0.dll,sha256=h1zxwNoqsy37O71f1cIatss1Z55NPAqfX8vnDtvL6Wk,38166567

1.26.0 numpy-1.26.0.dist-info/entry_points.txt,sha256=zddyYJuUw9Uud7LeLfynXk62_ry0lGihDwCIgugBdZM,144 numpy.libs/libopenblas64__v0.3.23-293-gc2f4bdbb-gcc_10_3_0-65e29aac85b9409a6008e2dc84b1cc09.dll,sha256=7A4P0sKw5BzY-9HJHVGFxvAMPRhJyscxBvLVoxoB56k,38168576

pandas is the same. pandas-2.1.0.dist-info/entry_points.txt,sha256=OVLKNEPs-Q7IWypWBL6fxv56_zt4sRnEI7zawo6y_0w,69 pandas.libs/msvcp140-59fdf63e48138046aebeb6ddb5b4e960.dll,sha256=7FK3VWFW3K6unFY4IOM-PV5H1NFinMeJpX1YtxDYoPw,621960

numpy-1.25.2.dist-info.zip numpy-1.26.0.dist-info.zip

[takluyver]

Have pandas and numpy installer made a mistake, or is this the new way forward.

I'm guessing it's the new way - probably a change in some of the tooling to create wheels. I don't know if or why it's better than the old way. :shrug:

For Pynsist, the easiest way to handle this is to list these packages in pypi_wheels in the config file rather than packages. This finds the Windows builds on PyPI and repacks their contents into your installer, which should include the 'libs' folders. packages works by finding importable packages, and the pandas.libs folder is outside the importable pandas package.

If you prefer not to use wheels from PyPI for any reason, you can make wheels yourself and use the extra_wheels_sources or local_wheels options to include them, or you can drop all the files you want in a pynsist_pkgs folder next to your config file.

[Rob-Hulley]

Thanks Thomas,

Using the wheels works great.

Thanks for your help.

Rob

[Include]
 # Packages from PyPI that your application requires, one per line
 # These must have wheels on PyPI:
pypi_wheels:
 pywin32==306
 numpy==1.26.0
 pandas==2.1.0
 tables==3.8.0

[takluyver]

No problem! I think getting packages from wheels is generally more reliable than the older packages option, so long as the packages you want have wheels available (if they don't, see the FAQ).

I'll close this now - it's a shame that this change has made the packages option less useful, but I can't think of any straightforward way to fix it, and hopefully using wheels should be a better option in most cases.