cross-compiling python 3.13 readline error - unable to find ncurses.pc while ncursesw.pc is available

[th0ma7]

Bug report

Bug description:

python package maintainer for the SynoCommunity. Preparing the migration path from our python 3.11 to 3.13 but the installer changed since previous version, no longer using setup.py.

Long-story short, the configure script finds ncursesw properly, altough testing for readline later on fails as only assuming ncurses.pc (vs the w variant).

error from our logs:

The necessary bits to build these optional modules were not found:
_tkinter                  readline                                         
To find the necessary bits, look in configure.ac and config.log.

Checked 112 modules (33 built-in, 61 shared, 1 n/a on linux-x86_64, 15 disabled, 2 missing, 0 failed on import)
make[4]: Leaving directory '/home/spksrc/python-wheels/spksrc/spk/python313/work-x64-7.1/Python-3.13.0'
===>  Installing for python313
find /home/spksrc/python-wheels/spksrc/spk/python313/work-x64-7.1/install//var/packages/python313/target/../ \! -type d -printf '%P\n' | sed 's?^target/??g' | sort > /home/spksrc/python-wheels/spksrc/spk/python313/work-x64-7.1/python313.plist.tmp

I've been digging into the configure.ac in hope to find why but no luck just yet... configure detection of ncursesw:

checking for ncursesw... yes
checking for ncursesw/curses.h... no
checking for ncursesw/ncurses.h... no
checking for ncursesw/panel.h... no
checking for ncurses/curses.h... no
checking for ncurses/ncurses.h... no
checking for ncurses/panel.h... no
checking for curses.h... yes
checking for ncurses.h... yes
checking for curses function is_pad... yes
checking for curses function is_term_resized... yes
checking for curses function resize_term... yes
checking for curses function resizeterm... yes
checking for curses function immedok... yes
checking for curses function syncok... yes
checking for curses function wchgat... yes
checking for curses function filter... yes
checking for curses function has_key... yes
checking for curses function typeahead... yes
checking for curses function use_env... yes
checking for stdlib extension module _curses... yes
checking for stdlib extension module _curses_panel... yes

I presume relatively simple to fix, but haven't found how just yet.

CPython versions tested on:

3.13

Operating systems tested on:

Linux

And sections from the config.log

configure:24789: checking for readline
configure:24796: $PKG_CONFIG --exists --print-errors "readline"
Package ncurses was not found in the pkg-config search path.
Perhaps you should add the directory containing `ncurses.pc' 
to the PKG_CONFIG_PATH environment variable
Package 'ncurses', required by 'readline', not found
configure:24799: $? = 1
configure:24813: $PKG_CONFIG --exists --print-errors "readline"
Package ncurses was not found in the pkg-config search path.
Perhaps you should add the directory containing `ncurses.pc'
to the PKG_CONFIG_PATH environment variable
Package 'ncurses', required by 'readline', not found
configure:24816: $? = 1
configure:24830: result: no
Package 'ncurses', required by 'readline', not found

[hacscred]

Linux is not an operating system. An example of an operating system is Debian Woody.

It's a rather bad bug report, since we just need to believe you in that it works in 3.11 without proof.

[th0ma7]

Hey, nice to meet you too @hacscred! My latest PR relatively to python packaging is https://github.com/SynoCommunity/spksrc/pull/6200 Our python packages are available here: https://synocommunity.com/package/python311 (3.10+3.11 package builds undergoing+publishing)

Now, Synology is a linux distribution called DSM. Their toolchain are made available on which since years we've develop a framework to build from source and make available open sources packages through our repository that you can then install on your Synology NAS. We support cross-compiling from multiple archs, armv5-7-8, ppc, i686 and x86_64.

In hope to remediate at this, here's an early draft of my next PR https://github.com/SynoCommunity/spksrc/pull/6282. I had not yet intented to push that online, but in hope this helps subsequent discussions.

[th0ma7]

Adding, here the build log output (kept the python part only): build-x64-7.1.log And here is the config.log

Just le me know if you need anything else.

[hacscred]

I don't know whether this is Python's fault or yours, but this is not professional.

configure: WARNING: unrecognized options: --with-system-ffi

In general, the way to debug this is to just isolate the part of configure.ac in which the code is generated die the readline tests. That requires no intelligence and probably an AI could do that.

Then, it's a matter of figuring out whether readline ever even looks for ncursesw, which you claim it did in some other version. You could diff the relevant parts of configure.ac and possibly the indirections of the possible M4 files.

Then you will either find some mismatch between the library parts where you instructed it to look and where it actually looks. Diff those too.

If you have done all of that, you will only need to actually change the test or your input. Easy.

It's easy. If this doesn't work in nixpkgs, you could also report it there.

https://roguebasin.com/index.php/Ncursesw documents all the requirements. If all of them are ostensibly met and it still doesn't work, report back.

In nixpkgs the binary packaging of ghc configures ncursesw for various platforms, probably with similar as you have (commercial distribution).

I am not certain of whether ncursesw can be used as a drop in replacement in Nix, but adding support seems easy.

[ZeroIntensity]

Apologies @th0ma7 for the nuisance, typically people are nicer on here

[th0ma7]

I can confirm that the bug isn't with python but rather with readline. As readline is compiled against ncursesw only, there is an issue in its resulting readline.pc pkg-config file whereas it depends to ncurses instead of ncursesw.

Now fixed with a local build patch: https://github.com/SynoCommunity/spksrc/pull/6282/commits/9c764a7e729517ed04a633ada0f5c933f308e227

Note that this patch will probably be moved to its own PR due to numerous dependencies, thus history may get broken for future reference.

And thnx @ZeroIntensity for intervening, civility being much appreciated.