Update macOS installer builds to use ncurses 6.3

[ned-deily]

BPO	46976
Nosy	@ronaldoussoren, @ned-deily

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = 'https://github.com/ned-deily' closed_at = None created_at = labels = ['OS-mac', '3.9', '3.10', '3.11'] title = 'Update macOS installer builds to use ncurses 6.3' updated_at = user = 'https://github.com/ned-deily' ``` bugs.python.org fields: ```python activity = actor = 'ned.deily' assignee = 'ned.deily' closed = False closed_date = None closer = None components = ['macOS'] creation = creator = 'ned.deily' dependencies = [] files = [] hgrepos = [] issue_num = 46976 keywords = [] message_count = 2.0 messages = ['414869', '414874'] nosy_count = 2.0 nosy_names = ['ronaldoussoren', 'ned.deily'] pr_nums = [] priority = 'high' resolution = None stage = 'needs patch' status = 'open' superseder = None type = None url = 'https://bugs.python.org/issue46976' versions = ['Python 3.9', 'Python 3.10', 'Python 3.11'] ```

[ned-deily]

The python.org macOS installers include a private copy of the ncurses library; it has not been updated from 5.9 in a long time. The current upstream version is 6.3 and includes bug and security fixes; we should update to it.

[ned-deily]

Assigning to myself as this will require some installer build testing.

[vstinner]

The python.org macOS installers include a private copy of the ncurses library; it has not been updated from 5.9 in a long time. The current upstream version is 6.3 and includes bug and security fixes; we should update to it.

$ grep ncurses Mac/BuildScript/build-installer.py
              url="http://ftp.gnu.org/pub/gnu/ncurses/ncurses-5.9.tar.gz",
                  ("ftp://ftp.invisible-island.net/ncurses//5.9/ncurses-5.9-20120616-patch.sh.bz2",

Ah right, ncurses 5.9 is still used. This version has multiple 3 known security vulnerabilities:

• CVE-2021-39537
• CVE-2019-17595
• CVE-2019-17594

ncurses-5.9-20120616-patch.sh.bz2 is a shell script updating ncurses to 5.9 (20120616).

# Use this script to patch ncurses 5.9 to 5.9 (20120616)
# Run this script inside the ncurses 5.9 source directory et voila! Updated.

[ned-deily]

Sorry, I had planned to do an update prior to the 3.11.0 release but it didn't get done. I will get to it shortly.

[erlend-aasland]

Should we try to get this done for 3.13? Is there anything I can help with? Is the "installer build testing" documented somewhere (I did not find much in the READMEs)?

[ronaldoussoren]

FWIW, I have some code locally that installs ncurses 6.4 + patches for when I work on Python. That build passes the entire test suite with this invocation of configure:

    subprocess.check_call(
        [   
            "./configure",
            "--enable-widec",
            "--without-cxx",
            "--without-cxx-binding",
            "--without-ada",
            "--without-curses-h",
            "--enable-shared",
            "--with-shared",
            "--without-debug",
            "--without-normal",
            "--without-tests",
            "--without-manpages",
            "--without-gpm",
            "--datadir=/usr/share",
            "--sysconfdir=/etc",
            "--sharedstatedir=/usr/com",
            "--with-terminfo-dirs=/usr/share/terminfo",
            "--with-default-terminfo-dir=/usr/share/terminfo",
            "--enable-pc-files",
            f"--prefix={PREFIX}",
            f"CFLAGS=-mmacosx-version-min={DEPLOYMENT_TARGET} -arch arm64 -arch x86_64 -Werror=unguarded-availability-new -g",
            f"LDFLAGS=-mmacosx-version-min={DEPLOYMENT_TARGET} -arch arm64 -arch x86_64 -g",
        ],
        cwd=src_path,
        stdout=subprocess.DEVNULL,
    )

I did have to do some tweaking of a makefile to avoid trying to install termini files in a system location, but that's a trivial change as well.

[ned-deily]

Thanks, I will try that soon.

[cmang]

The python.org macOS installers include a private copy of the ncurses library; it has not been updated from 5.9 in a long time. The current upstream version is 6.3 and includes bug and security fixes; we should update to it.

Hi. I'm sorry to butt in on y'alls issue, but would this update mean that Python for macOS would switch from using the Ncurses 5 ABI to the Ncurses 6 ABI? My understanding is that Ncurses 6 provides a build-time option as to whether it will support ABI version 5 or 6.

There are features I would like to use from the Ncurses 6 ABI, specifically extended color pairs, which have never worked for me in Python (despite curses.has_extended_color_support() returning True across multiple versions, including the current python.org binaries).

If this isn't a good place to ask, then I apologize and will try to put it into a real bug report. Thanks.