Partial support for clang and Windows in configure

[TheShermanTanker]

Feature or enhancement

Enable building cpython on Windows through the traditional configure route, for instance with mingw32.

Pitch

Enables other compilers such as clang to build cpython on Windows, instead of constraining it to the traditional PCbuild toolchain, to narrow the gap between Unix and Windows styles in the build.

Previous discussion

Not Applicable

Note: Only allows configure to run outside of PCbuild on Windows, to successfully build cpython for Windows with clang, the following modification needs to be added to Setup.local:

array arraymodule.c
binascii binascii.c
cmath cmathmodule.c
math mathmodule.c
_md5 md5module.c -I$(srcdir)/Modules/_hacl/include _hacl/Hacl_Hash_MD5.c
_sha1 sha1module.c -I$(srcdir)/Modules/_hacl/include _hacl/Hacl_Hash_SHA1.c
_sha2 sha2module.c -I$(srcdir)/Modules/_hacl/include _hacl/Hacl_Hash_SHA2.c
_sha3 sha3module.c -I$(srcdir)/Modules/_hacl/include _hacl/Hacl_Hash_SHA3.c _hacl/include
_blake2 _blake2/blake2module.c _blake2/blake2b_impl.c _blake2/blake2s_impl.c
_statistics _statisticsmodule.c
_winapi _winapi.c
_random _randommodule.c
_bisect _bisectmodule.c
_heapq _heapqmodule.c
_lsprof _lsprof.c rotatingtree.c
mmap mmapmodule.c
_csv _csv.c
_struct _struct.c
_datetime _datetimemodule.c
_json _json.c
xxsubtype xxsubtype.c
_xxsubinterpreters _xxsubinterpretersmodule.c
_xxinterpchannels _xxinterpchannelsmodule.c
_multibytecodec cjkcodecs/multibytecodec.c
_codecs_cn cjkcodecs/_codecs_cn.c
_codecs_hk cjkcodecs/_codecs_hk.c
_codecs_iso2022 cjkcodecs/_codecs_iso2022.c
_codecs_jp cjkcodecs/_codecs_jp.c
_codecs_kr cjkcodecs/_codecs_kr.c
_codecs_tw cjkcodecs/_codecs_tw.c
_pickle _pickle.c
_opcode _opcode.c
_contextvars _contextvarsmodule.c
_io _io/winconsoleio.c _io/iobase.c _io/_iomodule.c _io/bufferedio.c _io/bytesio.c _io/fileio.c _io/stringio.c _io/textio.c

Linked PRs

• gh-105789

[terryjreedy]

I suspect that CPython's configure, etc, are limited to CPython supported platforms, where a platform is a CPU architecture, an operating system, and a compiler family. See PEP 11. At minimum, a reliable buildbot and involved core developer are needed. 'Previous discussion' for new platform support is needed.

[erlend-aasland]

@zooba: thoughts about this?

[TheShermanTanker]

I suspect that CPython's configure, etc, are limited to CPython supported platforms, where a platform is a CPU architecture, an operating system, and a compiler family. See PEP 11. At minimum, a reliable buildbot and involved core developer are needed. 'Previous discussion' for new platform support is needed.

I don't believe that is the case, since configure still has (broken) support for Cygwin, although PEP-11 doesn't mention it as a platform anywhere

[TheShermanTanker]

Small side tangent question: Where are the flags for the C extension modules in the stdlib set? Setup.stdlib has them, but comments in the file state that it is not used in the build process yet

[erlend-aasland]

Small side tangent question: Where are the flags for the C extension modules in the stdlib set?

IIRC, they are compiler flags (typically include dirs).

[erlend-aasland]

For a long time, a lot of Cygwin stuff has been torn out, simply because it was broken and there was no-one around with a dev environment and/or who knew the specifics.

[erlend-aasland]

When Christian and I reworked the configure stuff in 2021-2022, we ditched a lot of Cygwin stuff. Even then, Cygwin support was severely broken. Also, there were no buildbots (I don't think there are even now).

[TheShermanTanker]

Yep, back in 2021 I tried but failed to fix the broken Cygwin support, but anyways I think support for a more modern platform like mingw and a modern toolchain like clang is more helpful than something like Cygwin

[erlend-aasland]

Sorry, I implicitly think of Cygwin whenever I see mingw mentioned. I did not know you could use mingw without Cygwin (I don't know anything about Windows).

[zooba]

Since I was asked, I have no thoughts about this.

The whole configure/make system is as opaque to me as the XML for MSBuild is to everyone else. If we ever move the Windows build into that, I probably won't be able to contribute for a while until I have time to get up to speed on it.

clang-cl works to build with clang through MSBuild, if that's a goal. No need to switch to make unless you specifically want gcc.

[TheShermanTanker]

I've halted work on this for now, too hacky and too much edge case code required. This seems to be a symptom of how the make system is set up, which makes (haha) any change required to support this necessarily hacky and messy. I'll have to take the approach from another angle, maybe after a build system improvement

clang-cl works to build with clang through MSBuild, if that's a goal. No need to switch to make unless you specifically want gcc.

I'm more accurately performing this change so MinGW's cpython-mingw port can axe most of their changes required for cpython to build under a Unix environment

[erlend-aasland]

I've halted work on this for now, too hacky and too much edge case code required. This seems to be a symptom of how the make system is set up, which makes (haha) any change required to support this necessarily hacky and messy. I'll have to take the approach from another angle, maybe after a build system improvement

Yeah, the current build system has grown into a hack; it is hard to make adjustments. Feel free to express complaint and wishes here:

• https://discuss.python.org/t/what-do-you-consider-pain-points-of-todays-build-system/31815
• https://discuss.python.org/t/what-do-you-want-to-see-in-tomorrow-s-cpython-build-system/28197