python / cpython

The Python programming language
https://www.python.org
Other
63.7k stars 30.52k forks source link

Windows builds sometimes fail on Azure and GitHub Action: fatal error RC1116: RC terminating after preprocessor errors #89383

Closed sobolevn closed 3 years ago

sobolevn commented 3 years ago
BPO 45220
Nosy @pfmoore, @vstinner, @tjguk, @ambv, @ericsnowcurrently, @zware, @serhiy-storchaka, @eryksun, @zooba, @miss-islington, @sobolevn
PRs
  • python/cpython#28393
  • python/cpython#28394
  • python/cpython#28395
  • python/cpython#28622
  • python/cpython#29396
  • python/cpython#29405
  • python/cpython#29406
  • python/cpython#29501
  • python/cpython#29503
  • python/cpython#29504
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields: ```python assignee = None closed_at = created_at = labels = ['3.10', 'build', '3.9', 'OS-windows', '3.11'] title = 'Windows builds sometimes fail on Azure and GitHub Action: fatal error RC1116: RC terminating after preprocessor errors' updated_at = user = 'https://github.com/sobolevn' ``` bugs.python.org fields: ```python activity = actor = 'steve.dower' assignee = 'none' closed = True closed_date = closer = 'steve.dower' components = ['Build', 'Windows'] creation = creator = 'sobolevn' dependencies = [] files = [] hgrepos = [] issue_num = 45220 keywords = ['patch'] message_count = 25.0 messages = ['401948', '401952', '401953', '401962', '401963', '401965', '401967', '401968', '401973', '401974', '401978', '401980', '401984', '402874', '405736', '405737', '405749', '406049', '406050', '406070', '406071', '406356', '406359', '412107', '412207'] nosy_count = 12.0 nosy_names = ['paul.moore', 'vstinner', 'tim.golden', 'lukasz.langa', 'eric.snow', 'zach.ware', 'serhiy.storchaka', 'eryksun', 'steve.dower', 'miss-islington', 'sobolevn', 'ntrischi'] pr_nums = ['28393', '28394', '28395', '28622', '29396', '29405', '29406', '29501', '29503', '29504'] priority = 'normal' resolution = 'fixed' stage = 'resolved' status = 'closed' superseder = None type = 'compile error' url = 'https://bugs.python.org/issue45220' versions = ['Python 3.9', 'Python 3.10', 'Python 3.11'] ```

    sobolevn commented 3 years ago

    I've started to notice that CPython's builds on Windows now simetimes fail with something like this:

    (both Azure and Github Actions are affected)

    Using "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Current\Bin\msbuild.exe" (found in the Visual Studio installation)
    Using py -3.9 (found 3.9 with py.exe)
    
    D:\a\cpython\cpython>"C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Current\Bin\msbuild.exe" "D:\a\cpython\cpython\PCbuild\pcbuild.proj" /t:Build /m /nologo /v:m /clp:summary /p:Configuration=Release /p:Platform=Win32 /p:IncludeExternals=true /p:IncludeCTypes=true /p:IncludeSSL=true /p:IncludeTkinter=true /p:UseTestMarker= /p:GIT="C:\Program Files\Git\bin\git.exe"          
      _freeze_module.c
      config_minimal.c
      atexitmodule.c
      faulthandler.c
      gcmodule.c
      getbuildinfo.c
      posixmodule.c
      signalmodule.c
      _tracemalloc.c
      _iomodule.c
      bufferedio.c
      bytesio.c
      fileio.c
      iobase.c
      stringio.c
      textio.c
      winconsoleio.c
      abstract.c
      accu.c
      boolobject.c
      Compiling...
      bytearrayobject.c
      bytes_methods.c
      bytesobject.c
      call.c
      capsule.c
      cellobject.c
      classobject.c
      codeobject.c
      complexobject.c
      descrobject.c
      dictobject.c
      enumobject.c
      exceptions.c
      fileobject.c
      floatobject.c
      frameobject.c
      funcobject.c
      genericaliasobject.c
      genobject.c
      interpreteridobject.c
      Compiling...
      iterobject.c
      listobject.c
      longobject.c
      memoryobject.c
      methodobject.c
      moduleobject.c
      namespaceobject.c
      object.c
      obmalloc.c
      odictobject.c
      picklebufobject.c
      rangeobject.c
      setobject.c
      sliceobject.c
      structseq.c
      tupleobject.c
      typeobject.c
      unicodectype.c
      unicodeobject.c
      unionobject.c
      Compiling...
      weakrefobject.c
      myreadline.c
      parser.c
      peg_api.c
      pegen.c
      string_parser.c
      token.c
      tokenizer.c
      getpathp.c
      invalid_parameter_handler.c
      msvcrtmodule.c
      winreg.c
      _warnings.c
      asdl.c
      ast.c
      ast_opt.c
      ast_unparse.c
      bltinmodule.c
      bootstrap_hash.c
      ceval.c
    D:\a\cpython\cpython\Python\ceval.c(3669,13): warning C4018: '>=': signed/unsigned mismatch [D:\a\cpython\cpython\PCbuild\_freeze_module.vcxproj]
    D:\a\cpython\cpython\Python\ceval.c(3777,13): warning C4018: '>=': signed/unsigned mismatch [D:\a\cpython\cpython\PCbuild\_freeze_module.vcxproj]
      Compiling...
      codecs.c
      compile.c
      context.c
      dtoa.c
      dynamic_annotations.c
      dynload_win.c
      errors.c
      fileutils.c
      formatter_unicode.c
      frame.c
      future.c
      getargs.c
      getcompiler.c
      getcopyright.c
      getopt.c
      getplatform.c
      getversion.c
      hamt.c
      hashtable.c
      import.c
      Compiling...
      importdl.c
      initconfig.c
      marshal.c
      modsupport.c
      mysnprintf.c
      mystrtoul.c
      pathconfig.c
      preconfig.c
      pyarena.c
      pyctype.c
      pyfpe.c
      pyhash.c
      pylifecycle.c
      pymath.c
      pystate.c
      pystrcmp.c
      pystrhex.c
      pystrtod.c
      Python-ast.c
      pythonrun.c
      Compiling...
      Python-tokenize.c
      pytime.c
      specialize.c
      structmember.c
      suggestions.c
      symtable.c
      sysmodule.c
      thread.c
      traceback.c
         Creating library D:\a\cpython\cpython\PCbuild\win32\_freeze_module.lib and object D:\a\cpython\cpython\PCbuild\win32\_freeze_module.exp
      Generating code
      Finished generating code
      _freeze_module.vcxproj -> D:\a\cpython\cpython\PCbuild\win32\_freeze_module.exe
      Updated files: __hello__.h
      Killing any running python.exe instances...
      Regenerate pycore_ast.h pycore_ast_state.h Python-ast.c
      D:\a\cpython\cpython\Python\Python-ast.c, D:\a\cpython\cpython\Include\internal\pycore_ast.h, D:\a\cpython\cpython\Include\internal\pycore_ast_state.h regenerated.
      Regenerate opcode.h opcode_targets.h
      Include\opcode.h regenerated from Lib\opcode.py
      Jump table written into Python\opcode_targets.h
      Regenerate token-list.inc token.h token.c token.py
      Generated sources are up to date
      Getting build info from "C:\Program Files\Git\bin\git.exe"
      Building heads/main-dirty:7dacb70 main
      _abc.c
      _bisectmodule.c
      blake2module.c
      blake2b_impl.c
      blake2s_impl.c
      _codecsmodule.c
      _collectionsmodule.c
      _contextvarsmodule.c
      _csv.c
      _functoolsmodule.c
      _heapqmodule.c
      _json.c
      _localemodule.c
      _lsprof.c
      _math.c
      _pickle.c
      _randommodule.c
      sha3module.c
      _sre.c
      _stat.c
      Compiling...
      _struct.c
      _weakref.c
      arraymodule.c
      atexitmodule.c
      audioop.c
      binascii.c
      cmathmodule.c
      _datetimemodule.c
      errnomodule.c
      faulthandler.c
      gcmodule.c
      itertoolsmodule.c
      main.c
      mathmodule.c
      md5module.c
      mmapmodule.c
      _opcode.c
      _operator.c
      posixmodule.c
      rotatingtree.c
      Compiling...
      sha1module.c
      sha256module.c
      sha512module.c
      signalmodule.c
      _statisticsmodule.c
      symtablemodule.c
      _threadmodule.c
      _tracemalloc.c
      _typingmodule.c
      timemodule.c
      xxsubtype.c
      _xxsubinterpretersmodule.c
      fileio.c
      bytesio.c
      stringio.c
      bufferedio.c
      iobase.c
      textio.c
      winconsoleio.c
      _iomodule.c
      Compiling...
      _codecs_cn.c
      _codecs_hk.c
      _codecs_iso2022.c
      _codecs_jp.c
      _codecs_kr.c
      _codecs_tw.c
      multibytecodec.c
      _winapi.c
      abstract.c
      accu.c
      boolobject.c
      bytearrayobject.c
      bytes_methods.c
      bytesobject.c
      call.c
      capsule.c
      cellobject.c
      classobject.c
      codeobject.c
      complexobject.c
      Compiling...
      descrobject.c
      dictobject.c
      enumobject.c
      exceptions.c
      fileobject.c
      floatobject.c
      frameobject.c
      funcobject.c
      genericaliasobject.c
      genobject.c
      interpreteridobject.c
      iterobject.c
      listobject.c
      longobject.c
      memoryobject.c
      methodobject.c
      moduleobject.c
      namespaceobject.c
      object.c
      obmalloc.c
      Compiling...
      odictobject.c
      picklebufobject.c
      rangeobject.c
      setobject.c
      sliceobject.c
      structseq.c
      tupleobject.c
      typeobject.c
      unicodectype.c
      unicodeobject.c
      unionobject.c
      weakrefobject.c
      myreadline.c
      tokenizer.c
      token.c
      pegen.c
      parser.c
      string_parser.c
      peg_api.c
      invalid_parameter_handler.c
      Compiling...
      winreg.c
      config.c
      getpathp.c
      msvcrtmodule.c
      pyhash.c
      _warnings.c
      asdl.c
      ast.c
      ast_opt.c
      ast_unparse.c
      bltinmodule.c
      bootstrap_hash.c
      ceval.c
    D:\a\cpython\cpython\Python\ceval.c(3669,13): warning C4018: '>=': signed/unsigned mismatch [D:\a\cpython\cpython\PCbuild\pythoncore.vcxproj]
    D:\a\cpython\cpython\Python\ceval.c(3777,13): warning C4018: '>=': signed/unsigned mismatch [D:\a\cpython\cpython\PCbuild\pythoncore.vcxproj]
      codecs.c
      compile.c
      context.c
      dynamic_annotations.c
      dynload_win.c
      errors.c
      fileutils.c
      Compiling...
      formatter_unicode.c
      frame.c
      frozen.c
      future.c
      getargs.c
      getcompiler.c
      getcopyright.c
      getopt.c
      getplatform.c
      getversion.c
      hamt.c
      hashtable.c
      import.c
      importdl.c
      initconfig.c
      marshal.c
      modsupport.c
      mysnprintf.c
      mystrtoul.c
      pathconfig.c
      Compiling...
      preconfig.c
      pyarena.c
      pyctype.c
      pyfpe.c
      pylifecycle.c
      pymath.c
      pytime.c
      pystate.c
      pystrcmp.c
      pystrhex.c
      pystrtod.c
      dtoa.c
      Python-ast.c
      Python-tokenize.c
      pythonrun.c
      specialize.c
      suggestions.c
      structmember.c
      symtable.c
      sysmodule.c
      Compiling...
      thread.c
      traceback.c
      zlibmodule.c
      adler32.c
      compress.c
      crc32.c
      deflate.c
      infback.c
      inffast.c
      inflate.c
      inftrees.c
      trees.c
      uncompr.c
      zutil.c
      dl_nt.c
      getbuildinfo.c
    C:\Program Files (x86)\Windows Kits\10\Include\10.0.22000.0\um\winnt.h(253): error RC2188: D:\a\cpython\cpython\PCbuild\obj\311win32_Release\pythoncore\RCa05056(47) : fatal error RC1116: RC terminating after preprocessor errors [D:\a\cpython\cpython\PCbuild\pythoncore.vcxproj]
    
    Build FAILED.
    
    D:\a\cpython\cpython\Python\ceval.c(3669,13): warning C4018: '>=': signed/unsigned mismatch [D:\a\cpython\cpython\PCbuild\_freeze_module.vcxproj]
    D:\a\cpython\cpython\Python\ceval.c(3777,13): warning C4018: '>=': signed/unsigned mismatch [D:\a\cpython\cpython\PCbuild\_freeze_module.vcxproj]
    D:\a\cpython\cpython\Python\ceval.c(3669,13): warning C4018: '>=': signed/unsigned mismatch [D:\a\cpython\cpython\PCbuild\pythoncore.vcxproj]
    D:\a\cpython\cpython\Python\ceval.c(3777,13): warning C4018: '>=': signed/unsigned mismatch [D:\a\cpython\cpython\PCbuild\pythoncore.vcxproj]
    C:\Program Files (x86)\Windows Kits\10\Include\10.0.22000.0\um\winnt.h(253): error RC2188: D:\a\cpython\cpython\PCbuild\obj\311win32_Release\pythoncore\RCa05056(47) : fatal error RC1116: RC terminating after preprocessor errors [D:\a\cpython\cpython\PCbuild\pythoncore.vcxproj]
        4 Warning(s)
        1 Error(s)

    Links:

    vstinner commented 3 years ago

    It also fails on the 3.9 and 3.10 on the Windows jobs of GitHub Action.

    vstinner commented 3 years ago

    It may be related to bpo-45020 which recently got a change:

    New changeset 9fd87a5fe5c468cf94265365091267838b004b7f by Eric Snow in branch 'main': bpo-45020: Revert "Drop the frozen .h files from the repo." (gh-28380) https://github.com/python/cpython/commit/9fd87a5fe5c468cf94265365091267838b004b7f

    ericsnowcurrently commented 3 years ago

    This is more likely to relate to bpo-45188, "De-couple the Windows builds from freezing modules.", for which the PR was merged yesterday:

    New changeset 09b4ad11f323f8702cde795e345b75e0fbb1a9a5 by Steve Dower in branch 'main': bpo-45188: Windows now regenerates frozen modules at the start of build instead of late (GH-28322) https://github.com/python/cpython/commit/09b4ad11f323f8702cde795e345b75e0fbb1a9a5

    (https://github.com/python/cpython/pull/28322)

    ericsnowcurrently commented 3 years ago

    Of course, it might also be unrelated.

    vstinner commented 3 years ago

    C:\Program Files (x86)\Windows Kits\10\Include\10.0.22000.0\um\winnt.h(253): error RC2188: D:\a\cpython\cpython\PCbuild\obj\311win32_Release\pythoncore\RCa05056(47) : fatal error RC1116: RC terminating after preprocessor errors [D:\a\cpython\cpython\PCbuild\pythoncore.vcxproj]

    Let's split it into sub-parts:

    RCxxxx errors are generated by the resource compiler

    C:\Program Files (x86)\Windows Kits\10\Include\10.0.22000.0\um\winnt.h(253)

    What is the code at line 253?

    error RC2188

    What is this error?

    311win32_Release\pythoncore\RCa05056(47)

    Is this a directory with a generated name, or is it an error code?

    fatal error RC1116

    What is this error?

    --

    Python contains the following .rc files:

    PC/pylauncher.rc PC/pyshellext.rc PC/python_exe.rc PC/python_nt.rc PC/pythonw_exe.rc PC/sqlite3.rc

    None of these files include winnt.h.

    In fact, "winnt.h" cannot be found in any file of the Python source code.

    PC/python_nt.rc includes PC/python_ver_rc.h which contains 3 includes:

    I'm not sure why modsupport.h is included.

    --

    I found one page mentioning RC2188 *and* RC1116 error codes together:

    https://social.msdn.microsoft.com/Forums/vstudio/en-US/7ba8eb72-12e9-4c78-af68-7f50c170040f/warning-rc4011-and-error-rc2188?forum=vclanguage

    "RCxxxx errors are generated by the resource compiler. One of your .rc files includes directly or indirectly headers that RC can't understand so you'll have to check that and remove the #includes."

    vstinner commented 3 years ago

    bpo-45188: Windows now regenerates frozen modules at the start of build instead of late (GH-28322)

    Maybe the resource compiler needs a header file which is not generated yet, and so the build fails.

    In msg401948 logs, I don't see when header files are generated. This build step seems to be fully quiet.

    vstinner commented 3 years ago

    This issue is serious: it prevents to merge many pull requests, it blocks the Python development workflow.

    zooba commented 3 years ago

    The build order change was not backported (and should not be), so it's unrelated to the failures on 3.9 and 3.10.

    More likely this is either a problematic compiler update, or some additional data leaking into the build information (e.g. I could see this happening if someone added a UTF-8 copyright symbol somewhere instead of an escaped version).

    zooba commented 3 years ago

    C:\Program Files (x86)\Windows Kits\10\Include\10.0.22000.0\um\winnt.h

    This appears to be the Windows 11 preview SDK, so most likely it's a bug in that. We'll obviously need some logic to not automatically upgrade to this SDK on build.

    zooba commented 3 years ago

    New changeset f4b94b1f57827083990272b5f282aa1493ae2bf4 by Steve Dower in branch 'main': bpo-45220: Avoid automatically selecting the Windows 11 SDK preview when building (GH-28393) https://github.com/python/cpython/commit/f4b94b1f57827083990272b5f282aa1493ae2bf4

    miss-islington commented 3 years ago

    New changeset 7ad07eed885ae333d5dfc470b9c11ff2f7b229aa by Miss Islington (bot) in branch '3.9': bpo-45220: Avoid automatically selecting the Windows 11 SDK preview when building (GH-28393) https://github.com/python/cpython/commit/7ad07eed885ae333d5dfc470b9c11ff2f7b229aa

    serhiy-storchaka commented 3 years ago

    New changeset f798cef8aaaa7faeb428bdb9ebf73df1a6146304 by Miss Islington (bot) in branch '3.10': bpo-45220: Avoid automatically selecting the Windows 11 SDK preview when building (GH-28393) (GH-28394) https://github.com/python/cpython/commit/f798cef8aaaa7faeb428bdb9ebf73df1a6146304

    ambv commented 3 years ago

    New changeset 456d6d9fefac828c19fc1b562f6301f00965fe59 by Miss Islington (bot) in branch '3.8': bpo-45220: Avoid automatically selecting the Windows 11 SDK preview when building (GH-28393) (GH-28622) https://github.com/python/cpython/commit/456d6d9fefac828c19fc1b562f6301f00965fe59

    zooba commented 3 years ago

    New changeset fd0c84dc28d00d68e4f43034dc41786a682390fd by Steve Dower in branch 'main': bpo-45220: Remove invalid include from resource definition files on Windows (GH-29396) https://github.com/python/cpython/commit/fd0c84dc28d00d68e4f43034dc41786a682390fd

    miss-islington commented 3 years ago

    New changeset 1a44d27e6f8a06cb7d56e39be48852f0c3f4502b by Miss Islington (bot) in branch '3.9': bpo-45220: Remove invalid include from resource definition files on Windows (GH-29396) https://github.com/python/cpython/commit/1a44d27e6f8a06cb7d56e39be48852f0c3f4502b

    ambv commented 3 years ago

    New changeset c0f3281d6ca5c59d4a11698364463d968b9ddd3c by Miss Islington (bot) in branch '3.10': bpo-45220: Remove invalid include from resource definition files on Windows (GH-29396) (GH-29406) https://github.com/python/cpython/commit/c0f3281d6ca5c59d4a11698364463d968b9ddd3c

    zooba commented 3 years ago

    New changeset a56fbad85ea655631bce68d4c0f47f1a8b500abd by Steve Dower in branch 'main': bpo-45220: Ensure RT_MANIFEST is defined when compiling Windows resource files (GH-29501) https://github.com/python/cpython/commit/a56fbad85ea655631bce68d4c0f47f1a8b500abd

    miss-islington commented 3 years ago

    New changeset d29f591dd6b1dcd4f36b5b49761cf8225be690bd by Miss Islington (bot) in branch '3.10': bpo-45220: Ensure RT_MANIFEST is defined when compiling Windows resource files (GH-29501) https://github.com/python/cpython/commit/d29f591dd6b1dcd4f36b5b49761cf8225be690bd

    eryksun commented 3 years ago

    Steve, instead of manually defining RT_MANIFEST, try including "winresrc.h" in the resource definition files. This SDK header includes "winuser.rh" (note the ".rh" extension), which, among other things, includes the following RC_INVOKED definitions from "winuser.h":

        #ifdef RC_INVOKED
        #define RT_MANIFEST                        24
        #define CREATEPROCESS_MANIFEST_RESOURCE_ID  1
        #define ISOLATIONAWARE_MANIFEST_RESOURCE_ID 2
        #define ISOLATIONAWARE_NOSTATICIMPORT_MANIFEST_RESOURCE_ID 3
        #define ISOLATIONPOLICY_MANIFEST_RESOURCE_ID 4
        #define ISOLATIONPOLICY_BROWSER_MANIFEST_RESOURCE_ID 5
        #define MINIMUM_RESERVED_MANIFEST_RESOURCE_ID 1   /* inclusive */
        #define MAXIMUM_RESERVED_MANIFEST_RESOURCE_ID 16  /* inclusive */
    eryksun commented 3 years ago

    Also, I suggest using CREATEPROCESS_MANIFEST_RESOURCE_ID (1) and ISOLATIONAWARE_MANIFEST_RESOURCE_ID (2) instead of hard-coded resource IDs.

    zooba commented 3 years ago

    New changeset 32d14b262d6d914f63485f8d6e7859df58cd2089 by Miss Islington (bot) in branch '3.9': bpo-45220: Ensure RT_MANIFEST is defined when compiling Windows resource files (GH-29501) https://github.com/python/cpython/commit/32d14b262d6d914f63485f8d6e7859df58cd2089

    zooba commented 3 years ago

    instead of manually defining RT_MANIFEST, try including "winresrc.h" in the resource definition files

    I just wish it were documented... we can keep it in mind if another one of these pops up.

    1983d6d2-e84b-4a8e-ab88-812ade7a187d commented 2 years ago

    Compiling Python3.8.9 from source on Windows 10 with VS2019. Running PCBuild\build.bat, I'm getting the following issue:

    C:\Program Files (x86)\Windows Kits\10\Include\10.0.22000.0\um\winnt.h(253): error RC2188: C:\Python-3.8.9\PCbuild\obj\38amd64_Release\pythoncore\RCa09752(53) : fatal error RC1116: RC terminating after preprocessor errors [C:\Python-3.8.9\PCbuild\pythoncore.vcxproj]

    I noticed this issue is closed but it doesn't seem to be resolved.

    zooba commented 2 years ago

    It was not backported to 3.8 because that's in security fix only mode.