touch profile-run-stamp exit code 2 for 3.11 in Dockerfile using ubuntu:22.04

[hkiang01]

Error

#0 580.3 make clean-retain-profile
#0 580.4 make[1]: Entering directory '/usr/src/Python-3.11.2'
#0 580.4 find . -depth -name '__pycache__' -exec rm -rf {} ';'
#0 580.5 find . -name '*.py[co]' -exec rm -f {} ';'
#0 580.5 find . -name '*.[oa]' -exec rm -f {} ';'
#0 581.0 find . -name '*.s[ol]' -exec rm -f {} ';'
#0 581.1 find . -name '*.so.[0-9]*.[0-9]*' -exec rm -f {} ';'
#0 581.2 find . -name '*.wasm' -exec rm -f {} ';'
#0 581.2 find . -name '*.lst' -exec rm -f {} ';'
#0 581.2 find build -name 'fficonfig.h' -exec rm -f {} ';' || true
#0 581.2 find build -name '*.py' -exec rm -f {} ';' || true
#0 581.2 find build -name '*.py[co]' -exec rm -f {} ';' || true
#0 581.2 rm -f pybuilddir.txt
#0 581.2 rm -f Lib/lib2to3/*Grammar*.pickle
#0 581.2 rm -f _bootstrap_python
#0 581.2 rm -f python.html python*.js python.data python*.symbols python*.map
#0 581.2 rm -f ./usr/local/lib/python3.11/os.py
#0 581.2 rm -f Programs/_testembed Programs/_freeze_module
#0 581.2 rm -f Python/deepfreeze/*.[co]
#0 581.2 rm -f Python/frozen_modules/*.h
#0 581.3 rm -f Python/frozen_modules/MANIFEST
#0 581.3 find build -type f -a ! -name '*.gc??' -exec rm -f {} ';'
#0 581.3 rm -f Include/pydtrace_probes.h
#0 581.3 rm -f profile-gen-stamp
#0 581.3 make[1]: Leaving directory '/usr/src/Python-3.11.2'
#0 581.3 # This is an expensive target to build and it does not have proper
#0 581.3 # makefile dependency information.  So, we create a "stamp" file
#0 581.3 # to record its completion and avoid re-running it.
#0 581.3 touch profile-run-stamp
------
Dockerfile:55
--------------------
  54 |     WORKDIR /usr/src/Python-${PYTHON_VERSION}
  55 | >>> RUN ./configure \
  56 | >>>   --prefix /usr/local \
  57 | >>>   --enable-optimizations \
  58 | >>>   --with-lto \
  59 | >>>   && make -j "$(nproc)" altinstall
  60 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c ./configure   --prefix /usr/local   --enable-optimizations   --with-lto   && make -j \"$(nproc)\" altinstall" did not complete successfully: exit code: 2

Steps to reproduce

# Dockerfile
FROM ubuntu:22.04 as base
WORKDIR /

# update packages to get patches for security purposes, etc.
# hadolint ignore=DL3009
RUN apt-get update \
  && apt-get -y upgrade

# hadolint ignore=DL3059
RUN apt-get clean \
  && rm -rf /var/lib/apt/lists/*

#######################################
# INSTALL PYTHON FROM OFFICIAL SOURCE #
#######################################
ENV DEBIAN_FRONTEND noninteractive
ARG PYTHON_VERSION
# install dependencies
# see https://devguide.python.org/getting-started/setup-building/index.html#build-dependencies
# hadolint ignore=DL3008
RUN echo "deb-src http://archive.ubuntu.com/ubuntu/ jammy main" >> /etc/apt/sources.list \
  && echo "deb-src http://archive.ubuntu.com/ubuntu/ jammy-updates universe" >> /etc/apt/sources.list \
  && apt-get update \
  && apt-get build-dep -y "python${PYTHON_VERSION%.*}" \
  && apt-get install -y --no-install-recommends pkg-config \
  && apt-get install -y --no-install-recommends build-essential gdb lcov pkg-config \
  libbz2-dev libffi-dev libgdbm-dev libgdbm-compat-dev liblzma-dev \
  libncurses5-dev libreadline6-dev libsqlite3-dev libssl-dev \
  lzma lzma-dev tk-dev uuid-dev zlib1g-dev \
  && rm -rf /var/lib/apt/lists/ \
  && update-ca-certificates
# required to pull python source code
# hadolint ignore=DL3008
RUN apt-get update \
  && apt-get -y install --no-install-recommends \
  curl \
  xz-utils \
  && apt-get clean \
  && rm -rf /var/lib/apt/lists/

# see python.org/downloads/source/
# see https://devguide.python.org/getting-started/setup-building/index.html#getting-the-source-code
WORKDIR /usr/src
ARG PYTHON_VERSION
RUN curl --tlsv1.3 -O \
  "https://www.python.org/ftp/python/${PYTHON_VERSION}/Python-${PYTHON_VERSION}.tar.xz" \
  && tar -xf "Python-${PYTHON_VERSION}.tar.xz"
# remove packges only used to extract python tar.xz
# hadolint ignore=DL3059
RUN apt-get remove -y \
  curl \
  xz-utils
WORKDIR /usr/src/Python-${PYTHON_VERSION}
RUN ./configure \
  --prefix /usr/local \
  --enable-optimizations \
  --with-lto \
  && make -j "$(nproc)" altinstall

This works

docker build --build-arg PYTHON_VERSION=3.10.10 -t python3.10 -f Dockerfile .

but this errors out as above

docker build --build-arg PYTHON_VERSION=3.11.2 -t python3.11-touch-bug -f Dockerfile .

I've also observed this error when setting PYTHON_VERSION to 3.11.1, 3.11.0. Am I missing something or is there a bug?

[sunmy2019]

Building with support for profile generation fails (make build_all_generate_profile)

It seems that the linker was runed before the object was compiled on my machine. The dependency might not be handled correctly.

gcc   -fno-semantic-interposition -flto -fuse-linker-plugin -ffat-lto-objects -flto-partition=none -g  -o Programs/_freeze_module Programs/_freeze_module.o Modules/getpath_noop.o Modules/getbuildinfo.o Parser/token.o  Parser/pegen.o Parser/pegen_errors.o Parser/action_helpers.o Parser/parser.o Parser/string_parser.o Parser/peg_api.o Parser/myreadline.o Parser/tokenizer.o Objects/abstract.o Objects/accu.o Objects/boolobject.o Objects/bytes_methods.o Objects/bytearrayobject.o Objects/bytesobject.o Objects/call.o Objects/capsule.o Objects/cellobject.o Objects/classobject.o Objects/codeobject.o Objects/complexobject.o Objects/descrobject.o Objects/enumobject.o Objects/exceptions.o Objects/genericaliasobject.o Objects/genobject.o Objects/fileobject.o Objects/floatobject.o Objects/frameobject.o Objects/funcobject.o Objects/interpreteridobject.o Objects/iterobject.o Objects/listobject.o Objects/longobject.o Objects/dictobject.o Objects/odictobject.o Objects/memoryobject.o Objects/methodobject.o Objects/moduleobject.o Objects/namespaceobject.o Objects/object.o Objects/obmalloc.o Objects/picklebufobject.o Objects/rangeobject.o Objects/setobject.o Objects/sliceobject.o Objects/structseq.o Objects/tupleobject.o Objects/typeobject.o Objects/unicodeobject.o Objects/unicodectype.o Objects/unionobject.o Objects/weakrefobject.o Python/_warnings.o Python/Python-ast.o Python/Python-tokenize.o Python/asdl.o Python/ast.o Python/ast_opt.o Python/ast_unparse.o Python/bltinmodule.o Python/ceval.o Python/codecs.o Python/compile.o Python/context.o Python/dynamic_annotations.o Python/errors.o Python/frame.o Python/frozenmain.o Python/future.o Python/getargs.o Python/getcompiler.o Python/getcopyright.o Python/getplatform.o Python/getversion.o Python/hamt.o Python/hashtable.o Python/import.o Python/importdl.o Python/initconfig.o Python/marshal.o Python/modsupport.o Python/mysnprintf.o Python/mystrtoul.o Python/pathconfig.o Python/preconfig.o Python/pyarena.o Python/pyctype.o Python/pyfpe.o Python/pyhash.o Python/pylifecycle.o Python/pymath.o Python/pystate.o Python/pythonrun.o Python/pytime.o Python/bootstrap_hash.o Python/specialize.o Python/structmember.o Python/symtable.o Python/sysmodule.o Python/thread.o Python/traceback.o Python/getopt.o Python/pystrcmp.o Python/pystrtod.o Python/pystrhex.o Python/dtoa.o Python/formatter_unicode.o Python/fileutils.o Python/suggestions.o Python/dynload_shlib.o     Modules/config.o Modules/main.o Modules/gcmodule.o Modules/atexitmodule.o  Modules/faulthandler.o  Modules/posixmodule.o  Modules/signalmodule.o  Modules/_tracemalloc.o  Modules/_codecsmodule.o  Modules/_collectionsmodule.o  Modules/errnomodule.o  Modules/_io/_iomodule.o Modules/_io/iobase.o Modules/_io/fileio.o Modules/_io/bytesio.o Modules/_io/bufferedio.o Modules/_io/textio.o Modules/_io/stringio.o  Modules/itertoolsmodule.o  Modules/_sre/sre.o  Modules/_threadmodule.o  Modules/timemodule.o  Modules/_weakref.o  Modules/_abc.o  Modules/_functoolsmodule.o  Modules/_localemodule.o  Modules/_operator.o  Modules/_stat.o  Modules/symtablemodule.o  Modules/pwdmodule.o  Modules/xxsubtype.o -ldl                         -lm 
/usr/bin/ld: cannot find Programs/_freeze_module.o: No such file or directory
/usr/bin/ld: cannot find Modules/getpath_noop.o: No such file or directory
/usr/bin/ld: cannot find Objects/memoryobject.o: No such file or directory
...
/usr/bin/ld: cannot find Modules/pwdmodule.o: No such file or directory
/usr/bin/ld: cannot find Modules/xxsubtype.o: No such file or directory
collect2: error: ld returned 1 exit status
make: *** [Makefile:1197: Programs/_freeze_module] Error 1
make: *** Waiting for unfinished jobs....

[sunmy2019]

I don't know if it was documented, but by analyzing the generated Makefile, you can run

make all && make altinstall

to handle the dependency correctly. all is the default target.

As in your case, just do

make -j "$(nproc)" && make -j "$(nproc)" altinstall

[sunmy2019]

The doc says the same thing: Use

make -j "$(nproc)" && make -j "$(nproc)" altinstall

https://docs.python.org/3/using/unix.html#building-python

[erlend-aasland]

See also gh-104523. FTR, I don't think we'll be backporting those changes to 3.11.