boring-cyborg[bot]
commented
3 years ago Thanks for opening your first issue here! Be sure to follow the issue template!
Closed zkan closed 3 years ago
boring-cyborg[bot]
commented
3 years ago Thanks for opening your first issue here! Be sure to follow the issue template!
uranusjr
commented
3 years ago Looks like the doc just builds to slowly on the machine. I don’t think there’s much we can do here (except increasing the timeout, which is probably not that meaningful…)
potiuk
commented
3 years ago Also you can try to increase the number of CPUs that are available to docker engine. The Build will take advantage of as many CPUS as many you give it to and it will build the documentation in parallel. It looks like the timeout happens because some of the builds were scheduled to run (via paralell) and waited too long, so i f you increase number of CPUs, they will run much faster in general.
(Please let us know how it works).
Root caus of it is that M1 uses emulation (our Docker image is x86 one) so it will run much slower than native execution.
Also another workaround is try to run it locally on Mac - as long as you have locally instlaled pip installl ".[all,docs]" virtualenv, building docs should work with your natively installed python packages - hopefully all sphinx-related libraries can run natively on Mac M1, so you can run it there (it will not take advantage of parallell runs though).
Just run ./docs/build_docs.py
Yet another workaround is that you can just build a single package using for example ./brees build-docs -- --package-filter 'apache-airflow' to only build airflow documentation.
Taking into account a number of workarounds available and the fact that there is another ticket to implement native M1 docker https://github.com/apache/airflow/issues/15635 I close this ticket as "won't fix".
zkan
commented
3 years ago I've tried to increase CPUs to 8 and still got the same issue.
However, the good news is that building a single package ./breeze --python 3.8 build-docs -- --package-filter apache-airflow works fine. Many thanks! 😄
potiuk
commented
3 years ago Glad to hear it helped :)
subkanthi
commented
3 years ago Interestingly there are no issues running breeze on mac os (m1), but getting this error in a Linux VM(Ubuntu) Parallels. Need to check if docker on Mac is emulating x86.
potiuk
commented
3 years ago Never saw this issue on my linux/chromebook/MacOs :(
potiuk
commented
3 years ago BTW. Try --python 3.6 :)
subkanthi
commented
3 years ago `Step 64/90 : RUN ln -sf /usr/bin/dumb-init /usr/local/bin/dumb-init ---> Using cache ---> 8bc2b7e35a64 Step 65/90 : COPY airflow/www/yarn.lock airflow/www/package.json ${AIRFLOW_SOURCES}/airflow/www/ ---> Using cache ---> 2101286441e1 Step 66/90 : RUN yarn --cwd airflow/www install --frozen-lockfile --no-cache && yarn cache clean ---> Using cache ---> 9c2480204136 Step 67/90 : COPY setup.py ${AIRFLOW_SOURCES}/setup.py ---> 806de9b1671c Step 68/90 : COPY setup.cfg ${AIRFLOW_SOURCES}/setup.cfg ---> 3b83e41ca3be Step 69/90 : COPY airflow/init.py ${AIRFLOW_SOURCES}/airflow/init.py ---> c69bf4520461 Step 70/90 : RUN if [[ ${INSTALL_FROM_PYPI} == "true" ]]; then bash /scripts/docker/install_airflow.sh; fi ---> [Warning] The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested ---> Running in a890b3ba0c58 standard_init_linux.go:228: exec user process caused: exec format error The command '/bin/bash -o pipefail -e -u -x -c if [[ ${INSTALL_FROM_PYPI} == "true" ]]; then bash /scripts/docker/install_airflow.sh; fi' returned a non-zero code: 1
ERROR: The previous step completed with error. Please take a look at output above
`
potiuk
commented
3 years ago Ah yeah. I believe the Docker Desktop for Mac has fixed the emulation but Ubuntu on Parallels on ARM still does not have it :(
Apache Airflow version: 2.2.0.dev0 Python version: 3.8 Docker image: ghcr.io/apache/airflow-main-python3.8-ci-v2
Environment:
uname -a): Darwin Kans-Mac-mini.local 20.5.0 Darwin Kernel Version 20.5.0: Sat May 8 05:10:31 PDT 2021; root:xnu-7195.121.3~9/RELEASE_ARM64_T8101 arm64What happened:
Got the error below when I run the command
./breeze --python 3.8 build-docs.I think this is not related to the memory for Docker Engine since I've tried to increase it to 15GB.
What you expected to happen:
The build should be successful.
How to reproduce it:
./breeze --python 3.8 build-docson Apple M1 chip.TimeoutExpirederror.Anything else we need to know:
When I tried on my MacBook Pro 15", it works just fine. Here is the spec:
uname -a): Darwin BearCamp.local 20.6.0 Darwin Kernel Version 20.6.0: Wed Jun 23 00:26:31 PDT 2021; root:xnu-7195.141.2~5/RELEASE_X86_64 x86_64Please see the entire log below.
Entire Log