search

ONLYOFFICE / build_tools

Used to build ONLYOFFICE DocumentServer-related products
https://www.onlyoffice.com
GNU Affero General Public License v3.0
99 stars 162 forks source link

Could not find Ninja during build #829

Open zsy1314 opened 1 week ago

zsy1314 commented 1 week ago

I installed Ubuntu 16.04 in Docker Then used git clone https://github.com/ONLYOFFICE/build_tools.git Then ran ./automate.py server in cd build_tools/tools/linux to start the build After a long wait, the following error message appeared.

[fetch & build]: boost
[fetch & build]: cef
[fetch & build]: icu
[fetch & build]: openssl
gn gen out.gn/linux_64 --args="v8_static_library=true is_component_build=false v8_monolithic=true v8_use_external_startup_data=false use_custom_libcxx=false treat_warnings_as_errors=false target_cpu=\"x64\" v8_target_cpu=\"x64\" is_debug=false is_clang=true use_sysroot=false"
Done. Made 165 targets from 92 files in 378ms
depot_tools/ninja.py: Could not find Ninja in the third_party of the current project, nor in your PATH.
Please take one of the following actions to install Ninja:
- If your project has DEPS, add a CIPD Ninja dependency to DEPS.
- Otherwise, add Ninja to your PATH *after* depot_tools.
Error (ninja): 1
Error (./make.py): 1

I checked "\core\Common\3dParty\v8_89\depot_tools" and found the "ninja" file. Why did this error appear? How should I solve it?

igwyd commented 1 week ago

I think there was some failture, yesterday I checked the compilation in docker (Ubuntu 16.04) https://github.com/ONLYOFFICE/build_tools?tab=readme-ov-file#using-docker - there was no error. Try again, on the machine 4cpu 8 ram (8gb swap), ~60 free space ssd, clean updated server.

zsy1314 commented 1 week ago

I think there was some failture, yesterday I checked the compilation in docker (Ubuntu 16.04) https://github.com/ONLYOFFICE/build_tools?tab=readme-ov-file#using-docker - there was no error. Try again, on the machine 4cpu 8 ram (8gb swap), ~60 free space ssd, clean updated server.

I recreated the container today using the Ubuntu 16.04 image of Docker and tried again. Follow the steps below to run the command:

apt-get update
apt-get upgrade
apt-get install -y python git sudo
git clone https://github.com/ONLYOFFICE/build_tools.git
cd build_tools/tools/linux
./automate.py server

The complete record in the terminal is as follows:

# cd build_tools/tools/linux
# ./automate.py server
install dependencies...
---------------------------------------------
build branch: master
---------------------------------------------
---------------------------------------------
build modules: server
---------------------------------------------
[git] update: core
Already on 'master'
Your branch is up-to-date with 'origin/master'.
Already up-to-date.
[git] update: document-server-integration
Already on 'master'
Your branch is up-to-date with 'origin/master'.
Submodule path 'web/documentserver-example/csharp-mvc/assets/document-formats': checked out '2c3e099f1e8eaaf7070cdb81643de9586a2235e2'
Submodule path 'web/documentserver-example/csharp-mvc/assets/document-templates': checked out '1fc823afa909e4c49551e4e5b945189a21ff1999'
Submodule path 'web/documentserver-example/csharp/assets/document-formats': checked out '2c3e099f1e8eaaf7070cdb81643de9586a2235e2'
Submodule path 'web/documentserver-example/csharp/assets/document-templates': checked out '1fc823afa909e4c49551e4e5b945189a21ff1999'
Submodule path 'web/documentserver-example/java-spring/src/main/resources/assets/document-formats': checked out '05bf7a6e6d0a483a117a9853dc51bb517679677a'
Submodule path 'web/documentserver-example/java-spring/src/main/resources/assets/document-templates': checked out '1fc823afa909e4c49551e4e5b945189a21ff1999'
Submodule path 'web/documentserver-example/java/src/main/resources/assets/document-formats': checked out '05bf7a6e6d0a483a117a9853dc51bb517679677a'
Submodule path 'web/documentserver-example/java/src/main/resources/assets/document-templates': checked out '1fc823afa909e4c49551e4e5b945189a21ff1999'
Already up-to-date.
Submodule path 'web/documentserver-example/csharp-mvc/assets/document-formats': checked out '2b592018adbf03d9d6a9f1ab69b1e9d8af73e22c'
Submodule path 'web/documentserver-example/csharp-mvc/assets/document-templates': checked out 'c9fc1ee6beac118a7d9472f971a61cd2eb285293'
Submodule path 'web/documentserver-example/csharp/assets/document-formats': checked out '2b592018adbf03d9d6a9f1ab69b1e9d8af73e22c'
Submodule path 'web/documentserver-example/csharp/assets/document-templates': checked out 'c9fc1ee6beac118a7d9472f971a61cd2eb285293'
Submodule path 'web/documentserver-example/java-spring/src/main/resources/assets/document-formats': checked out '2b592018adbf03d9d6a9f1ab69b1e9d8af73e22c'
Submodule path 'web/documentserver-example/java-spring/src/main/resources/assets/document-templates': checked out 'c9fc1ee6beac118a7d9472f971a61cd2eb285293'
Submodule path 'web/documentserver-example/java/src/main/resources/assets/document-formats': checked out '2b592018adbf03d9d6a9f1ab69b1e9d8af73e22c'
Submodule path 'web/documentserver-example/java/src/main/resources/assets/document-templates': checked out 'c9fc1ee6beac118a7d9472f971a61cd2eb285293'
fatal: Needed a single revision
Unable to find current origin/feature/v8.0 revision in submodule path 'web/documentserver-example/nodejs/public/assets/document-formats'
[git] update: sdkjs
Already on 'master'
Your branch is up-to-date with 'origin/master'.
Already up-to-date.
[git] update: document-templates
Already on 'master'
Your branch is up-to-date with 'origin/master'.
Already up-to-date.
[git] update: sdkjs-forms
Already on 'master'
Your branch is up-to-date with 'origin/master'.
Already up-to-date.
[git] update: core-fonts
Already on 'master'
Your branch is up-to-date with 'origin/master'.
Already up-to-date.
[git] update: server
Already on 'master'
Your branch is up-to-date with 'origin/master'.
Already up-to-date.
[git] update: dictionaries
Already on 'master'
Your branch is up-to-date with 'origin/master'.
Already up-to-date.
[git] update: onlyoffice.github.io
Already on 'master'
Your branch is up-to-date with 'origin/master'.
Already up-to-date.
[git] update: web-apps
Already on 'master'
Your branch is up-to-date with 'origin/master'.
Already up-to-date.
[fetch & build]: boost
[fetch & build]: cef
[fetch & build]: icu
[fetch & build]: openssl
gn gen out.gn/linux_64 --args="v8_static_library=true is_component_build=false v8_monolithic=true v8_use_external_startup_data=false use_custom_libcxx=false treat_warnings_as_errors=false target_cpu=\"x64\" v8_target_cpu=\"x64\" is_debug=false is_clang=true use_sysroot=false"
Done. Made 165 targets from 92 files in 348ms
depot_tools/ninja.py: Could not find Ninja in the third_party of the current project, nor in your PATH.
Please take one of the following actions to install Ninja:
- If your project has DEPS, add a CIPD Ninja dependency to DEPS.
- Otherwise, add Ninja to your PATH *after* depot_tools.
Error (ninja): 1
Error (./make.py): 1
#

It still says that Ninja is missing.

igwyd commented 6 days ago

apt-get update apt-get upgrade apt-get install -y python git sudo git clone https://github.com/ONLYOFFICE/build_tools.git cd build_tools/tools/linux ./automate.py server

I repeted it on my server with clean Dockerfile FROM ubuntu:16.04 and still got no erros. maybe you forget any steps in your case?

zsy1314 commented 1 day ago

The reason why I didn't use Dockerfile to build directly is that if a package fails to download in the middle of a bad network, I need to start the build again from the beginning. I don't know if there is a way to continue from where it was interrupted?

So I ran the commands line by line according to the content in Dockerfile:

docker pull ubuntu:16.04
docker run -i -t -p 8022:22 --name ubuntu ubuntu:16.04
apt-get -y update
apt-get -y install python python3 sudo
rm /usr/bin/python
ln -s /usr/bin/python2 /usr/bin/python
git clone https://github.com/ONLYOFFICE/build_tools.git
cd build_tools/tools/linux
python3 ./automate.py server

But I still got the same error as before.

zsy1314 commented 1 day ago

I just tried to build using Dockerfile again. 

mkdir out
docker build --tag onlyoffice-document-editors-builder .
docker run -v $PWD/out:/build_tools/out onlyoffice-document-editors-builder

Fortunately, the network condition was good and it did not stop because of a package download failure. However, in the end, there was still an error message similar to that about Ninja:

[0:13:19] Started.
----------------------------------------
Traceback (most recent call last):
  File "/core/Common/3dParty/v8_89/depot_tools/metrics.py", line 290, in print_notice_and_exit
    yield
  File "/build_tools/scripts/../../core/Common/3dParty/v8_89/depot_tools/gclient.py", line 4622, in <module>
    sys.exit(main(sys.argv[1:]))
             ^^^^^^^^^^^^^^^^^^
  File "/build_tools/scripts/../../core/Common/3dParty/v8_89/depot_tools/gclient.py", line 4608, in main
    return dispatcher.execute(OptionParser(), argv)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/core/Common/3dParty/v8_89/depot_tools/subcommand.py", line 254, in execute
    return command(parser, args[1:])
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/build_tools/scripts/../../core/Common/3dParty/v8_89/depot_tools/gclient.py", line 3961, in CMDsync
    ret = client.RunOnDeps('update', args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/build_tools/scripts/../../core/Common/3dParty/v8_89/depot_tools/gclient.py", line 2434, in RunOnDeps
    work_queue.flush(revision_overrides,
  File "/core/Common/3dParty/v8_89/depot_tools/gclient_utils.py", line 1026, in flush
    reraise(e[0], e[1], e[2])
  File "/core/Common/3dParty/v8_89/depot_tools/gclient_utils.py", line 53, in reraise
    raise value
  File "/core/Common/3dParty/v8_89/depot_tools/gclient_utils.py", line 1105, in run
    self.item.run(*self.args, **self.kwargs)
  File "/build_tools/scripts/../../core/Common/3dParty/v8_89/depot_tools/gclient.py", line 2720, in run
    self.DownloadGoogleStorage()
  File "/build_tools/scripts/../../core/Common/3dParty/v8_89/depot_tools/gclient.py", line 2825, in DownloadGoogleStorage
    raise Exception(f'{code}: {err}')
Exception: 1: Traceback (most recent call last):
  File "/core/Common/3dParty/v8_89/depot_tools/lockfile.py", line 90, in _lock
    return _try_lock(path + '.locked')
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/core/Common/3dParty/v8_89/depot_tools/lockfile.py", line 73, in _try_lock
    _lock_file(f)
  File "/core/Common/3dParty/v8_89/depot_tools/lockfile.py", line 67, in _lock_file
    fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
BlockingIOError: [Errno 11] Resource temporarily unavailable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/core/Common/3dParty/v8_89/depot_tools/gsutil.py", line 318, in <module>
    sys.exit(main())
             ^^^^^^
  File "/core/Common/3dParty/v8_89/depot_tools/gsutil.py", line 314, in main
    return run_gsutil(args.target, args.args, clean=args.clean)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/core/Common/3dParty/v8_89/depot_tools/gsutil.py", line 222, in run_gsutil
    gsutil_bin = ensure_gsutil(VERSION, target, clean)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/core/Common/3dParty/v8_89/depot_tools/gsutil.py", line 106, in ensure_gsutil
    with lockfile.lock(bin_dir, timeout=30):
  File "/root/.cache/.vpython-root/store/cpython+qqq5mqooe6c0qbcqhqdvpqhqlc/contents/lib/python3.11/contextlib.py", line 137, in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
  File "/core/Common/3dParty/v8_89/depot_tools/lockfile.py", line 113, in lock
    release_fn = _lock(path, timeout)
                 ^^^^^^^^^^^^^^^^^^^^
  File "/core/Common/3dParty/v8_89/depot_tools/lockfile.py", line 99, in _lock
    raise LockError("Error locking %s (err: %s)" % (path, str(e)))
lockfile.LockError: Error locking /core/Common/3dParty/v8_89/depot_tools/external_bin/gsutil/gsutil_4.68 (err: [Errno 11] Resource temporarily unavailable)

Running: gclient root
Running: gclient config --spec 'solutions = [
  {
    "name": "v8",
    "url": "https://chromium.googlesource.com/v8/v8.git",
    "deps_file": "DEPS",
    "managed": False,
    "custom_deps": {},
  },
]
'
Running: gclient sync --with_branch_heads
Subprocess failed with return code 1.
./cipd: line 137: ./depot_tools/cipd_client_version.digests: No such file or directory
Platform linux-amd64 is not supported by the CIPD client bootstrap: there's no pinned SHA256 hash for it in the *.digests file.
error: unknown option `type'
usage: git config [<options>]

Config file location
    --global              use global config file
    --system              use system config file
    --local               use repository config file
    -f, --file <file>     use given config file
    --blob <blob-id>      read config from given blob object

Action
    --get                 get value: name [value-regex]
    --get-all             get all values: key [value-regex]
    --get-regexp          get values for regexp: name-regex [value-regex]
    --get-urlmatch        get value specific for the URL: section[.var] URL
    --replace-all         replace all matching variables: name value [value_regex]
    --add                 add a new variable: name value
    --unset               remove a variable: name [value-regex]
    --unset-all           remove all matches: name [value-regex]
    --rename-section      rename section: old-name new-name
    --remove-section      remove a section: name
    -l, --list            list all
    -e, --edit            open an editor
    --get-color           find the color configured: slot [default]
    --get-colorbool       find the color setting: slot [stdout-is-tty]

Type
    --bool                value is "true" or "false"
    --int                 value is decimal number
    --bool-or-int         value is --bool or --int
    --path                value is a path (file or directory name)

Other
    -z, --null            terminate values with NUL byte
    --name-only           show variable names only
    --includes            respect include directives on lookup

________ running 'python third_party/depot_tools/update_depot_tools_toggle.py --disable' in '/core/Common/3dParty/v8_89/v8'
________ running 'python build/landmines.py --landmine-scripts tools/get_landmines.py' in '/core/Common/3dParty/v8_89/v8'
________ running 'download_from_google_storage --no_resume --platform=linux* --no_auth --bucket chromium-clang-format -s buildtools/linux64/clang-format.sha1' in '/core/Common/3dParty/v8_89/v8'
0> Downloading buildtools/linux64/clang-format@1baf0089e895c989a311b6a38ed94d0e8be4c0a7...
Downloading 1 files took 26.283965 second(s)
Hook 'download_from_google_storage --no_resume '--platform=linux*' --no_auth --bucket chromium-clang-format -s buildtools/linux64/clang-format.sha1' took 26.39 secs
________ running 'download_from_google_storage --no_resume --no_auth -u --bucket v8-wasm-spec-tests -s test/wasm-spec-tests/tests.tar.gz.sha1' in '/core/Common/3dParty/v8_89/v8'
0> Downloading test/wasm-spec-tests/tests.tar.gz@38edac624024750d4d35619df1bbdc7902f9cb7c...
0> Extracting 471 entries from test/wasm-spec-tests/tests.tar.gz to test/wasm-spec-tests/tests
Downloading 1 files took 16.069340 second(s)
Hook 'download_from_google_storage --no_resume --no_auth -u --bucket v8-wasm-spec-tests -s test/wasm-spec-tests/tests.tar.gz.sha1' took 16.27 secs
________ running 'download_from_google_storage --no_resume --no_auth -u --bucket v8-wasm-spec-tests -s test/wasm-js/tests.tar.gz.sha1' in '/core/Common/3dParty/v8_89/v8'
0> Downloading test/wasm-js/tests.tar.gz@31c11a41026c56be3c6d6470755d476840ce0132...
0> Extracting 142 entries from test/wasm-js/tests.tar.gz to test/wasm-js/tests
Downloading 1 files took 6.827601 second(s)
________ running 'python build/linux/sysroot_scripts/install-sysroot.py --arch=x86' in '/core/Common/3dParty/v8_89/v8'
Installing Debian sid i386 root image: /core/Common/3dParty/v8_89/v8/build/linux/debian_sid_i386-sysroot
Downloading https://commondatastorage.googleapis.com/chrome-linux-sysroot/toolchain/5a09b89bfdd9ff4c6b49f5f418124c325259c854/debian_sid_i386_sysroot.tar.xz
Hook 'python build/linux/sysroot_scripts/install-sysroot.py --arch=x86' took 39.57 secs
________ running 'python build/linux/sysroot_scripts/install-sysroot.py --arch=x64' in '/core/Common/3dParty/v8_89/v8'
Installing Debian sid amd64 root image: /core/Common/3dParty/v8_89/v8/build/linux/debian_sid_amd64-sysroot
Downloading https://commondatastorage.googleapis.com/chrome-linux-sysroot/toolchain/22f2db7711f7426a364617bb6d78686cce09a8f9/debian_sid_amd64_sysroot.tar.xz
Hook 'python build/linux/sysroot_scripts/install-sysroot.py --arch=x64' took 37.52 secs
________ running 'python tools/clang/scripts/update.py' in '/core/Common/3dParty/v8_89/v8'
Downloading https://commondatastorage.googleapis.com/chromium-browser-clang/Linux_x64/clang-llvmorg-12-init-16296-g5e476061-1.tgz .......... Done.
Hook 'python tools/clang/scripts/update.py' took 36.71 secs
________ running 'python build/util/lastchange.py -o build/util/LASTCHANGE' in '/core/Common/3dParty/v8_89/v8'
error: unknown option `type'
usage: git config [<options>]

Config file location
    --global              use global config file
    --system              use system config file
    --local               use repository config file
    -f, --file <file>     use given config file
    --blob <blob-id>      read config from given blob object

Action
    --get                 get value: name [value-regex]
    --get-all             get all values: key [value-regex]
    --get-regexp          get values for regexp: name-regex [value-regex]
    --get-urlmatch        get value specific for the URL: section[.var] URL
    --replace-all         replace all matching variables: name value [value_regex]
    --add                 add a new variable: name value
    --unset               remove a variable: name [value-regex]
    --unset-all           remove all matches: name [value-regex]
    --rename-section      rename section: old-name new-name
    --remove-section      remove a section: name
    -l, --list            list all
    -e, --edit            open an editor
    --get-color           find the color configured: slot [default]
    --get-colorbool       find the color setting: slot [stdout-is-tty]

Type
    --bool                value is "true" or "false"
    --int                 value is decimal number
    --bool-or-int         value is --bool or --int
    --path                value is a path (file or directory name)

Other
    -z, --null            terminate values with NUL byte
    --name-only           show variable names only
    --includes            respect include directives on lookup

________ running 'python third_party/depot_tools/update_depot_tools_toggle.py --disable' in '/core/Common/3dParty/v8_89/v8'
________ running 'python build/landmines.py --landmine-scripts tools/get_landmines.py' in '/core/Common/3dParty/v8_89/v8'
________ running 'download_from_google_storage --no_resume --platform=linux* --no_auth --bucket chromium-clang-format -s buildtools/linux64/clang-format.sha1' in '/core/Common/3dParty/v8_89/v8'
________ running 'download_from_google_storage --no_resume --no_auth -u --bucket v8-wasm-spec-tests -s test/wasm-spec-tests/tests.tar.gz.sha1' in '/core/Common/3dParty/v8_89/v8'
________ running 'download_from_google_storage --no_resume --no_auth -u --bucket v8-wasm-spec-tests -s test/wasm-js/tests.tar.gz.sha1' in '/core/Common/3dParty/v8_89/v8'
________ running 'python build/linux/sysroot_scripts/install-sysroot.py --arch=x86' in '/core/Common/3dParty/v8_89/v8'
________ running 'python build/linux/sysroot_scripts/install-sysroot.py --arch=x64' in '/core/Common/3dParty/v8_89/v8'
________ running 'python tools/clang/scripts/update.py' in '/core/Common/3dParty/v8_89/v8'
________ running 'python build/util/lastchange.py -o build/util/LASTCHANGE' in '/core/Common/3dParty/v8_89/v8'
Done. Made 165 targets from 92 files in 435ms
depot_tools/ninja.py: Could not find Ninja in the third_party of the current project, nor in your PATH.
Please take one of the following actions to install Ninja:
- If your project has DEPS, add a CIPD Ninja dependency to DEPS.
- Otherwise, add Ninja to your PATH *after* depot_tools.
[git] update: core
[git] update: document-server-integration
[git] update: sdkjs
[git] update: document-templates
[git] update: sdkjs-forms
[git] update: core-fonts
[git] update: server
[git] update: dictionaries
[git] update: onlyoffice.github.io
[git] update: web-apps
[fetch & build]: boost
delete warning [file not exist]: ./boost.data
[fetch & build]: cef
delete warning [file not exist]: ./cef_binary.7z.data
[fetch & build]: icu
[fetch & build]: openssl
delete warning [file not exist]: ./openssl.data
Directory not copied
gn gen out.gn/linux_64 --args="v8_static_library=true is_component_build=false v8_monolithic=true v8_use_external_startup_data=false use_custom_libcxx=false treat_warnings_as_errors=false target_cpu=\"x64\" v8_target_cpu=\"x64\" is_debug=false is_clang=true use_sysroot=false"
Error (ninja): 1
install dependencies...
Node.js version cannot be less 16
Reinstall
install qt...
---------------------------------------------
build branch:
---------------------------------------------
---------------------------------------------
build modules: server
---------------------------------------------
Error (./make.py): 1

But this time it mentioned Node.js version cannot be less 16

igwyd commented 1 day ago

Unfortunately, there are issue in countries with Internet blocks. Try using a VPN or cloud server in a country without such restrictions.