Closed moonnee closed 5 years ago
@moonnee Thanks for reporting it. The same issue was reported from other people and I confirmed to reproduce it.
This is because an incompatible change regarding git_repository
rule was introduced at Bazel 0.20 https://github.com/bazelbuild/bazel/issues/6569 .
I fill fix this issue. In addition, I will add a documentation to run Tacotron without Bazel. Bazel is still beta status so we will have this kind of incompatibility in the future as well. I use Bazel for development but do not use it for experiment. There are some environment that Bazel does not work, e.g. on NFS, and our lab uses NFS for experiment.
Thanks for your work! I am trying to run this repository using LJSpeech. After preprocessing, I run:
bazel run train -- --source-data-root=exp/self-attention-tacotron/training/LJSpeech/source --target-data-root=exp/self-attention-tacotron/training/LJSpeech/target --checkpoint-dir=exp/self-attention-tacotron/logs --selected-list-dir=exp/self-attention-tacotron/examples/ljspeech --hparam-json-file=exp/self-attention-tacotron/examples/ljspeech
I encounter the following errors:INFO: Invocation ID: 74bf1eaf-5af9-4bee-8e0c-9ae8548329c3 ERROR: exp/self-attention-tacotron/BUILD:7:1: no such package '@tacotron2//': The native git_repository rule is deprecated. load("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository") for a replacement. Use --incompatible_remove_native_git_repository=false to temporarily continue using the native rule. and referenced by '//:train' ERROR: Analysis of target '//:train' failed; build aborted: no such package '@tacotron2//': The native git_repository rule is deprecated. load("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository") for a replacement. Use --incompatible_remove_native_git_repository=false to temporarily continue using the native rule. INFO: Elapsed time: 0.082s INFO: 0 processes.
I tried to work around by running
bazel run train -- --source-data-root=exp/self-attention-tacotron/training/LJSpeech/source --target-data-root=exp/self-attention-tacotron/training/LJSpeech/target --checkpoint-dir=exp/self-attention-tacotron/logs --selected-list-dir=exp/self-attention-tacotron/examples/ljspeech --hparam-json-file=exp/self-attention-tacotron/examples/ljspeech --incompatible_remove_native_git_repository=false
, but still no luck. Any ideas? Thanks! The bazel version I use is 0.21.0.
I installed bazel-0.19.0, but still no luck. The errors are as followed:
ERROR: exp/self-attention-tacotron/BUILD:7:1: no such package '@tacotron2//': Error cloning repository: git@github.com:nii-yamagishilab/tacotron2.git: reject HostKey: github.com caused by git@github.com:nii-yamagishilab/tacotron2.git: reject HostKey: github.com caused by reject HostKey: github.com and referenced by '//:train' ERROR: Analysis of target '//:train' failed; build aborted: no such package '@tacotron2//': Error cloning repository: git@github.com:nii-yamagishilab/tacotron2.git: reject HostKey: github.com caused by git@github.com:nii-yamagishilab/tacotron2.git: reject HostKey: github.com caused by reject HostKey: github.com
@moonnee Could you try #8 with new Bazel version?
FYI this is the old version I used. It seems version 0.18 works.
bazel version
WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown".
Build label: 0.18.1-homebrew
Build target: bazel-out/darwin-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Fri Nov 2 11:16:56 2018 (1541157416)
Build timestamp: 1541157416
Build timestamp as int: 1541157416
@moonnee Could you try #8 with new Bazel version?
FYI this is the old version I used. It seems version 0.18 works.
bazel version WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown". Build label: 0.18.1-homebrew Build target: bazel-out/darwin-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar Build time: Fri Nov 2 11:16:56 2018 (1541157416) Build timestamp: 1541157416 Build timestamp as int: 1541157416
Thank you for the reply. I install bazel-0.18.1 but still no luck:
Build label: 0.18.1 Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar Build time: Fri Nov 2 10:09:48 2018 (1541153388) Build timestamp: 1541153388 Build timestamp as int: 1541153388
After I ran
bazel run train -- --source-data-root=exp/self-attention-tacotron/training/LJSpeech/source --target-data-root=exp/self-attention-tacotron/training/LJSpeech/target --checkpoint-dir=exp/self-attention-tacotron/logs --selected-list-dir=exp/self-attention-tacotron/examples/ljspeech --hparam-json-file=exp/self-attention-tacotron/examples/ljspeech
I got the following message:
Starting local Bazel server and connecting to it... ERROR: exp/self-attention-tacotron/BUILD:7:1: no such package '@tacotron2//': Error cloning repository: git@github.com:nii-yamagishilab/tacotron2.git: reject HostKey: github.com caused by git@github.com:nii-yamagishilab/tacotron2.git: reject HostKey: github.com caused by reject HostKey: github.com and referenced by '//:train' ERROR: Analysis of target '//:train' failed; build aborted: no such package '@tacotron2//': Error cloning repository: git@github.com:nii-yamagishilab/tacotron2.git: reject HostKey: github.com caused by git@github.com:nii-yamagishilab/tacotron2.git: reject HostKey: github.com caused by reject HostKey: github.com INFO: Elapsed time: 3.198s INFO: 0 processes. FAILED: Build did NOT complete successfully (8 packages loaded) FAILED: Build did NOT complete successfully (8 packages loaded)
How about Bazel 0.21.0 with this patch #8 ?
How about Bazel 0.21.0 with this patch #8 ?
I tried Bazel 0.21.0 with the patch you modified and encountered the following error :
Starting local Bazel server and connecting to it... INFO: Invocation ID: 3d2220c9-620f-4905-b82b-80c07d4b3bce ERROR: /exp/self-attention-tacotron/BUILD:7:1: no such package '@tacotron2//': Traceback (most recent call last): File "/.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/bazel_tools/tools/build_defs/repo/git.bzl", line 170 _clone_or_update(ctx) File "/.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/bazel_tools/tools/build_defs/repo/git.bzl", line 73, in _clone_or_update fail(("error cloning %s:\n%s" % (ctx....))) error cloning tacotron2:
- cd /.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external
- rm -rf /.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/tacotron2 /.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/tacotron2
- git clone git@github.com:nii-yamagishilab/tacotron2.git /.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/tacotron2 Host key verification failed. fatal: Could not read from remote repository.
Please make sure you have the correct access rights and the repository exists.
- git clone git@github.com:nii-yamagishilab/tacotron2.git /.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/tacotron2 Host key verification failed. fatal: Could not read from remote repository.
Please make sure you have the correct access rights and the repository exists. fatal: Could not read from remote repository.
Please make sure you have the correct access rightsand the repository exists. and referenced by '//:train' ERROR: Analysis of target '//:train' failed; build aborted: no such package '@tacotron2//': Traceback (most recent call last): File "/.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/bazel_tools/tools/build_defs/repo/git.bzl", line 170 _clone_or_update(ctx) File "/.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/bazel_tools/tools/build_defs/repo/git.bzl", line 73, in _clone_or_update fail(("error cloning %s:\n%s" % (ctx....))) error cloning tacotron2:+ cd /.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external
- rm -rf /.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/tacotron2 /.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/tacotron2
- git clone git@github.com:nii-yamagishilab/tacotron2.git /.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/tacotron2 Host key verification failed. fatal: Could not read from remote repository.
Please make sure you have the correct access rights and the repository exists.+ git clone git@github.com:nii-yamagishilab/tacotron2.git /.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/tacotron2 Host key verification failed. fatal: Could not read from remote repository.
Please make sure you have the correct access rights and the repository exists. INFO: Elapsed time: 35.248s INFO: 0 processes. FAILED: Build did NOT complete successfully (9 packages loaded, 22 targets configured) FAILED: Build did NOT complete successfully (9 packages loaded, 22 targets configured) Fetching @tacotron2; Cloning 236169296ca2b5d58f2fa5876867d737769901b4 of git@github.com:nii-yamagishilab/tacotron2.git Starting local Bazel server and connecting to it... INFO: Invocation ID: 3d2220c9-620f-4905-b82b-80c07d4b3bce ERROR: /scratch/usr/exp/self-attention-tacotron/BUILD:7:1: no such package '@tacotron2//': Traceback (most recent call last): File "/.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/bazel_tools/tools/build_defs/repo/git.bzl", line 170 _clone_or_update(ctx) File "/.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/bazel_tools/tools/build_defs/repo/git.bzl", line 73, in _clone_or_update fail(("error cloning %s:\n%s" % (ctx....))) error cloning tacotron2:
- cd /.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external
- rm -rf /.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/tacotron2 /.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/tacotron2
- git clone git@github.com:nii-yamagishilab/tacotron2.git /.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/tacotron2 Host key verification failed. fatal: Could not read from remote repository.
Please make sure you have the correct access rights and the repository exists.
- git clone git@github.com:nii-yamagishilab/tacotron2.git /.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/tacotron2 Host key verification failed. fatal: Could not read from remote repository.
Please make sure you have the correct access rights and the repository exists. fatal: Could not read from remote repository.
Please make sure you have the correct access rightsand the repository exists. and referenced by '//:train' ERROR: Analysis of target '//:train' failed; build aborted: no such package '@tacotron2//': Traceback (most recent call last): File "/.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/bazel_tools/tools/build_defs/repo/git.bzl", line 170 _clone_or_update(ctx) File "/.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/bazel_tools/tools/build_defs/repo/git.bzl", line 73, in _clone_or_update fail(("error cloning %s:\n%s" % (ctx....))) error cloning tacotron2:+ cd /.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external
- rm -rf /.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/tacotron2 /.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/tacotron2
- git clone git@github.com:nii-yamagishilab/tacotron2.git /.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/tacotron2 Host key verification failed. fatal: Could not read from remote repository.
Please make sure you have the correct access rights and the repository exists.+ git clone git@github.com:nii-yamagishilab/tacotron2.git /.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/tacotron2 Host key verification failed. fatal: Could not read from remote repository.
Please make sure you have the correct access rights and the repository exists. INFO: Elapsed time: 35.248s INFO: 0 processes. FAILED: Build did NOT complete successfully (9 packages loaded, 22 targets configured) FAILED: Build did NOT complete successfully (9 packages loaded, 22 targets configured) Fetching @tacotron2; Cloning 236169296ca2b5d58f2fa5876867d737769901b4 of git@github.com:nii-yamagishilab/tacotron2.git
@moonnee It seems your ssh key is rejected.
The new git_repository
rule supports https protocol so I replaced ssh to https in https://github.com/nii-yamagishilab/self-attention-tacotron/pull/8/commits/3fff7da2c5ead32b8581babd73b1ffe2558472c2 .
Please update the patch #8 and try again.
@moonnee It seems your ssh key is rejected. The new
git_repository
rule supports https protocol so I replaced ssh to https in 3fff7da . Please update the patch #8 and try again.
Thanks for your patience. But this doesn't work either.
INFO: Invocation ID: 737c4aed-b698-417e-be27-30efdd83260c
ERROR: /usr/exp/self-attention-tacotron/BUILD:7:1: no such package '@tacotron2//': Traceback (most recent call last):
File "/home/usr/.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/bazel_tools/tools/build_defs/repo/git.bzl", line 170
_clone_or_update(ctx)
File "/home/usr/.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/bazel_tools/tools/build_defs/repo/git.bzl", line 73, in _clone_or_update
fail(("error cloning %s:\n%s" % (ctx....)))
error cloning tacotron2:
+ cd /home/usr/.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external
+ rm -rf /home/usr/.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/tacotron2 /home/usr/.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/tacotron2
+ git clone https://github.com/nii-yamagishilab/tacotron2.git /home/usr/.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/tacotron2
+ git -C /home/usr/.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/tacotron2 reset --hard 236169296ca2b5d58f2fa5876867d737769901b4
Unknown option: -C
usage: git [--version] [--help] [-c name=value]
[--exec-path[=<path>]] [--html-path] [--man-path] [--info-path]
[-p|--paginate|--no-pager] [--no-replace-objects] [--bare]
[--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>]
<command> [<args>]
+ git -C /home/usr/.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/tacotron2 fetch origin 236169296ca2b5d58f2fa5876867d737769901b4:236169296ca2b5d58f2fa5876867d737769901b4
Unknown option: -C
usage: git [--version] [--help] [-c name=value]
[--exec-path[=<path>]] [--html-path] [--man-path] [--info-path]
[-p|--paginate|--no-pager] [--no-replace-objects] [--bare]
[--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>]
<command> [<args>]
+ git -C /home/usr/.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/tacotron2 fetch origin 236169296ca2b5d58f2fa5876867d737769901b4:236169296ca2b5d58f2fa5876867d737769901b4
Unknown option: -C
usage: git [--version] [--help] [-c name=value]
[--exec-path[=<path>]] [--html-path] [--man-path] [--info-path]
[-p|--paginate|--no-pager] [--no-replace-objects] [--bare]
[--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>]
<command> [<args>]
and referenced by '//:train'
ERROR: Analysis of target '//:train' failed; build aborted: no such package '@tacotron2//': Traceback (most recent call last):
File "/home/usr/.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/bazel_tools/tools/build_defs/repo/git.bzl", line 170
_clone_or_update(ctx)
File "/home/usr/.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/bazel_tools/tools/build_defs/repo/git.bzl", line 73, in _clone_or_update
fail(("error cloning %s:\n%s" % (ctx....)))
error cloning tacotron2:
+ cd /home/usr/.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external
+ rm -rf /home/usr/.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/tacotron2 /home/usr/.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/tacotron2
+ git clone https://github.com/nii-yamagishilab/tacotron2.git /home/usr/.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/tacotron2
+ git -C /home/usr/.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/tacotron2 reset --hard 236169296ca2b5d58f2fa5876867d737769901b4
Unknown option: -C
usage: git [--version] [--help] [-c name=value]
[--exec-path[=<path>]] [--html-path] [--man-path] [--info-path]
[--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>]
<command> [<args>]
+ git -C /home/usr/.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/tacotron2 fetch origin 236169296ca2b5d58f2fa5876867d737769901b4:236169296ca2b5d58f2fa5876867d737769901b4Unknown option: -C
usage: git [--version] [--help] [-c name=value]
[--exec-path[=<path>]] [--html-path] [--man-path] [--info-path]
[-p|--paginate|--no-pager] [--no-replace-objects] [--bare]
[--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>]
<command> [<args>]
+ git -C /home/usr/.cache/bazel/_bazel_usr/729f6ad6dc23e393698065b338ee63ee/external/tacotron2 fetch origin 236169296ca2b5d58f2fa5876867d737769901b4:236169296ca2b5d58f2fa5876867d737769901b4Unknown option: -C
usage: git [--version] [--help] [-c name=value]
[--exec-path[=<path>]] [--html-path] [--man-path] [--info-path]
[-p|--paginate|--no-pager] [--no-replace-objects] [--bare]
[--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>]
<command> [<args>]
INFO: Elapsed time: 8.218s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded, 0 targets configured)
FAILED: Build did NOT complete successfully (0 packages loaded, 0 targets configured)
Fetching ...otron2; Cloning 236169296ca2b5d58f2fa5876867d737769901b4 of https://github.com/nii-yamagishilab/tacotron2.git 8s
@moonnee Which version of git do you use? As of 0.20 Bazel uses local git command instead of embedded JGit.
I added a document that describes how to run commands without Bazel. With this way you can avoid Bazel related issues. https://github.com/nii-yamagishilab/self-attention-tacotron/blob/e093804af8c21cda8f5bd5db3be72c9c7fefada2/Bazel.md
@moonnee Which version of git do you use? As of 0.20 Bazel uses local git command instead of embedded JGit.
I added a document that describes how to run commands without Bazel. With this way you can avoid Bazel related issues. https://github.com/nii-yamagishilab/self-attention-tacotron/blob/e093804af8c21cda8f5bd5db3be72c9c7fefada2/Bazel.md
Thanks a lot for your help. My git version is 1.8.3.1. But I still don't know how to run "py_binary listed in the BUILD file" after searching on the Internet (╯︵╰). Hope you can give some detailed explanation. Thank you.
@moonnee Your git version is old so please install the latest version.
FYI my git version is 2.17.1.
git --version
git version 2.17.1
Updating git may solve your ploblems.
I also updated the Bazel documentation for clearity. https://github.com/nii-yamagishilab/self-attention-tacotron/blob/583e0255daa8a1093f5a0ca37e3b5faafbdca106/Bazel.md
@moonnee Your git version is old so please install the latest version.
FYI my git version is 2.17.1.
git --version git version 2.17.1
Updating git may solve your ploblems.
I also updated the Bazel documentation for clearity. https://github.com/nii-yamagishilab/self-attention-tacotron/blob/583e0255daa8a1093f5a0ca37e3b5faafbdca106/Bazel.md
I upgraded my git to 2.20.1, and got new errors after running preprocessing commonds:
bazel run preprocess_ljspeech -- --source-only --hparam-json-file=./examples/ljspeech/self-attention-tacotron.json /data/LJSpeech-1.1 exp/github/self-attention-tacotron/training/source
Starting local Bazel server and connecting to it... INFO: Invocation ID: 644adc8d-7603-4cd2-a91e-70dcf3efbef3 INFO: Analysed target //:preprocess_ljspeech (14 packages loaded, 119 targets configured). INFO: Found 1 target... Target //:preprocess_ljspeech up-to-date: bazel-out/k8-py3-fastbuild/bin/preprocess_ljspeech INFO: Elapsed time: 26.432s, Critical Path: 0.03s INFO: 0 processes. INFO: Build completed successfully, 4 total actions INFO: Running command line: bazel-out/k8-py3-fastbuild/bin/preprocess_ljspeech --source-only '--hparam-json-file=./examples/ljspeech/self-attention-tacotron.json' /data/LJSpeech-1.1 /project/bdda/y INFO: Build completed successfully, 4 total actions Traceback (most recent call last): File "/.cache/bazel/_bazel/5a5901de42a7f8700ed00205840ff26a/execroot/main/bazel-out/k8-py3-fastbuild/bin/preprocess_ljspeech.runfiles/main/preprocess_ljspeech.py", line 23, in
from pyspark import SparkContext ModuleNotFoundError: No module named 'pyspark'
But I already installed pyspark.
Requirement already satisfied: pyspark in /users/anaconda3/lib/python3.6/site-packages (2.4.0) Requirement already satisfied: py4j==0.10.7 in /users/anaconda3/lib/python3.6/site-packages (from pyspark) (0.10.7) I searched a lot but didn't figure out why. Do you know the possible reason? Thank you.
@moonnee It is different topic so please move to a new issue #12
The original issue is fixed so this issue is closed.
Thanks for your work! I am trying to run this repository using LJSpeech. After preprocessing, I run:
bazel run train -- --source-data-root=exp/self-attention-tacotron/training/LJSpeech/source --target-data-root=exp/self-attention-tacotron/training/LJSpeech/target --checkpoint-dir=exp/self-attention-tacotron/logs --selected-list-dir=exp/self-attention-tacotron/examples/ljspeech --hparam-json-file=exp/self-attention-tacotron/examples/ljspeech
I encounter the following errors:I tried to work around by running
bazel run train -- --source-data-root=exp/self-attention-tacotron/training/LJSpeech/source --target-data-root=exp/self-attention-tacotron/training/LJSpeech/target --checkpoint-dir=exp/self-attention-tacotron/logs --selected-list-dir=exp/self-attention-tacotron/examples/ljspeech --hparam-json-file=exp/self-attention-tacotron/examples/ljspeech --incompatible_remove_native_git_repository=false
, but still no luck. Any ideas? Thanks! The bazel version I use is 0.21.0.