search

google / voice-builder

An opensource text-to-speech (TTS) voice building tool
http://tinyurl.com/voice-builder
Apache License 2.0
654 stars 135 forks source link

stuck at exporting resource #9

Closed mirfan899 closed 4 years ago

mirfan899 commented 5 years ago

I'm having an issue with running TTS. I've set up all the things. Here is the setup shown in the screenshot.

Screen Shot 2019-06-19 at 11 13 01 AM

Error when I try to look pipeline logs

Screen Shot 2019-06-19 at 11 14 06 AM

When I checked the logs it shows

{
    "insertId": "1ne32gpfr3cga3",
    "jsonPayload": {
      "container_id": "2d612ea2b6e42ca617ffb83af121e8165d8dc3987dd6c1f8c373038332e9ea2f",
      "stream": "stderr",
      "message": "  message: 'No such object: mytts-jobs/1/pipelineInfo.json' }\n"
    },
    "resource": {
      "type": "gce_instance",
      "labels": {
        "instance_id": "5637591319231887099",
        "zone": "asia-east1-c",
        "project_id": "mytts-241901"
      }
    },
    "timestamp": "2019-06-19T05:57:56.663192638Z",
    "labels": {
      "compute.googleapis.com/resource_name": "16a2495c45d0"
    },
    "logName": "projects/mytts-241901/logs/cos_containers",
    "receiveTimestamp": "2019-06-19T05:58:02.550378725Z"
  }

It seems Genomics API is not working after enabling it.

Screen Shot 2019-06-19 at 11 22 21 AM Screen Shot 2019-06-19 at 11 20 12 AM

Did I do something wrong? Or missed a step.

mirfan899 commented 5 years ago

Here are logs of the ui instance logs.zip

mirfan899 commented 5 years ago

After doing a lot of debugging, finally found out that genomics function related to pipelineInfo.json not working while other functions are working. https://github.com/google/voice-builder/blob/ed86261f3953e8256cba9ba60a66fdbcbf265c3e/cloud_functions/functions/jobStatus.js#L86 This thing does not work, I tried to find the logs but even logs are not being generated.

Screen Shot 2019-06-21 at 7 47 43 PM

Need help.

pasindud commented 5 years ago

Hi, Let me look into this.

mirfan899 commented 5 years ago

@pasindud any update?

pasindud commented 5 years ago

Can you delete the file _resource_export_successful and re-create that file in GCS, that should start the genomics pipeline. That should restart the genomics pipeline.

mirfan899 commented 5 years ago

I've deleted the file using gsutil and then created(no gsutil command to create a file) with by uploading a new file. But the issue still persists. There are still no logs indicating Genomics Operations are running.

mirfan899 commented 5 years ago

any update on the issue @pasindud?

pasindud commented 5 years ago

can you check whether genomics is actually enabled ?

can you try the genomics example here to verify that genomics is actually working

mirfan899 commented 5 years ago
Screen Shot 2019-06-27 at 10 18 29 PM

and logs

Screen Shot 2019-06-27 at 10 19 13 PM

bucket

Screen Shot 2019-06-27 at 10 20 55 PM
mirfan899 commented 5 years ago

Still stuck at this point. If you want @pasindud, I can add you to the GCP project to see what is the issue.

mirfan899 commented 5 years ago

Here are logs for setting up the ui, maybe it helps to find out where is the issue.

Creating gs://mytts-jobs/...
ServiceException: 409 Bucket mytts-jobs already exists.
Creating gs://mytts-resource-library/...
ServiceException: 409 Bucket mytts-resource-library already exists.
Setting a job counter (appData/jobCounter.txt) to 0...
Copying file:///tmp/jobCounter.txt [Content-Type=text/plain]...
/ [1 files][    2.0 B/    2.0 B]
Operation completed over 1 objects/2.0 B.
Start deploying cloud_function to /tmp/tmp.KMCdWvf3RV...
/tmp/tmp.KMCdWvf3RV
yarn install v1.17.3
[1/5] Validating package.json...
[2/5] Resolving packages...
[3/5] Fetching packages...
[4/5] Linking dependencies...
warning "firebase-admin > @firebase/database@0.2.2" has unmet peer dependency "@firebase/app-types@0.x".
warning "firebase-admin > @firebase/database > @firebase/database-types@0.2.1" has unmet peer dependency "@firebase/app-types@0.x".
warning " > firebase-functions@0.7.5" has incorrect peer dependency "firebase-admin@~5.5.0".
warning " > eslint-config-airbnb@16.1.0" has unmet peer dependency "eslint-plugin-jsx-a11y@^6.0.2".
warning " > eslint-config-airbnb@16.1.0" has unmet peer dependency "eslint-plugin-react@^7.4.0".
[5/5] Building fresh packages...
Done in 24.73s.
⚠  functions: package.json indicates an outdated version of firebase-functions.
 Please upgrade using npm install --save firebase-functions@latest in your functions directory.

=== Deploying to 'mytts-241901'...

i  deploying functions
i  functions: ensuring necessary APIs are enabled...
✔  functions: all necessary APIs are enabled
i  functions: preparing functions directory for uploading...
⚠  functions: You must have a firebase-functions version that is at least 2.0.0. Please run npm i --save firebase-functions@latest in the functions folder.
i  functions: packaged functions (44.86 KB) for uploading
✔  functions: functions folder uploaded successfully
i  functions: updating Node.js 8 function onJobBucketChange(us-central1)...
✔  functions[onJobBucketChange(us-central1)]: Successful update operation.

✔  Deploy complete!

Project Console: https://console.firebase.google.com/project/mytts-241901/overview
Deploying ui server to /tmp/tmp.0ZIqLLAGHd...
+ '[' 3 -ne 3 ']'
+ IMAGE=gcr.io/mytts-241901/ui:latest
+ DIR=/tmp/tmp.0ZIqLLAGHd
+ PROJECT_ID=mytts-241901
+ DOCKER_CMD='docker build --no-cache -t gcr.io/mytts-241901/ui:latest /tmp/tmp.0ZIqLLAGHd'
++ date +%s
+ TIMESTAMP=1564470960
+ echo 'timestamp: 1564470960'
timestamp: 1564470960
+ echo 'running: docker build --no-cache -t gcr.io/mytts-241901/ui:latest /tmp/tmp.0ZIqLLAGHd'
running: docker build --no-cache -t gcr.io/mytts-241901/ui:latest /tmp/tmp.0ZIqLLAGHd
+ eval 'docker build --no-cache -t gcr.io/mytts-241901/ui:latest /tmp/tmp.0ZIqLLAGHd'
++ docker build --no-cache -t gcr.io/mytts-241901/ui:latest /tmp/tmp.0ZIqLLAGHd
latest: Pulling from google_appengine/nodejs
40a5c2875f88: Pull complete
f038781a2d28: Pull complete
3c2cba919283: Pull complete
c0bc3a0cb318: Pull complete
e2eb46df15bc: Pull complete
52d86b9db865: Pull complete
b3fa3378d476: Pull complete
c950d97006b5: Pull complete
79fe06aff4da: Pull complete
1cf7a72014ca: Pull complete
Digest: sha256:fb2607cf25076fe8e1ab4feaa957816d3b6672d4c61151d072307808eb4e10c4
Status: Downloaded newer image for gcr.io/google_appengine/nodejs:latest
 ---> e7d01e511ceb
Step 2/12 : RUN /usr/local/bin/install_node '>=v8.6.0'
 ---> Running in 134eef6a6334
Removing intermediate container 134eef6a6334
 ---> 8226ba49eeeb
Step 3/12 : WORKDIR /app
 ---> Running in b647bd41a628
Removing intermediate container b647bd41a628
 ---> a8e49cba1ca4
Step 4/12 : ENV NODE_ENV development
 ---> Running in 785a9d44b03a
Removing intermediate container 785a9d44b03a
 ---> e2fbcf9a9a65
Step 5/12 : COPY package.json /app
 ---> b582e304b8d9
Step 6/12 : RUN yarn --unsafe-perm ||   ((if [ -f npm-debug.log ]; then       cat npm-debug.log;     fi) && false)
 ---> Running in 8abbe765b4fc
yarn install v1.17.3
warning package.json: No license field
info No lockfile found.
warning voicebuilder_ui@1.0.0: No license field
[1/4] Resolving packages...
warning autoprefixer > browserslist@1.7.7: Browserslist 2 could fail on reading Browserslist >3.0 config used in other tools.
warning babel-preset-es2015@6.24.1: 🙌  Thanks for using Babel: we recommend using babel-preset-env now: please read https://babeljs.io/env to update!
warning css-loader > cssnano > postcss-merge-rules > browserslist@1.7.7: Browserslist 2 could fail on reading Browserslist >3.0 config used in other tools.
warning css-loader > cssnano > postcss-merge-rules > caniuse-api > browserslist@1.7.7: Browserslist 2 could fail on reading Browserslist >3.0 config used in other tools.
warning css-loader > cssnano > postcss-merge-rules > postcss-selector-parser > flatten@1.0.2: I wrote this module a very long time ago; you should use something else.
warning eslint > file-entry-cache > flat-cache > circular-json@0.3.3: CircularJSON is in maintenance only, flatted is its successor.
warning karma > socket.io > socket.io-parser > json3@3.3.2: Please use the native JSON object instead of JSON 3
warning karma-coverage > istanbul@0.4.5: This module is no longer maintained, try this instead:
  npm i nyc
Visit https://istanbul.js.org/integrations for other alternatives.
warning karma-phantomjs-launcher > phantomjs-prebuilt@2.1.16: this package is now deprecated
warning phantomjs-prebuilt@2.1.16: this package is now deprecated
[2/4] Fetching packages...
info fsevents@1.2.9: The platform "linux" is incompatible with this module.
info "fsevents@1.2.9" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
warning " > angular-material@1.1.19" has unmet peer dependency "angular-messages@1.4 - 1.7.0 || ^1.7.2".
warning " > eslint-config-airbnb@15.1.0" has incorrect peer dependency "eslint-plugin-jsx-a11y@^5.1.1".
[4/4] Building fresh packages...
success Saved lockfile.
Done in 79.52s.
Removing intermediate container 8abbe765b4fc
 ---> b1ef3f7df594
Step 7/12 : COPY . /app
 ---> d9dd91386974
Step 8/12 : RUN yarn build
 ---> Running in f661abe221b6
yarn run v1.17.3
warning package.json: No license field
$ ./node_modules/webpack/bin/webpack.js --config=webpack.prod.js
clean-webpack-plugin: /app/dist has been removed.
Hash: 2c5eb8bba4e45e7b8e2e
Version: webpack 2.7.0
Time: 42436ms
                                                       Asset       Size  Chunks                    Chunk Names
                          components/jobs/job-detail-view.ng     3.4 kB          [emitted]
                              client.2c5eb8bba4e45e7b8e2e.js     909 kB       0  [emitted]  [big]  client
                                          css/client.css.map     413 kB       0  [emitted]         client
                                                  index.html    2.76 kB          [emitted]
                components/create-voice/create-voice-view.ng    4.07 kB          [emitted]
components/create-voice/resource-library-listing-template.ng    2.24 kB          [emitted]
    components/create-voice/voice-combined-input-template.ng  395 bytes          [emitted]
             components/create-voice/voice-input-template.ng   92 bytes          [emitted]
            components/create-voice/voice-select-template.ng  195 bytes          [emitted]
                                              css/client.css     344 kB       0  [emitted]  [big]  client
                           components/jobs/list-jobs-view.ng  738 bytes          [emitted]
                 components/pipelines/list-pipelines-view.ng  969 bytes          [emitted]
                components/pipelines/pipeline-detail-view.ng  143 bytes          [emitted]
 components/resource-library/add-resource-dialog-template.ng    1.39 kB          [emitted]
      components/resource-library/add-wav-dialog-template.ng    1.67 kB          [emitted]
        components/resource-library/nested-table-template.ng  962 bytes          [emitted]
        components/resource-library/resource-library-view.ng  490 bytes          [emitted]
               components/wavesurfer/wave-player-template.ng  587 bytes          [emitted]
   [1] ./~/jquery/dist/jquery.js 280 kB {0} [built]
   [3] ./~/angular-animate/index.js 60 bytes {0} [built]
   [4] ./~/angular-aria/index.js 54 bytes {0} [built]
   [5] ./~/angular/index.js 48 bytes {0} [built]
[6] ./src/app/app.js 7.18 kB {0} [built]
   [7] ./~/angular-animate/angular-animate.js 157 kB {0} [built]
   [8] ./~/angular-aria/angular-aria.js 16.5 kB {0} [built]
   [9] ./~/angular-material/angular-material.js 1.33 MB {0} [built]
  [10] ./~/angular-material/index.js 270 bytes {0} [built]
  [11] ./~/angular-route/angular-route.js 46.9 kB {0} [built]
  [12] ./~/angular-route/index.js 56 bytes {0} [built]
  [28] ./~/datatables.net/js/jquery.dataTables.js 448 kB {0} [built]
  [29] ./~/angular-material/angular-material.min.css 41 bytes {0} [built]
  [30] ./src/css/style.css 41 bytes {0} [built]
  [35] multi ./src/app/app.js 28 bytes {0} [built]
    + 21 hidden modules
Child html-webpack-plugin for "index.html":
       [0] ./~/html-webpack-plugin/lib/loader.js!./src/public/index.html 2.76 kB {0} [built]
Child extract-text-webpack-plugin:
       [0] ./~/css-loader/lib/css-base.js 1.51 kB {0} [built]
       [1] ./~/css-loader?{"sourceMap":true}!./~/postcss-loader!./src/css/style.css 8.37 kB {0} [built]
Child extract-text-webpack-plugin:
       [0] ./~/css-loader/lib/css-base.js 1.51 kB {0} [built]
       [1] ./~/css-loader?{"sourceMap":true}!./~/postcss-loader!./~/angular-material/angular-material.min.css 758 kB {0} [built]
Done in 45.71s.
Removing intermediate container f661abe221b6
 ---> 4511963a2230
Step 9/12 : ENV PORT 3389
 ---> Running in 320ab59b0ec5
Removing intermediate container 320ab59b0ec5
 ---> f306c3bb26ee
Step 10/12 : ENV NODE_ENV production
 ---> Running in d1c19c22ea4b
Removing intermediate container d1c19c22ea4b
 ---> 08c783fe4c69
Step 11/12 : EXPOSE 3389
 ---> Running in 6a0dd9e8b636
Removing intermediate container 6a0dd9e8b636
 ---> 4842c0682009
Step 12/12 : CMD ["node", "server.js"]
 ---> Running in 87893bbca713
Removing intermediate container 87893bbca713
 ---> ca844317705e
Successfully built ca844317705e
Successfully tagged gcr.io/mytts-241901/ui:latest
+ gcloud docker --project=mytts-241901 -- push gcr.io/mytts-241901/ui:latest
WARNING: `gcloud docker` will not be supported for Docker client versions above 18.03.

As an alternative, use `gcloud auth configure-docker` to configure `docker` to
use `gcloud` as a credential helper, then use `docker` as you would for non-GCR
registries, e.g. `docker pull gcr.io/project-id/my-image`. Add
`--verbosity=error` to silence this warning: `gcloud docker
--verbosity=error -- pull gcr.io/project-id/my-image`.

See: https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker

The push refers to repository [gcr.io/mytts-241901/ui]
a091da3396f0: Pushed
c5f2fecf8227: Pushed
d6b4c7b95539: Pushed
1dc68d9a18fc: Pushed
c7d89decece1: Mounted from google_appengine/nodejs
b1eb4063fc9b: Mounted from google_appengine/nodejs
85d3fca18591: Mounted from google_appengine/nodejs
b5524d177a41: Mounted from google_appengine/nodejs
dee362282716: Mounted from google_appengine/nodejs
4e7161a1e8f4: Mounted from google_appengine/nodejs
c1d64ebc5913: Mounted from google_appengine/nodejs
84ff92691f90: Layer already exists
a8cabf03bc75: Mounted from google_appengine/nodejs
f66b9865f45c: Mounted from google_appengine/nodejs
latest: digest: sha256:e13dd5ed0612dd1ea4c6a4d6d3f42dc54d77de2417894d04605f7634109ca7e7 size: 3256
WARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance.
Created [https://www.googleapis.com/compute/beta/projects/mytts-241901/zones/asia-east1-c/instances/ui].
WARNING: Some requests generated warnings:
 - Disk size: '50 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.

NAME  ZONE          MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP      STATUS
ui    asia-east1-c  n1-standard-1               x.x.x.x  x.x.x.x  RUNNING
mirfan899 commented 5 years ago

any update?

pasindud commented 5 years ago

Hi @mirfan899, I am currently quite busy, so I wouldn't be able to look into this at least till end of September. Thanks Pasindu

pasindud commented 5 years ago

Can you change n1-standard-1 -> n1-standard-16 and try again

https://github.com/google/voice-builder/blob/3a449a3e8d5100ff323161c89b897f6d5ccdb6f9/ui/backend/gcsApiService.js#L217

mirfan899 commented 5 years ago

Ok, Let's try this.

mirfan899 commented 5 years ago

After removing the cache and deleting the docker images and performing the fresh and clean install, it still stuck on Exporting Resources. Changing file n1-standard-1 -> n1-standard-16 does not work at all. It still uses n1-standard-1 standard machine.

Screen Shot 2019-08-15 at 4 00 53 PM
thejaigoyal commented 4 years ago

Hey @mirfan899 did you get a solution to the Exporting Resources Issue, I'm also stuck at this point

mirfan899 commented 4 years ago

Nope. The code is too old and has a lot of bugs and dependencies issues.

thejaigoyal commented 4 years ago

So did you found any other effective resource to build our own voice?

mirfan899 commented 4 years ago

Yes, this resource is also old but it works. https://github.com/CSTR-Edinburgh/Ossian

thejaigoyal commented 4 years ago

Thank you bud!!