search

ibm-ecosystem-engineering / Watson-Speech

This collection demonstrates how to help you to quickly embed Watson Speech in your own applications.
https://techzone.ibm.com/collection/embedded-ai
Apache License 2.0
12 stars 11 forks source link

Error when trying to build #15

Closed syahrul-aiman closed 1 year ago

syahrul-aiman commented 1 year ago

I was trying to follow the following guideline on CentOS 7 VM: https://github.com/ibm-build-lab/Watson-Speech/tree/main/single-container-stt

Tried to build with:

docker build . -t stt-standalone

and

docker build --no-cache . -t stt-standalone

But getting the following error:

STEP 14: RUN ./prepareModels.sh
Serving HTTP on 127.0.0.1 port 3333 (http://127.0.0.1:3333/) ...
127.0.0.1 - - [08/Jun/2023 03:34:37] "GET /pool2/en-US_Multimedia.standard.2022-03-15.4a1f1a7e.tar.pzstd HTTP/1.1" 200 -
127.0.0.1 - - [08/Jun/2023 03:34:37] "GET /pool2/en-US_Multimedia.low-latency.2022-03-15.f7dec0bc.tar.pzstd HTTP/1.1" 200 -
127.0.0.1 - - [08/Jun/2023 03:34:39] "GET /pool2/fr-FR_Multimedia.standard.2022-03-15.5023420e.tar.pzstd HTTP/1.1" 200 -
127.0.0.1 - - [08/Jun/2023 03:34:39] "GET /pool2/spkInfo_16k.2020-10-08.895a1741.tar.pzstd HTTP/1.1" 200 -
127.0.0.1 - - [08/Jun/2023 03:34:40] "GET /pool2/fr-FR_Multimedia.low-latency.2022-03-15.cbf4c476.tar.pzstd HTTP/1.1" 200 -
{'modelSetInitElapsedTime': 4.065191984176636}
<2023-06-08 03:34:41,879 src/global.cc:28>  RD_INFO     RAPID recognizer 5.4.0 (C) IBM Corp. 2015-2020 (git revision 4e52e03fe57718461388d29838b4d269bbd1fb91-modified                    )
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
childDiverter::allHeadersReceived rrr timenow=2023-06-08 03:35:01.524135: rrr 127.0.0.1 b'GET' - /v1/miniHealthCheck    headers: {"Host": ["localhost:1080"], "User-Agent": ["curl/7.61.1"], "Accept": ["*/*"]}
Model initialization complete
6f530c7dec33ba7def765188694bf9370fcca89e2f170d9ed04b975b40a3c4b8
STEP 15: FROM 9648b3eb6a9f6b4563efceb5154232b19ccc66059196e414920b7e02bf77bcb1 AS release
STEP 16: COPY --from=model_cache ${CONFIG_DIR}/cache/ ${CONFIG_DIR}/cache/
Error: error dry-running "COPY --from=model_cache ${CONFIG_DIR}/cache/ ${CONFIG_DIR}/cache/": no files found matching "/var/lib/containers/storage/overlay/40fcdc4c86fcce34368711f238f7f739e23a26be5c70281abd9b6039833e1d62/merged/cache": no such file or directory

Full message:

# docker build --no-cache . -t stt-standalone
STEP 1: FROM cp.icr.io/cp/ai/watson-stt-generic-models:1.0.0 AS catalog
56f1a98978a3adde14d2e8753424a51853b02584835942d4a8d36b5ec3599aad
STEP 2: FROM cp.icr.io/cp/ai/watson-stt-en-us-multimedia:1.0.0 AS en-us-multimedia
a781c0a8e98d3f7448223c63234bf500c20967fac9ff8d1070c54e3cbe32d52f
STEP 3: FROM cp.icr.io/cp/ai/watson-stt-fr-fr-multimedia:1.0.0 AS fr-fr-multimedia
3ce6be74903c7321bc1835890692982151623837f10a165e837c3c81cd87db53
STEP 4: FROM cp.icr.io/cp/ai/watson-stt-runtime:1.0.0 AS runtime
STEP 5: ENV LOCAL_DIR=chuck_var
614f885a2e1dddfe230f07edcb00eaed40e6803b6ad5d03a25c62ccb6024c7c9
STEP 6: ENV CONFIG_DIR=/opt/ibm/chuck.x86_64/var
e4d7b9e25a47ba524abdae169bd8cc1255e994e9e1ef66660959c22c929b172e
STEP 7: COPY --chown=watson:0 --from=catalog catalog.json ${CONFIG_DIR}/catalog.json
c1f6e1c6a2ddcd176ec9af2acc0efa20e2774ff56b3e68dd56458dd29d2bd66e
STEP 8: COPY --chown=watson:0 ./${LOCAL_DIR}/* ${CONFIG_DIR}/
9648b3eb6a9f6b4563efceb5154232b19ccc66059196e414920b7e02bf77bcb1
STEP 9: FROM 9648b3eb6a9f6b4563efceb5154232b19ccc66059196e414920b7e02bf77bcb1 AS model_cache
STEP 10: RUN sudo mkdir -p /models/pool2
6e91658aba12a2410a4c46e0edda9cdb0a8af310eafe215623025b45aa8df7fc
STEP 11: COPY --chown=watson:0 --from=en-us-multimedia model/* /models/pool2/
5a36edfb869397543c23dc9f3748d8e3fad0638e001bb676a1ca1416e378604c
STEP 12: COPY --chown=watson:0 --from=fr-fr-multimedia model/* /models/pool2/
5c7a343f0b172471b55879071ee6afc1fa29a46075c25e6e85978e600180f73d
STEP 13: COPY ./prepareModels.sh .
b400d9976412ec42fcb0e21af0e7810c3620cd680eed25c8fc1afe2df3b888d2
STEP 14: RUN ./prepareModels.sh
Serving HTTP on 127.0.0.1 port 3333 (http://127.0.0.1:3333/) ...
127.0.0.1 - - [08/Jun/2023 03:34:37] "GET /pool2/en-US_Multimedia.standard.2022-03-15.4a1f1a7e.tar.pzstd HTTP/1.1" 200 -
127.0.0.1 - - [08/Jun/2023 03:34:37] "GET /pool2/en-US_Multimedia.low-latency.2022-03-15.f7dec0bc.tar.pzstd HTTP/1.1" 200 -
127.0.0.1 - - [08/Jun/2023 03:34:39] "GET /pool2/fr-FR_Multimedia.standard.2022-03-15.5023420e.tar.pzstd HTTP/1.1" 200 -
127.0.0.1 - - [08/Jun/2023 03:34:39] "GET /pool2/spkInfo_16k.2020-10-08.895a1741.tar.pzstd HTTP/1.1" 200 -
127.0.0.1 - - [08/Jun/2023 03:34:40] "GET /pool2/fr-FR_Multimedia.low-latency.2022-03-15.cbf4c476.tar.pzstd HTTP/1.1" 200 -
{'modelSetInitElapsedTime': 4.065191984176636}
<2023-06-08 03:34:41,879 src/global.cc:28>  RD_INFO     RAPID recognizer 5.4.0 (C) IBM Corp. 2015-2020 (git revision 4e52e03fe57718461388d29838b4d269bbd1fb91-modified                    )
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
childDiverter::allHeadersReceived rrr timenow=2023-06-08 03:35:01.524135: rrr 127.0.0.1 b'GET' - /v1/miniHealthCheck    headers: {"Host": ["localhost:1080"], "User-Agent": ["curl/7.61.1"], "Accept": ["*/*"]}
Model initialization complete
6f530c7dec33ba7def765188694bf9370fcca89e2f170d9ed04b975b40a3c4b8
STEP 15: FROM 9648b3eb6a9f6b4563efceb5154232b19ccc66059196e414920b7e02bf77bcb1 AS release
STEP 16: COPY --from=model_cache ${CONFIG_DIR}/cache/ ${CONFIG_DIR}/cache/
Error: error dry-running "COPY --from=model_cache ${CONFIG_DIR}/cache/ ${CONFIG_DIR}/cache/": no files found matching "/var/lib/containers/storage/overlay/40fcdc4c86fcce34368711f238f7f739e23a26be5c70281abd9b6039833e1d62/merged/cache": no such file or directory
syahrul-aiman commented 1 year ago

Similar issue with Text-to-Speech:

# docker build --no-cache . -t tts-standalone
STEP 1: FROM cp.icr.io/cp/ai/watson-tts-generic-models:1.0.0 AS catalog
550102e2cba422f40840f3fd2bc04825fc9d1779aa84f058a38cfa88d1cef5c2
STEP 2: FROM cp.icr.io/cp/ai/watson-tts-en-us-michaelv3voice:1.0.0 AS en-us-voice
1d09ce6d3a0c05690e1a34fd1967bc2a761bd4163407ace720b392c98b977027
STEP 3: FROM cp.icr.io/cp/ai/watson-tts-fr-ca-louisev3voice:1.0.0 AS fr-ca-voice
a59af99489e0cc5b495b0148623333141c5f5f9b84c4f2b23dcda7405c88674c
STEP 4: FROM cp.icr.io/cp/ai/watson-tts-runtime:1.0.0 AS runtime
STEP 5: ENV CONFIG_DIR=/opt/ibm/chuck.x86_64/var
21dcad53ecfc1d3ba666edf86463276c44142407b73fea7b68db3f75f047c2ca
STEP 6: COPY --chown=watson:0 --from=catalog catalog.json ${CONFIG_DIR}/catalog.json
76bb2f5c17693d4d3ccb3af543cb85b8a6c644d7fc866a30c316526371d4ec1d
STEP 7: COPY --chown=watson:0 ./config/* ${CONFIG_DIR}/
16d4e19754de671880514b01daf9b63baed09bf08396b0bae9373fcfefd971df
STEP 8: FROM 16d4e19754de671880514b01daf9b63baed09bf08396b0bae9373fcfefd971df AS model_cache
STEP 9: RUN sudo mkdir -p /models/pool2
a0026231770b783176fcabd9d2a0c9a57a894d331688b54970844d4cde272cb8
STEP 10: COPY --chown=watson:0 --from=en-us-voice model/* /models/pool2/
c38abad6e385ca5a4a73a68cc651132c38f949dc4a9acee366dde95d173b087e
STEP 11: COPY --chown=watson:0 --from=fr-ca-voice model/* /models/pool2/
818a20a4354e1f493128c405abd5ea432a51d50fba2fdaa2fe742fa800a800ad
STEP 12: COPY ./prepareModels.sh .
6005f43b3a0b2c2cf54e79ac115785c42d3ce65e0eecca2d307f36ef074eaeb5
STEP 13: RUN ./prepareModels.sh
Serving HTTP on 127.0.0.1 port 3333 (http://127.0.0.1:3333/) ...
127.0.0.1 - - [08/Jun/2023 03:41:39] "GET /pool2/en_us_michael_lpc_only.e245edda.tar.pzstd HTTP/1.1" 200 -
127.0.0.1 - - [08/Jun/2023 03:41:39] "GET /pool2/fr_ca_louise_lpc_only.fa40acd3.tar.pzstd HTTP/1.1" 200 -
{'modelSetInitElapsedTime': 4.557852268218994}
childDiverter::allHeadersReceived rrr timenow=2023-06-08 03:41:58.341023: rrr 127.0.0.1 b'GET' - /v1/miniHealthCheck    headers: {"Host": ["localhost:1080"], "User-Agent": ["curl/7.61.1"], "Accept": ["*/*"]}
Model initialization complete
dde3ee0218cef4b2750c5cd6387c3b72aa285e727ea50c89c02c92fd45df6b67
STEP 14: FROM 16d4e19754de671880514b01daf9b63baed09bf08396b0bae9373fcfefd971df AS release
STEP 15: COPY --from=model_cache ${CONFIG_DIR}/cache/ ${CONFIG_DIR}/cache/
Error: error dry-running "COPY --from=model_cache ${CONFIG_DIR}/cache/ ${CONFIG_DIR}/cache/": no files found matching "/var/lib/containers/storage/overlay/5cca7519d8aeaf644d5af95544ea42a997d13cf98357d74031b1e8dfd7717cd2/merged/cache": no such file or directory
kevinxhuang commented 1 year ago

@syahrul-aiman I'm running into similar issue with the docker build command on CentOS 7.9, with the latest Docker version 24.0.2. The same command works for me on Ubuntu 22.04 with Docker version 23.0.2. I think the issue might have something to do with docker build on CentOS 7. The good news is, I was able to run the container on CentOS 7.9 successfully, using the image built on Ubuntu 22.04. Hopefully that helps.

syahrul-aiman commented 1 year ago

Thanks @kevinxhuang

managed to build on ubuntu 22.04 with errors, but still got built and running:

# docker build . -t stt-standalone
Sending build context to Docker daemon  583.7kB
Step 1/16 : FROM cp.icr.io/cp/ai/watson-stt-generic-models:1.0.0 as catalog
1.0.0: Pulling from cp/ai/watson-stt-generic-models
d6e3788a121c: Pull complete 
b336a633f9e2: Pull complete 
26f8eacd231d: Pull complete 
ce4f3aa04a2f: Pull complete 
5b989cea83ad: Pull complete 
7277a345802a: Pull complete 
bb556f886d1d: Pull complete 
431e9e7a5a19: Pull complete 
8b25fb391a0e: Pull complete 
47ba7e68ec14: Pull complete 
aec9f0050026: Pull complete 
dc3a3881f91d: Pull complete 
61e45cc4391b: Pull complete 
6491669f0461: Pull complete 
7ba879fc5fa0: Pull complete 
97bb7c87c7b3: Pull complete 
9b3e0a1fc3f1: Pull complete 
Digest: sha256:4bfe7e0eba7c56458032d0a580cfa43f5cb6fd4a9db8a4407c3fa250b547b541
Status: Downloaded newer image for cp.icr.io/cp/ai/watson-stt-generic-models:1.0.0
 ---> 56f1a98978a3
Step 2/16 : FROM cp.icr.io/cp/ai/watson-stt-en-us-multimedia:1.0.0 as en-us-multimedia
1.0.0: Pulling from cp/ai/watson-stt-en-us-multimedia
d6e3788a121c: Already exists 
b336a633f9e2: Already exists 
26f8eacd231d: Already exists 
ce4f3aa04a2f: Already exists 
5b989cea83ad: Already exists 
7277a345802a: Already exists 
bb556f886d1d: Already exists 
431e9e7a5a19: Already exists 
8b25fb391a0e: Already exists 
47ba7e68ec14: Already exists 
7f43ae183a19: Pull complete 
897cc686c58a: Pull complete 
56f5f242805a: Pull complete 
59022b59def7: Pull complete 
c7199cfa141f: Pull complete 
Digest: sha256:ad0b85f018b6ccd2ebe07f6b2979840c94a3e4076368d59fd6ebcb57b2df842b
Status: Downloaded newer image for cp.icr.io/cp/ai/watson-stt-en-us-multimedia:1.0.0
 ---> a781c0a8e98d
Step 3/16 : FROM cp.icr.io/cp/ai/watson-stt-fr-fr-multimedia:1.0.0 as fr-fr-multimedia
1.0.0: Pulling from cp/ai/watson-stt-fr-fr-multimedia
d6e3788a121c: Already exists 
b336a633f9e2: Already exists 
26f8eacd231d: Already exists 
ce4f3aa04a2f: Already exists 
5b989cea83ad: Already exists 
7277a345802a: Already exists 
bb556f886d1d: Already exists 
431e9e7a5a19: Already exists 
8b25fb391a0e: Already exists 
47ba7e68ec14: Already exists 
61f1bc0e9271: Pull complete 
9baef924f2c1: Pull complete 
afdbf8b3d33f: Pull complete 
1b328057bea9: Pull complete 
12eb5f28541c: Pull complete 
Digest: sha256:9818391a631df91eda10a26120efa0c699e1c4511e2c17b511468d80be20e8f3
Status: Downloaded newer image for cp.icr.io/cp/ai/watson-stt-fr-fr-multimedia:1.0.0
 ---> 3ce6be74903c
Step 4/16 : FROM cp.icr.io/cp/ai/watson-stt-runtime:1.0.0 AS runtime
1.0.0: Pulling from cp/ai/watson-stt-runtime
d6e3788a121c: Already exists 
b336a633f9e2: Already exists 
4f32aa87151d: Pull complete 
531729cddfdb: Pull complete 
09490a1576a1: Pull complete 
3bfefad9cfcb: Pull complete 
0c96a7af71f6: Pull complete 
824524f0a64b: Pull complete 
3a00388fd886: Pull complete 
0cd644dabbbf: Pull complete 
bd5899944289: Pull complete 
463e59802bd3: Pull complete 
926e1be05286: Pull complete 
e9f3307605a7: Pull complete 
ee0f6fa3007c: Pull complete 
eeec346076c7: Pull complete 
a1f97965739e: Pull complete 
c8ca6b9ea3a7: Pull complete 
07b3703f1cc4: Pull complete 
95be69ad6013: Pull complete 
0835f4ea7461: Pull complete 
baeb14eace2f: Pull complete 
84d719dd75e8: Pull complete 
bbf088f4654b: Pull complete 
7a1a772f0ef0: Pull complete 
4acdb5a1e4b0: Pull complete 
22f2089f137b: Pull complete 
ceaedc5a8d26: Pull complete 
fa727633da4d: Pull complete 
2a06cdab217f: Pull complete 
b8f26c9b79b2: Pull complete 
43706260bb5f: Pull complete 
3e266e852867: Pull complete 
bc28d9d6faf7: Pull complete 
96f463688ca2: Pull complete 
181a6fb97abf: Pull complete 
5436fb602400: Pull complete 
a40e3b65f5cc: Pull complete 
Digest: sha256:00a8bb35458333431b2740f81c5581eae76b693911c7eb9b5b6b541fac923a09
Status: Downloaded newer image for cp.icr.io/cp/ai/watson-stt-runtime:1.0.0
 ---> dd661cca5815
Step 5/16 : ENV LOCAL_DIR=chuck_var
 ---> Running in 4bdbdb25df7b
Removing intermediate container 4bdbdb25df7b
 ---> 9482f5392c00
Step 6/16 : ENV CONFIG_DIR=/opt/ibm/chuck.x86_64/var
 ---> Running in 33a90f66dcee
Removing intermediate container 33a90f66dcee
 ---> 6ec56ea5edde
Step 7/16 : COPY --chown=watson:0 --from=catalog catalog.json ${CONFIG_DIR}/catalog.json
 ---> cf41f31c1820
Step 8/16 : COPY --chown=watson:0 ./${LOCAL_DIR}/* ${CONFIG_DIR}/
 ---> 7efe81e16767
Step 9/16 : FROM runtime as model_cache
 ---> 7efe81e16767
Step 10/16 : RUN sudo mkdir -p /models/pool2
 ---> Running in 1c511a71d70b
Removing intermediate container 1c511a71d70b
 ---> ca455e64c0d2
Step 11/16 : COPY --chown=watson:0 --from=en-us-multimedia model/* /models/pool2/
 ---> 4042051725c6
Step 12/16 : COPY --chown=watson:0 --from=fr-fr-multimedia model/* /models/pool2/
 ---> 01ea8fdc1d6a
Step 13/16 : COPY ./prepareModels.sh .
 ---> 108d09608dc8
Step 14/16 : RUN ./prepareModels.sh
 ---> Running in 91cf839127ae
127.0.0.1 - - [09/Jun/2023 00:41:13] "GET /pool2/en-US_Multimedia.low-latency.2022-03-15.f7dec0bc.tar.pzstd HTTP/1.1" 200 -
127.0.0.1 - - [09/Jun/2023 00:41:13] "GET /pool2/en-US_Multimedia.standard.2022-03-15.4a1f1a7e.tar.pzstd HTTP/1.1" 200 -
127.0.0.1 - - [09/Jun/2023 00:41:14] "GET /pool2/spkInfo_16k.2020-10-08.895a1741.tar.pzstd HTTP/1.1" 200 -
127.0.0.1 - - [09/Jun/2023 00:41:14] "GET /pool2/fr-FR_Multimedia.standard.2022-03-15.5023420e.tar.pzstd HTTP/1.1" 200 -
127.0.0.1 - - [09/Jun/2023 00:41:15] "GET /pool2/fr-FR_Multimedia.low-latency.2022-03-15.cbf4c476.tar.pzstd HTTP/1.1" 200 -
<2023-06-09 00:41:16,162 src/global.cc:28>  RD_INFO     RAPID recognizer 5.4.0 (C) IBM Corp. 2015-2020 (git revision 4e52e03fe57718461388d29838b4d269bbd1fb91-modified                    )
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
Model initialization complete
Removing intermediate container 91cf839127ae
 ---> c4e32950fbf3
Step 15/16 : FROM runtime as release
 ---> 7efe81e16767
Step 16/16 : COPY --from=model_cache ${CONFIG_DIR}/cache/ ${CONFIG_DIR}/cache/
 ---> 1161de6896d7
Successfully built 1161de6896d7
Successfully tagged stt-standalone:latest
# docker run --rm -it --env ACCEPT_LICENSE=true --publish 1080:1080 stt-standalone
{'modelSetInitElapsedTime': 1.3479135036468506}
<2023-06-09 00:43:56,654 src/global.cc:28>  RD_INFO     RAPID recognizer 5.4.0 (C) IBM Corp. 2015-2020 (git revision 4e52e03fe57718461388d29838b4d269bbd1fb91-modified                    )
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
^C<2023-06-09 00:44:50,552 src/global.cc:56>    RD_INFO     RAPID has finished: RAPID_Wall_Time[     2.198,    53.898,         1] peak memory usage 252MB
kevinxhuang commented 1 year ago

@syahrul-aiman I suppose you are referring to the error messages below. I always got those, but they seem harmless.

fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
syahrul-aiman commented 1 year ago

Hi @kevinxhuang

yeah, the error you’ve pointed out seems harmless since i can docker run on ubuntu because the build was success.

But on CentOS, the build failed on step 16.

So I’m not sure if I should close this issue, or can we conclude that the build is not supported on CentOS?

kevinxhuang commented 1 year ago

@syahrul-aiman I think it's pretty clear that build doesn't work on CentOS 7. Have you tried CentOS 8/9? If the workaround is sufficient for what you want to do now, then you can close this issue. You can always reopen it, or create a new issue if you run into further problems, say on CentOS 8/9.

syahrul-aiman commented 1 year ago

Sure, i'll proceed to close this then. And may revisit this if it is happening on CentOS 8/9. Thanks!