search

Helsinki-NLP / Opus-MT

Open neural machine translation models and web services
MIT License
592 stars 71 forks source link

Docker compile for CPU fails due to sentencepiece.pc missing #20

Open akangasr opened 4 years ago

akangasr commented 4 years ago

When compiling the dockerfile with docker build . error is raised at make install step of marian compilation:

CMake Error at src/3rd_party/sentencepiece/cmake_install.cmake:41 (file):
  file INSTALL cannot find "/usr/src/app/marian/sentencepiece.pc".
Call Stack (most recent call first):
  src/3rd_party/cmake_install.cmake:46 (include)
  src/cmake_install.cmake:42 (include)
  cmake_install.cmake:42 (include)
santhoshtr commented 4 years ago

I have created a fix for this: https://github.com/Helsinki-NLP/Opus-MT/pull/22 Can you please verify if that resolves the issue?

This was a regression with update to MarianNMT 1.9

y-bo commented 4 years ago

Had the same error. With new fix the image builds but when running:

opus-mt_1 | marian-server: error while loading shared libraries: libsentencepiece_train.so.0: cannot open shared object file: No such file or directory opus-mt_1 | ERROR:asyncio:Future exception was never retrieved opus-mt_1 | future: <Future finished exception=CalledProcessError(127, 'unknown')> opus-mt_1 | Traceback (most recent call last): opus-mt_1 | File "/usr/local/lib/python3.7/dist-packages/tornado/gen.py", line 742, in run opus-mt_1 | yielded = self.gen.throw(*exc_info) # type: ignore opus-mt_1 | File "server.py", line 49, in run opus-mt_1 | ret = yield self.p.wait_for_exit() opus-mt_1 | File "/usr/local/lib/python3.7/dist-packages/tornado/gen.py", line 735, in run opus-mt_1 | value = future.result() opus-mt_1 | subprocess.CalledProcessError: Command 'unknown' returned non-zero exit status 127.

santhoshtr commented 4 years ago

That is strange. I am sharing the output of docker run opus-mt log below. Can you find where does it differs from your result?

docker run opus-mt log

``` docker run opus-mt [2020-06-23 03:56:34] [marian] Marian v1.9.0 3c7a88f4 2020-03-10 11:34:07 -0700 [2020-06-23 03:56:34] [marian] Running on 700b5d8fff2d as process 11 with command line: [2020-06-23 03:56:34] [marian] marian-server -c ./models/en-es/decoder.yml -p 10001 --allow-unk -b 6 --mini-batch 64 --normalize 0.6 --maxi-batch-sort src --maxi-batch 100 [2020-06-23 03:56:34] [config] alignment: "" [2020-06-23 03:56:34] [config] allow-unk: true [2020-06-23 03:56:34] [config] authors: false [2020-06-23 03:56:34] [config] beam-size: 6 [2020-06-23 03:56:34] [config] bert-class-symbol: "[CLS]" [2020-06-23 03:56:34] [config] bert-mask-symbol: "[MASK]" [2020-06-23 03:56:34] [config] bert-masking-fraction: 0.15 [2020-06-23 03:56:34] [config] bert-sep-symbol: "[SEP]" [2020-06-23 03:56:34] [config] bert-train-type-embeddings: true [2020-06-23 03:56:34] [config] bert-type-vocab-size: 2 [2020-06-23 03:56:34] [config] best-deep: false [2020-06-23 03:56:34] [config] build-info: "" [2020-06-23 03:56:34] [config] cite: false [2020-06-23 03:56:34] [config] clip-gemm: 0 [2020-06-23 03:56:34] [config] cpu-threads: 1 [2020-06-23 03:56:34] [config] dec-cell: gru [2020-06-23 03:56:34] [config] dec-cell-base-depth: 2 [2020-06-23 03:56:34] [config] dec-cell-high-depth: 1 [2020-06-23 03:56:34] [config] dec-depth: 6 [2020-06-23 03:56:34] [config] devices: [2020-06-23 03:56:34] [config] - 0 [2020-06-23 03:56:34] [config] dim-emb: 512 [2020-06-23 03:56:34] [config] dim-rnn: 1024 [2020-06-23 03:56:34] [config] dim-vocabs: [2020-06-23 03:56:34] [config] - 65000 [2020-06-23 03:56:34] [config] - 65000 [2020-06-23 03:56:34] [config] dump-config: "" [2020-06-23 03:56:34] [config] enc-cell: gru [2020-06-23 03:56:34] [config] enc-cell-depth: 1 [2020-06-23 03:56:34] [config] enc-depth: 6 [2020-06-23 03:56:34] [config] enc-type: bidirectional [2020-06-23 03:56:34] [config] ignore-model-config: false [2020-06-23 03:56:34] [config] input: [2020-06-23 03:56:34] [config] - stdin [2020-06-23 03:56:34] [config] input-types: [2020-06-23 03:56:34] [config] [] [2020-06-23 03:56:34] [config] interpolate-env-vars: false [2020-06-23 03:56:34] [config] layer-normalization: false [2020-06-23 03:56:34] [config] lemma-dim-emb: 0 [2020-06-23 03:56:34] [config] log: "" [2020-06-23 03:56:34] [config] log-level: info [2020-06-23 03:56:34] [config] log-time-zone: "" [2020-06-23 03:56:34] [config] max-length: 1000 [2020-06-23 03:56:34] [config] max-length-crop: false [2020-06-23 03:56:34] [config] max-length-factor: 3 [2020-06-23 03:56:34] [config] maxi-batch: 100 [2020-06-23 03:56:34] [config] maxi-batch-sort: src [2020-06-23 03:56:34] [config] mini-batch: 64 [2020-06-23 03:56:34] [config] mini-batch-words: 0 [2020-06-23 03:56:34] [config] models: [2020-06-23 03:56:34] [config] - /usr/src/app/models/en-es/opus.bpe32k-bpe32k.transformer.model1.npz.best-perplexity.npz [2020-06-23 03:56:34] [config] n-best: false [2020-06-23 03:56:34] [config] no-spm-decode: false [2020-06-23 03:56:34] [config] normalize: 0.6 [2020-06-23 03:56:34] [config] num-devices: 0 [2020-06-23 03:56:34] [config] optimize: false [2020-06-23 03:56:34] [config] output: stdout [2020-06-23 03:56:34] [config] output-sampling: false [2020-06-23 03:56:34] [config] port: 10001 [2020-06-23 03:56:34] [config] precision: [2020-06-23 03:56:34] [config] - float32 [2020-06-23 03:56:34] [config] quiet: false [2020-06-23 03:56:34] [config] quiet-translation: false [2020-06-23 03:56:34] [config] relative-paths: false [2020-06-23 03:56:34] [config] right-left: false [2020-06-23 03:56:34] [config] seed: 0 [2020-06-23 03:56:34] [config] shortlist: [2020-06-23 03:56:34] [config] [] [2020-06-23 03:56:34] [config] skip: false [2020-06-23 03:56:34] [config] skip-cost: false [2020-06-23 03:56:34] [config] tied-embeddings: false [2020-06-23 03:56:34] [config] tied-embeddings-all: true [2020-06-23 03:56:34] [config] tied-embeddings-src: false [2020-06-23 03:56:34] [config] transformer-aan-activation: swish [2020-06-23 03:56:34] [config] transformer-aan-depth: 2 [2020-06-23 03:56:34] [config] transformer-aan-nogate: false [2020-06-23 03:56:34] [config] transformer-decoder-autoreg: self-attention [2020-06-23 03:56:34] [config] transformer-depth-scaling: false [2020-06-23 03:56:34] [config] transformer-dim-aan: 2048 [2020-06-23 03:56:34] [config] transformer-dim-ffn: 2048 [2020-06-23 03:56:34] [config] transformer-ffn-activation: swish [2020-06-23 03:56:34] [config] transformer-ffn-depth: 2 [2020-06-23 03:56:34] [config] transformer-guided-alignment-layer: last [2020-06-23 03:56:34] [config] transformer-heads: 8 [2020-06-23 03:56:34] [config] transformer-no-projection: false [2020-06-23 03:56:34] [config] transformer-postprocess: dan [2020-06-23 03:56:34] [config] transformer-postprocess-emb: d [2020-06-23 03:56:34] [config] transformer-preprocess: "" [2020-06-23 03:56:34] [config] transformer-tied-layers: [2020-06-23 03:56:34] [config] [] [2020-06-23 03:56:34] [config] transformer-train-position-embeddings: false [2020-06-23 03:56:34] [config] type: transformer [2020-06-23 03:56:34] [config] version: v1.7.6 1d4ba73 2019-05-11 17:16:31 +0100 [2020-06-23 03:56:34] [config] vocabs: [2020-06-23 03:56:34] [config] - /usr/src/app/models/en-es/opus.bpe32k-bpe32k.vocab.yml [2020-06-23 03:56:34] [config] - /usr/src/app/models/en-es/opus.bpe32k-bpe32k.vocab.yml [2020-06-23 03:56:34] [config] weights: [2020-06-23 03:56:34] [config] [] [2020-06-23 03:56:34] [config] word-penalty: 0 [2020-06-23 03:56:34] [config] word-scores: false [2020-06-23 03:56:34] [config] workspace: 512 [2020-06-23 03:56:34] [config] Loaded model has been created with Marian v1.7.6 1d4ba73 2019-05-11 17:16:31 +0100 [2020-06-23 03:56:34] [data] Loading vocabulary from JSON/Yaml file /usr/src/app/models/en-es/opus.bpe32k-bpe32k.vocab.yml [2020-06-23 03:56:35] [marian] Marian v1.9.0 3c7a88f4 2020-03-10 11:34:07 -0700 [2020-06-23 03:56:35] [marian] Running on 700b5d8fff2d as process 14 with command line: [2020-06-23 03:56:35] [marian] marian-server -c ./models/en-fi/decoder.yml -p 10002 --allow-unk -b 6 --mini-batch 64 --normalize 0.6 --maxi-batch-sort src --maxi-batch 100 [2020-06-23 03:56:35] [config] alignment: "" [2020-06-23 03:56:35] [config] allow-unk: true [2020-06-23 03:56:35] [config] authors: false [2020-06-23 03:56:35] [config] beam-size: 6 [2020-06-23 03:56:35] [config] bert-class-symbol: "[CLS]" [2020-06-23 03:56:35] [config] bert-mask-symbol: "[MASK]" [2020-06-23 03:56:35] [config] bert-masking-fraction: 0.15 [2020-06-23 03:56:35] [config] bert-sep-symbol: "[SEP]" [2020-06-23 03:56:35] [config] bert-train-type-embeddings: true [2020-06-23 03:56:35] [config] bert-type-vocab-size: 2 [2020-06-23 03:56:35] [config] best-deep: false [2020-06-23 03:56:35] [config] build-info: "" [2020-06-23 03:56:35] [config] cite: false [2020-06-23 03:56:35] [config] clip-gemm: 0 [2020-06-23 03:56:35] [config] cpu-threads: 1 [2020-06-23 03:56:35] [config] dec-cell: gru [2020-06-23 03:56:35] [config] dec-cell-base-depth: 2 [2020-06-23 03:56:35] [config] dec-cell-high-depth: 1 [2020-06-23 03:56:35] [config] dec-depth: 6 [2020-06-23 03:56:35] [config] devices: [2020-06-23 03:56:35] [config] - 0 [2020-06-23 03:56:35] [config] dim-emb: 512 [2020-06-23 03:56:35] [config] dim-rnn: 1024 [2020-06-23 03:56:35] [config] dim-vocabs: [2020-06-23 03:56:35] [config] - 61547 [2020-06-23 03:56:35] [config] - 61547 [2020-06-23 03:56:35] [config] dump-config: "" [2020-06-23 03:56:35] [config] enc-cell: gru [2020-06-23 03:56:35] [config] enc-cell-depth: 1 [2020-06-23 03:56:35] [config] enc-depth: 6 [2020-06-23 03:56:35] [config] enc-type: bidirectional [2020-06-23 03:56:35] [config] ignore-model-config: false [2020-06-23 03:56:35] [config] input: [2020-06-23 03:56:35] [config] - stdin [2020-06-23 03:56:35] [config] input-types: [2020-06-23 03:56:35] [config] [] [2020-06-23 03:56:35] [config] interpolate-env-vars: false [2020-06-23 03:56:35] [config] layer-normalization: false [2020-06-23 03:56:35] [config] lemma-dim-emb: 0 [2020-06-23 03:56:35] [config] log: "" [2020-06-23 03:56:35] [config] log-level: info [2020-06-23 03:56:35] [config] log-time-zone: "" [2020-06-23 03:56:35] [config] max-length: 1000 [2020-06-23 03:56:35] [config] max-length-crop: false [2020-06-23 03:56:35] [config] max-length-factor: 3 [2020-06-23 03:56:35] [config] maxi-batch: 100 [2020-06-23 03:56:35] [config] maxi-batch-sort: src [2020-06-23 03:56:35] [config] mini-batch: 64 [2020-06-23 03:56:35] [config] mini-batch-words: 0 [2020-06-23 03:56:35] [config] models: [2020-06-23 03:56:35] [config] - /usr/src/app/models/en-fi/opus.spm32k-spm32k.transformer-align.model1.npz.best-perplexity.npz [2020-06-23 03:56:35] [config] n-best: false [2020-06-23 03:56:35] [config] no-spm-decode: false [2020-06-23 03:56:35] [config] normalize: 0.6 [2020-06-23 03:56:35] [config] num-devices: 0 [2020-06-23 03:56:35] [config] optimize: false [2020-06-23 03:56:35] [config] output: stdout [2020-06-23 03:56:35] [config] output-sampling: false [2020-06-23 03:56:35] [config] port: 10002 [2020-06-23 03:56:35] [config] precision: [2020-06-23 03:56:35] [config] - float32 [2020-06-23 03:56:35] [config] quiet: false [2020-06-23 03:56:35] [config] quiet-translation: false [2020-06-23 03:56:35] [config] relative-paths: false [2020-06-23 03:56:35] [config] right-left: false [2020-06-23 03:56:35] [config] seed: 0 [2020-06-23 03:56:35] [config] shortlist: [2020-06-23 03:56:35] [config] [] [2020-06-23 03:56:35] [config] skip: false [2020-06-23 03:56:35] [config] skip-cost: false [2020-06-23 03:56:35] [config] tied-embeddings: false [2020-06-23 03:56:35] [config] tied-embeddings-all: true [2020-06-23 03:56:35] [config] tied-embeddings-src: false [2020-06-23 03:56:35] [config] transformer-aan-activation: swish [2020-06-23 03:56:35] [config] transformer-aan-depth: 2 [2020-06-23 03:56:35] [config] transformer-aan-nogate: false [2020-06-23 03:56:35] [config] transformer-decoder-autoreg: self-attention [2020-06-23 03:56:35] [config] transformer-depth-scaling: false [2020-06-23 03:56:35] [config] transformer-dim-aan: 2048 [2020-06-23 03:56:35] [config] transformer-dim-ffn: 2048 [2020-06-23 03:56:35] [config] transformer-ffn-activation: swish [2020-06-23 03:56:35] [config] transformer-ffn-depth: 2 [2020-06-23 03:56:35] [config] transformer-guided-alignment-layer: last [2020-06-23 03:56:35] [config] transformer-heads: 8 [2020-06-23 03:56:35] [config] transformer-no-projection: false [2020-06-23 03:56:35] [config] transformer-postprocess: dan [2020-06-23 03:56:35] [config] transformer-postprocess-emb: d [2020-06-23 03:56:35] [config] transformer-preprocess: "" [2020-06-23 03:56:35] [config] transformer-tied-layers: [2020-06-23 03:56:35] [config] [] [2020-06-23 03:56:35] [config] transformer-train-position-embeddings: false [2020-06-23 03:56:35] [config] type: transformer [2020-06-23 03:56:35] [config] ulr: false [2020-06-23 03:56:35] [config] ulr-dim-emb: 0 [2020-06-23 03:56:35] [config] ulr-trainable-transformation: false [2020-06-23 03:56:35] [config] version: v1.8.2 2111c28 2019-10-16 08:36:48 -0700 [2020-06-23 03:56:35] [config] vocabs: [2020-06-23 03:56:35] [config] - /usr/src/app/models/en-fi/opus.spm32k-spm32k.vocab.yml [2020-06-23 03:56:35] [config] - /usr/src/app/models/en-fi/opus.spm32k-spm32k.vocab.yml [2020-06-23 03:56:35] [config] weights: [2020-06-23 03:56:35] [config] [] [2020-06-23 03:56:35] [config] word-penalty: 0 [2020-06-23 03:56:35] [config] word-scores: false [2020-06-23 03:56:35] [config] workspace: 512 [2020-06-23 03:56:35] [config] Loaded model has been created with Marian v1.8.2 2111c28 2019-10-16 08:36:48 -0700 [2020-06-23 03:56:35] [data] Loading vocabulary from JSON/Yaml file /usr/src/app/models/en-fi/opus.spm32k-spm32k.vocab.yml [2020-06-23 03:56:35] [marian] Marian v1.9.0 3c7a88f4 2020-03-10 11:34:07 -0700 [2020-06-23 03:56:35] [marian] Running on 700b5d8fff2d as process 17 with command line: [2020-06-23 03:56:35] [marian] marian-server -c ./models/en-ml/decoder.yml -p 10003 --allow-unk -b 6 --mini-batch 64 --normalize 0.6 --maxi-batch-sort src --maxi-batch 100 [2020-06-23 03:56:35] [config] alignment: "" [2020-06-23 03:56:35] [config] allow-unk: true [2020-06-23 03:56:35] [config] authors: false [2020-06-23 03:56:35] [config] beam-size: 6 [2020-06-23 03:56:35] [config] bert-class-symbol: "[CLS]" [2020-06-23 03:56:35] [config] bert-mask-symbol: "[MASK]" [2020-06-23 03:56:35] [config] bert-masking-fraction: 0.15 [2020-06-23 03:56:35] [config] bert-sep-symbol: "[SEP]" [2020-06-23 03:56:35] [config] bert-train-type-embeddings: true [2020-06-23 03:56:35] [config] bert-type-vocab-size: 2 [2020-06-23 03:56:35] [config] best-deep: false [2020-06-23 03:56:35] [config] build-info: "" [2020-06-23 03:56:35] [config] cite: false [2020-06-23 03:56:35] [config] clip-gemm: 0 [2020-06-23 03:56:35] [config] cpu-threads: 1 [2020-06-23 03:56:35] [config] dec-cell: gru [2020-06-23 03:56:35] [config] dec-cell-base-depth: 2 [2020-06-23 03:56:35] [config] dec-cell-high-depth: 1 [2020-06-23 03:56:35] [config] dec-depth: 6 [2020-06-23 03:56:35] [config] devices: [2020-06-23 03:56:35] [config] - 0 [2020-06-23 03:56:35] [config] dim-emb: 512 [2020-06-23 03:56:35] [config] dim-rnn: 1024 [2020-06-23 03:56:35] [config] dim-vocabs: [2020-06-23 03:56:35] [config] - 63223 [2020-06-23 03:56:35] [config] - 63223 [2020-06-23 03:56:35] [config] dump-config: "" [2020-06-23 03:56:35] [config] enc-cell: gru [2020-06-23 03:56:35] [config] enc-cell-depth: 1 [2020-06-23 03:56:35] [config] enc-depth: 6 [2020-06-23 03:56:35] [config] enc-type: bidirectional [2020-06-23 03:56:35] [config] ignore-model-config: false [2020-06-23 03:56:35] [config] input: [2020-06-23 03:56:35] [config] - stdin [2020-06-23 03:56:35] [config] input-types: [2020-06-23 03:56:35] [config] [] [2020-06-23 03:56:35] [config] interpolate-env-vars: false [2020-06-23 03:56:35] [config] layer-normalization: false [2020-06-23 03:56:35] [config] lemma-dim-emb: 0 [2020-06-23 03:56:35] [config] log: "" [2020-06-23 03:56:35] [config] log-level: info [2020-06-23 03:56:35] [config] log-time-zone: "" [2020-06-23 03:56:35] [config] max-length: 1000 [2020-06-23 03:56:35] [config] max-length-crop: false [2020-06-23 03:56:35] [config] max-length-factor: 3 [2020-06-23 03:56:35] [config] maxi-batch: 100 [2020-06-23 03:56:35] [config] maxi-batch-sort: src [2020-06-23 03:56:35] [config] mini-batch: 64 [2020-06-23 03:56:35] [config] mini-batch-words: 0 [2020-06-23 03:56:35] [config] models: [2020-06-23 03:56:35] [config] - /usr/src/app/models/en-ml/opus+bt.spm32k-spm32k.transformer-align.model1.npz.best-perplexity.npz [2020-06-23 03:56:35] [config] n-best: false [2020-06-23 03:56:35] [config] no-spm-decode: false [2020-06-23 03:56:35] [config] normalize: 0.6 [2020-06-23 03:56:35] [config] num-devices: 0 [2020-06-23 03:56:35] [config] optimize: false [2020-06-23 03:56:35] [config] output: stdout [2020-06-23 03:56:35] [config] output-sampling: false [2020-06-23 03:56:35] [config] port: 10003 [2020-06-23 03:56:35] [config] precision: [2020-06-23 03:56:35] [config] - float32 [2020-06-23 03:56:35] [config] quiet: false [2020-06-23 03:56:35] [config] quiet-translation: false [2020-06-23 03:56:35] [config] relative-paths: false [2020-06-23 03:56:35] [config] right-left: false [2020-06-23 03:56:35] [config] seed: 0 [2020-06-23 03:56:35] [config] shortlist: [2020-06-23 03:56:35] [config] [] [2020-06-23 03:56:35] [config] skip: false [2020-06-23 03:56:35] [config] skip-cost: false [2020-06-23 03:56:35] [config] tied-embeddings: false [2020-06-23 03:56:35] [config] tied-embeddings-all: true [2020-06-23 03:56:35] [config] tied-embeddings-src: false [2020-06-23 03:56:35] [config] transformer-aan-activation: swish [2020-06-23 03:56:35] [config] transformer-aan-depth: 2 [2020-06-23 03:56:35] [config] transformer-aan-nogate: false [2020-06-23 03:56:35] [config] transformer-decoder-autoreg: self-attention [2020-06-23 03:56:35] [config] transformer-depth-scaling: false [2020-06-23 03:56:35] [config] transformer-dim-aan: 2048 [2020-06-23 03:56:35] [config] transformer-dim-ffn: 2048 [2020-06-23 03:56:35] [config] transformer-ffn-activation: swish [2020-06-23 03:56:35] [config] transformer-ffn-depth: 2 [2020-06-23 03:56:35] [config] transformer-guided-alignment-layer: last [2020-06-23 03:56:35] [config] transformer-heads: 8 [2020-06-23 03:56:35] [config] transformer-no-projection: false [2020-06-23 03:56:35] [config] transformer-postprocess: dan [2020-06-23 03:56:35] [config] transformer-postprocess-emb: d [2020-06-23 03:56:35] [config] transformer-preprocess: "" [2020-06-23 03:56:35] [config] transformer-tied-layers: [2020-06-23 03:56:35] [config] [] [2020-06-23 03:56:35] [config] transformer-train-position-embeddings: false [2020-06-23 03:56:35] [config] type: transformer [2020-06-23 03:56:35] [config] ulr: false [2020-06-23 03:56:35] [config] ulr-dim-emb: 0 [2020-06-23 03:56:35] [config] ulr-trainable-transformation: false [2020-06-23 03:56:35] [config] version: v1.8.2 2111c28 2019-10-16 08:36:48 -0700 [2020-06-23 03:56:35] [config] vocabs: [2020-06-23 03:56:35] [config] - /usr/src/app/models/en-ml/opus+bt.spm32k-spm32k.vocab.yml [2020-06-23 03:56:35] [config] - /usr/src/app/models/en-ml/opus+bt.spm32k-spm32k.vocab.yml [2020-06-23 03:56:35] [config] weights: [2020-06-23 03:56:35] [config] [] [2020-06-23 03:56:35] [config] word-penalty: 0 [2020-06-23 03:56:35] [config] word-scores: false [2020-06-23 03:56:35] [config] workspace: 512 [2020-06-23 03:56:35] [config] Loaded model has been created with Marian v1.8.2 2111c28 2019-10-16 08:36:48 -0700 [2020-06-23 03:56:35] [data] Loading vocabulary from JSON/Yaml file /usr/src/app/models/en-ml/opus+bt.spm32k-spm32k.vocab.yml [2020-06-23 03:56:35] [marian] Marian v1.9.0 3c7a88f4 2020-03-10 11:34:07 -0700 [2020-06-23 03:56:35] [marian] Running on 700b5d8fff2d as process 20 with command line: [2020-06-23 03:56:35] [marian] marian-server -c ./models/en-mr/decoder.yml -p 10004 --allow-unk -b 6 --mini-batch 64 --normalize 0.6 --maxi-batch-sort src --maxi-batch 100 [2020-06-23 03:56:35] [config] alignment: "" [2020-06-23 03:56:35] [config] allow-unk: true [2020-06-23 03:56:35] [config] authors: false [2020-06-23 03:56:35] [config] beam-size: 6 [2020-06-23 03:56:35] [config] bert-class-symbol: "[CLS]" [2020-06-23 03:56:35] [config] bert-mask-symbol: "[MASK]" [2020-06-23 03:56:35] [config] bert-masking-fraction: 0.15 [2020-06-23 03:56:35] [config] bert-sep-symbol: "[SEP]" [2020-06-23 03:56:35] [config] bert-train-type-embeddings: true [2020-06-23 03:56:35] [config] bert-type-vocab-size: 2 [2020-06-23 03:56:35] [config] best-deep: false [2020-06-23 03:56:35] [config] build-info: "" [2020-06-23 03:56:35] [config] cite: false [2020-06-23 03:56:35] [config] clip-gemm: 0 [2020-06-23 03:56:35] [config] cpu-threads: 1 [2020-06-23 03:56:35] [config] dec-cell: gru [2020-06-23 03:56:35] [config] dec-cell-base-depth: 2 [2020-06-23 03:56:35] [config] dec-cell-high-depth: 1 [2020-06-23 03:56:35] [config] dec-depth: 6 [2020-06-23 03:56:35] [config] devices: [2020-06-23 03:56:35] [config] - 0 [2020-06-23 03:56:35] [config] dim-emb: 512 [2020-06-23 03:56:35] [config] dim-rnn: 1024 [2020-06-23 03:56:35] [config] dim-vocabs: [2020-06-23 03:56:35] [config] - 61673 [2020-06-23 03:56:35] [config] - 61673 [2020-06-23 03:56:35] [config] dump-config: "" [2020-06-23 03:56:35] [config] enc-cell: gru [2020-06-23 03:56:35] [config] enc-cell-depth: 1 [2020-06-23 03:56:35] [config] enc-depth: 6 [2020-06-23 03:56:35] [config] enc-type: bidirectional [2020-06-23 03:56:35] [config] ignore-model-config: false [2020-06-23 03:56:35] [config] input: [2020-06-23 03:56:35] [config] - stdin [2020-06-23 03:56:35] [config] input-types: [2020-06-23 03:56:35] [config] [] [2020-06-23 03:56:35] [config] interpolate-env-vars: false [2020-06-23 03:56:35] [config] layer-normalization: false [2020-06-23 03:56:35] [config] lemma-dim-emb: 0 [2020-06-23 03:56:35] [config] log: "" [2020-06-23 03:56:35] [config] log-level: info [2020-06-23 03:56:35] [config] log-time-zone: "" [2020-06-23 03:56:35] [config] max-length: 1000 [2020-06-23 03:56:35] [config] max-length-crop: false [2020-06-23 03:56:35] [config] max-length-factor: 3 [2020-06-23 03:56:35] [config] maxi-batch: 100 [2020-06-23 03:56:35] [config] maxi-batch-sort: src [2020-06-23 03:56:35] [config] mini-batch: 64 [2020-06-23 03:56:35] [config] mini-batch-words: 0 [2020-06-23 03:56:35] [config] models: [2020-06-23 03:56:35] [config] - /usr/src/app/models/en-mr/opus.spm32k-spm32k.transformer-align.model1.npz.best-perplexity.npz [2020-06-23 03:56:35] [config] n-best: false [2020-06-23 03:56:35] [config] no-spm-decode: false [2020-06-23 03:56:35] [config] normalize: 0.6 [2020-06-23 03:56:35] [config] num-devices: 0 [2020-06-23 03:56:35] [config] optimize: false [2020-06-23 03:56:35] [config] output: stdout [2020-06-23 03:56:35] [config] output-sampling: false [2020-06-23 03:56:35] [config] port: 10004 [2020-06-23 03:56:35] [config] precision: [2020-06-23 03:56:35] [config] - float32 [2020-06-23 03:56:35] [config] quiet: false [2020-06-23 03:56:35] [config] quiet-translation: false [2020-06-23 03:56:35] [config] relative-paths: false [2020-06-23 03:56:35] [config] right-left: false [2020-06-23 03:56:35] [config] seed: 0 [2020-06-23 03:56:35] [config] shortlist: [2020-06-23 03:56:35] [config] [] [2020-06-23 03:56:35] [config] skip: false [2020-06-23 03:56:35] [config] skip-cost: false [2020-06-23 03:56:35] [config] tied-embeddings: false [2020-06-23 03:56:35] [config] tied-embeddings-all: true [2020-06-23 03:56:35] [config] tied-embeddings-src: false [2020-06-23 03:56:35] [config] transformer-aan-activation: swish [2020-06-23 03:56:35] [config] transformer-aan-depth: 2 [2020-06-23 03:56:35] [config] transformer-aan-nogate: false [2020-06-23 03:56:35] [config] transformer-decoder-autoreg: self-attention [2020-06-23 03:56:35] [config] transformer-depth-scaling: false [2020-06-23 03:56:35] [config] transformer-dim-aan: 2048 [2020-06-23 03:56:35] [config] transformer-dim-ffn: 2048 [2020-06-23 03:56:35] [config] transformer-ffn-activation: swish [2020-06-23 03:56:35] [config] transformer-ffn-depth: 2 [2020-06-23 03:56:35] [config] transformer-guided-alignment-layer: last [2020-06-23 03:56:35] [config] transformer-heads: 8 [2020-06-23 03:56:35] [config] transformer-no-projection: false [2020-06-23 03:56:35] [config] transformer-postprocess: dan [2020-06-23 03:56:35] [config] transformer-postprocess-emb: d [2020-06-23 03:56:35] [config] transformer-preprocess: "" [2020-06-23 03:56:35] [config] transformer-tied-layers: [2020-06-23 03:56:35] [config] [] [2020-06-23 03:56:35] [config] transformer-train-position-embeddings: false [2020-06-23 03:56:35] [config] type: transformer [2020-06-23 03:56:35] [config] ulr: false [2020-06-23 03:56:35] [config] ulr-dim-emb: 0 [2020-06-23 03:56:35] [config] ulr-trainable-transformation: false [2020-06-23 03:56:35] [config] version: v1.8.2 2111c28 2019-10-16 08:36:48 -0700 [2020-06-23 03:56:35] [config] vocabs: [2020-06-23 03:56:35] [config] - /usr/src/app/models/en-mr/opus.spm32k-spm32k.vocab.yml [2020-06-23 03:56:35] [config] - /usr/src/app/models/en-mr/opus.spm32k-spm32k.vocab.yml [2020-06-23 03:56:35] [config] weights: [2020-06-23 03:56:35] [config] [] [2020-06-23 03:56:35] [config] word-penalty: 0 [2020-06-23 03:56:35] [config] word-scores: false [2020-06-23 03:56:35] [config] workspace: 512 [2020-06-23 03:56:35] [config] Loaded model has been created with Marian v1.8.2 2111c28 2019-10-16 08:36:48 -0700 [2020-06-23 03:56:35] [data] Loading vocabulary from JSON/Yaml file /usr/src/app/models/en-mr/opus.spm32k-spm32k.vocab.yml [2020-06-23 03:56:35] [data] Loading vocabulary from JSON/Yaml file /usr/src/app/models/en-es/opus.bpe32k-bpe32k.vocab.yml [2020-06-23 03:56:36] [data] Loading vocabulary from JSON/Yaml file /usr/src/app/models/en-fi/opus.spm32k-spm32k.vocab.yml [2020-06-23 03:56:36] [data] Loading vocabulary from JSON/Yaml file /usr/src/app/models/en-ml/opus+bt.spm32k-spm32k.vocab.yml [2020-06-23 03:56:36] [memory] Extending reserved space to 512 MB (device cpu0) [2020-06-23 03:56:36] Loading scorer of type transformer as feature F0 [2020-06-23 03:56:36] Loading model from /usr/src/app/models/en-es/opus.bpe32k-bpe32k.transformer.model1.npz.best-perplexity.npz [2020-06-23 03:56:37] [data] Loading vocabulary from JSON/Yaml file /usr/src/app/models/en-mr/opus.spm32k-spm32k.vocab.yml [2020-06-23 03:56:37] [memory] Extending reserved space to 512 MB (device cpu0) [2020-06-23 03:56:37] Loading scorer of type transformer as feature F0 [2020-06-23 03:56:37] Loading model from /usr/src/app/models/en-fi/opus.spm32k-spm32k.transformer-align.model1.npz.best-perplexity.npz [2020-06-23 03:56:38] [memory] Extending reserved space to 512 MB (device cpu0) [2020-06-23 03:56:38] Loading scorer of type transformer as feature F0 [2020-06-23 03:56:38] Loading model from /usr/src/app/models/en-ml/opus+bt.spm32k-spm32k.transformer-align.model1.npz.best-perplexity.npz [2020-06-23 03:56:38] [memory] Extending reserved space to 512 MB (device cpu0) [2020-06-23 03:56:38] Loading scorer of type transformer as feature F0 [2020-06-23 03:56:38] Loading model from /usr/src/app/models/en-mr/opus.spm32k-spm32k.transformer-align.model1.npz.best-perplexity.npz [2020-06-23 03:56:39] Server is listening on port 10001 [2020-06-23 03:56:39] Server is listening on port 10002 [2020-06-23 03:56:40] Server is listening on port 10003 [2020-06-23 03:56:40] Server is listening on port 10004 ```

y-bo commented 4 years ago

Sorry, my comment was misleading. The docker run works fine. docker-compose up does not (my log is from compose)

santhoshtr commented 4 years ago

Sorry, my comment was misleading. The docker run works fine. docker-compose up does not (my log is from compose)

Perhaps you need to check what is the problem with docker-compose script? Anyway, it is independent of the issue being discussed here, right? Let me know if it is not.

xirect commented 4 years ago

Any updates on progress perhaps?

santhoshtr commented 4 years ago

Any updates on progress perhaps?

https://github.com/Helsinki-NLP/Opus-MT/pull/22 has the fix for this and I posted the log of that working fine. In case that is not helpful, please let us know what exact issue you see.

xirect commented 4 years ago

I had a lot of other issues which are now fixed.

jotline commented 3 years ago

I had a lot of other issues which are now fixed.

https://hub.docker.com/r/veer66/opus-mt-th-en This third-party docker image works fine for me，have a try？