marian-nmt / marian-examples

Examples, tutorials and use cases for Marian, including our WMT-2017/18 baselines.
Other
78 stars 34 forks source link

Error: There are option(s) in a config file that are not expected #17

Closed Supachan closed 5 years ago

Supachan commented 5 years ago

I got the weird problem. Before it happened, I ran the command /.run-me.sh with -w 1024 (by default:-w 3000. Of course, it would take very long time to finish translation between English-Romanian with marian-examples.

I decided to stop running, added --max-length 30, and ran the command again. The error is:

#!/bin/bash -v

MARIAN=../../build

# if we are in WSL, we need to add '.exe' to the tool names
if [ -e "/bin/wslpath" ]
then
    EXT=.exe
fi

MARIAN_TRAIN=$MARIAN/marian$EXT
MARIAN_DECODER=$MARIAN/marian-decoder$EXT
MARIAN_VOCAB=$MARIAN/marian-vocab$EXT
MARIAN_SCORER=$MARIAN/marian-scorer$EXT

# set chosen gpus
GPUS=0
if [ $# -ne 0 ]
then
    GPUS=$@
fi
echo Using GPUs: $GPUS
Using GPUs: 0

if [ ! -e $MARIAN_TRAIN ]
then
    echo "marian is not installed in $MARIAN, you need to compile the toolkit first"
    exit 1
fi

if [ ! -e ../tools/moses-scripts ] || [ ! -e ../tools/subword-nmt ]
then
    echo "missing tools in ../tools, you need to download them first"
    exit 1
fi

if [ ! -e "data/corpus.en" ]
then
    ./scripts/download-files.sh
fi

mkdir -p model

# preprocess data
if [ ! -e "data/corpus.bpe.en" ]
then
    ./scripts/preprocess-data.sh
fi

# train model
if [ ! -e "model/model.npz.best-translation.npz" ]
then
    $MARIAN_TRAIN \
        --devices $GPUS \
        --type amun \
        --model model/model.npz \
        --train-sets data/corpus.bpe.ro data/corpus.bpe.en \
        --vocabs model/vocab.ro.yml model/vocab.en.yml \
        --dim-vocabs 66000 50000 \
        --mini-batch-fit -w 1024 \
        --max-length 50
        --layer-normalization --dropout-rnn 0.2 --dropout-src 0.1 --dropout-trg 0.1 \
        --early-stopping 5 \
        --valid-freq 10000 --save-freq 10000 --disp-freq 1000 \
        --valid-metrics cross-entropy translation \
        --valid-sets data/newsdev2016.bpe.ro data/newsdev2016.bpe.en \
        --valid-script-path "bash ./scripts/validate.sh" \
        --log model/train.log --valid-log model/valid.log \
        --overwrite --keep-best \
        --seed 1111 --exponential-smoothing \
        --normalize=1 --beam-size=12 --quiet-translation
fi
[2019-05-17 09:17:00] Error: There are option(s) in a config file that are not expected
[2019-05-17 09:17:00] Error: Aborted from void marian::ConfigParser::parseOptions(int, char**, bool) in /tmp/marian/src/common/config_parser.cpp:707

[CALL STACK]
[0x6d134e]          marian::ConfigParser::  parseOptions  (int,  char**,  bool) + 0x5ae
[0x6aa64f]          marian::Config::  initialize  (int,  char**,  marian::cli::mode,  bool) + 0x7f
[0x6acd5d]          marian::Config::  Config  (int,  char**,  marian::cli::mode,  bool) + 0x2d
[0x6acdcd]          marian::  parseOptions  (int,  char**,  marian::cli::mode,  bool) + 0x3d
[0x59c722]          mainTrainer  (int,  char**)                        + 0x42
[0x57afba]          main                                               + 0x8a
[0x7f7ec316d830]    __libc_start_main                                  + 0xf0
[0x59a219]          _start                                             + 0x29

How can I fix it? Please let me know.

Thanks in advance. Supachan

snukky commented 5 years ago

Please follow this advice: https://github.com/marian-nmt/marian/issues/271 or just remove model.npz.yml.