Error when try to run supervised/semi supervised experiment (mac version)

[wandakinasih]

So i have download moses for mac, but it seems that there are some incomplete dependency to xmlrpc-c

Python 3.7.7

[haryoa]

Hello,

This is a duplicate from issue #6 ( The OP also mentioned it)

We didn't test our source code on mac. The error might be caused by the compiled binary file provided by the author of the Moses Toolkit.

Sorry for the inconvenience

[Picasso1503]

I have the same problem when I runned it from WSL2 (Windows Subsystem for Linux 2)

[haryoa]

Hello @Picasso1503,

Could you run this script on your stif-indonesia folder and paste the error here?

nice moses/moses/scripts/training/train-model.perl -root-dir output/supervised/train -corpus data/labelled/train -f inf -e for -alignment grow-diag-final-and                         -reordering msd-bidirectional-fe                         -lm 0:3:[ABSOLUTE_PATH_TO_STIF_INDONESIA]/stif-indonesia/output/supervised/lm/blm/data.blm.for -core 2 -external-bin-dir moses/training-tools --mgiza

Change the [ABSOLUTE_PATH_TO_STIF_INDONESIA] with the absolute path of stif-indonesia folder (use pwd).

The error on MAC and WSL2 is different. The error in MAC may be caused by xmlrpc but in WSL2, the error is different (you can run KEN-LM).

[Picasso1503]

and what do you mean about change the path to absolute path? I dont understand yet

[haryoa]

Absolute path means the path file from your root directory to your file or directory in your OS. For example: /home/muhammadrizky15/stif-indonesia/

use pwd command in your WSL2 terminal to see the absolute path of the folder stif-indonesia

I cannot see your Capture.PNG. Would you mind to edit it? Thank you.

[Picasso1503]

This what I've got

[Picasso1503]

[haryoa]

change [ABSOLUTE_PATH_TO_STIF_INDONESIA]/stif-indonesia/output/supervised/lm/blm/data.blm.for to /home/muhammadrizky15/stif-indonesia/stif-indonesia/output/supervised/lm/blm/data.blm.for in your -lm arguments. That /mnt/d/ path is for my local computer...

[Picasso1503]

change [ABSOLUTE_PATH_TO_STIF_INDONESIA]/stif-indonesia/output/supervised/lm/blm/data.blm.for to /home/muhammadrizky15/stif-indonesia/stif-indonesia/output/supervised/lm/blm/data.blm.for in your -lm arguments. That /mnt/d/ path is for my local computer...

I don't understand, can u explain about the command line, bcs I never used linux system before

[haryoa]

nice moses/moses/scripts/training/train-model.perl -root-dir output/supervised/train -corpus data/labelled/train -f inf -e for -alignment grow-diag-final-and                         -reordering msd-bidirectional-fe                         -lm 0:3:/home/muhammadrizky15/stif-indonesia/output/supervised/lm/blm/data.blm.for -core 2 -external-bin-dir moses/training-tools --mgiza

Here you go

[Picasso1503]

I've got error

is it important to install miniconda? but I've been installed python environment to WSL2

[haryoa]

As long as you have Python 3.7++ on your WSL2, it is good to go. You do not need to install miniconda.

Can you do these steps in a sequence:

1. rm -rf output (remove output folder)
2. python -m stif_indonesia --exp-scenario supervised (run supervised experiment that will output the error)
3. nice moses/moses/scripts/training/train-model.perl -root-dir output/supervised/train -corpus data/labelled/train -f inf -e for -alignment grow-diag-final-and -reordering msd-bidirectional-fe -lm 0:3:/home/roggl552vw/stif-indonesia/output/supervised/lm/blm/data.blm.for -core 2 -external-bin-dir moses/training-tools --mgiza (run training)
4. Screenshot your error here

Note the roggl552vw on the step 3. I use your username on your WSL2. If you use different username, you need to change that part.

[Picasso1503]

First, delete output folder.

The second command line, I've got this error

The third command line, I've got this error as the same as before

[haryoa]

I think the error is caused when the program do training process. It didn't output extract.o.sorted.gz on your folder. I think, I need your full log to inspect your error.

Before you do the following, are you sure you download the right moses by running this command bash scripts/download_moses.sh ?

I can run it perfectly with my WSL2 on WSL2 Ubuntu 20.04 (I think if you have Ubuntu 17.04++, it can work).

Do these:

1. rm -rf output (remove output folder)
2. python -m stif_indonesia --exp-scenario supervised (run supervised experiment that will output the error)
3. sudo nice moses/moses/scripts/training/train-model.perl -root-dir output/supervised/train -corpus data/labelled/train -f inf -e for -alignment grow-diag-final-and -reordering msd-bidirectional-fe -lm 0:3:/home/roggl552vw/stif-indonesia/output/supervised/lm/blm/data.blm.for -core 2 -external-bin-dir moses/training-tools --mgiza &> out.log (run training and redirect the output to out.log. I add sudo here because I suspect that you didn't give permission to write to the folder.)
4. upload out.log here.

Also I need you to screen shot your ls -al ls -al output/supervised/train/model after you do these steps.

[Picasso1503]

out.log

That's All

[Picasso1503]

Ya, I've double check, I runned bash scripts/download_moses.sh in stif-indonesia folder

[goFrendiAsgard]

First, delete output folder.

The second command line, I've got this error

The third command line, I've got this error as the same as before

Hello @Picasso1503 , have you try to delete output directory and retrain?

[Picasso1503]

First, delete output folder. The second command line, I've got this error The third command line, I've got this error as the same as before

Hello @Picasso1503 , have you try to delete output directory and retrain?

Yap @goFrendiAsgard , I've been removed it using rm -rf output