search

neulab / awesome-align

A neural word aligner based on multilingual BERT
https://arxiv.org/abs/2101.08231
BSD 3-Clause "New" or "Revised" License
325 stars 47 forks source link

setting output prob file and output word file #26

Closed pskadasi closed 3 years ago

pskadasi commented 3 years ago

how do i set output prob file and output word file to get the alignemnents as said in this snap ?

zdou0830 commented 3 years ago

Hi, you can see the options here https://github.com/neulab/awesome-align/blob/master/awesome_align/run_align.py#L222-L227.

An example:

DATA_FILE=/path/to/data/file
MODEL_NAME_OR_PATH=bert-base-multilingual-cased
OUTPUT_FILE=/path/to/output/file
OUTPUT_PROB_FILE=/path/to/output/prob/file
OUTPUT_WORD_FILE=/path/to/output/word/file

CUDA_VISIBLE_DEVICES=0 awesome-align \
    --output_file=$OUTPUT_FILE \
    --model_name_or_path=$MODEL_NAME_OR_PATH \
    --data_file=$DATA_FILE \
    --extraction 'softmax' \
    --output_prob_file $OUTPUT_PROB_FILE \
    --output_word_file $OUTPUT_WORD_FILE \
    --batch_size 32
pskadasi commented 3 years ago

thanks for the very quick response. it worked :)