Computation on large corpus

[box-key]

Implement a function to compute each score for large corpus. It reads sentence pairs line by line, compute score for each pair, then aggregate the result.

[box-key]

If I can read file by chunk, I can use tqdm to show the progress.

Reference:

• https://stackoverflow.com/questions/6475328/how-can-i-read-large-text-files-in-python-line-by-line-without-loading-it-into

• https://stackoverflow.com/questions/24890368/iterate-over-large-file-with-progress-indicator-in-python