I passed a directory as the -vocab-file argument, which apparently doesn't trigger an error from fopen. This led to vocab_size = 0, which caused a segfault during execution.
I've raised an error if the vocab size is still 0 after trying to read the vocab file. This seems a bit better than checking for a directory, since we still should raise if an incorrect but existing file path was passed.
The PR should still be useful in addition to #138, as that PR only checks the status from fread.
I passed a directory as the
-vocab-fileargument, which apparently doesn't trigger an error fromfopen. This led tovocab_size = 0, which caused a segfault during execution.I've raised an error if the vocab size is still 0 after trying to read the vocab file. This seems a bit better than checking for a directory, since we still should raise if an incorrect but existing file path was passed.
The PR should still be useful in addition to #138, as that PR only checks the status from
fread.