pommedeterresautee / fastrtext

R wrapper for fastText
https://pommedeterresautee.github.io/fastrtext/
Other
101 stars 15 forks source link

Question - how to load vec & bin file from external source? #22

Closed dockstreet closed 5 years ago

dockstreet commented 6 years ago

Hi,

I would like to load a pre-trained non binary model, but not sure how to do it with load_model?

Does it need to be in bin format for load_model to work, or is there an argument that could be passed such as binary=False?

I'm trying to use this: https://s3-us-west-1.amazonaws.com/fasttext-vectors/wiki-news-300d-1M.vec.zip

Not sure if this a duplicate of : https://github.com/pommedeterresautee/fastrtext/issues/17

I also tried to load a bin file from and got an error https://s3-us-west-1.amazonaws.com/fasttext-vectors/wiki.en.zip

modelwiki <- load_model(paste0('C:\Users\xxx\R\fastrtext_test\wiki.en\wiki.en')) add .bin extension to the path Error in (function (env, objName) : argument to 'findVar' is not an environment get_analogies(modelwiki, "PhD", "MS","school") Error in model$get_nn_by_vector(vec, c(w1, w2, w3), k) : Encountered NaN.

get_parameters(modelwiki) $learning_rate [1] 0.05 $learning_rate_update [1] 100 $dim [1] 300 $context_window_size [1] 5 $epoch [1] 5 $min_count [1] 5 $min_count_label [1] 0 $n_sampled_negatives [1] 5 $word_ngram [1] 1 $bucket [1] 2000000 $min_ngram [1] 3 $max_ngram [1] 6 $sampling_threshold [1] 1e-04 $label_prefix [1] "label" $pretrained_vectors_filename [1] "" $nlabels [1] 0 $n_words [1] 2519370 $loss_name [1] "ns" $model_name [1] "sg"

pommedeterresautee commented 6 years ago

Load model function is for binary model. if you need to init a learning with an existing model you can use the txt model in execute function. use print_help function for more information.

dockstreet commented 6 years ago

Hi - I was able to create and save a binary model from python using fasttextr using read.fasttext;

could it be related to this?

https://github.com/facebookresearch/fastText/issues/203

the same model doesn't load with load_model from fastrtext, any suggestions to help troubleshoot it? It tells me it has the wrong file format in the error , but nothing more - kind regards

pommedeterresautee commented 6 years ago

Indeed you need to load a text model.