vinhkhuc / JFastText

Java interface for fastText
Other
228 stars 100 forks source link

Loading the model only one time! #31

Closed Rowing0914 closed 4 years ago

Rowing0914 commented 6 years ago

I am thinking to implement this brilliant library into my project. But when it comes to the production level coding, one thing on my mind, which is loading model.

Could you tell us that how we just need to load the model only 1 time.

" One time" in a sense, once we load the model then we retain it while classifying the text.

For instance,

  1. load the model
  2. get text and classify it
  3. retain the model on memory --> go back to 2

The thing I know is the function below. // Load model from file jft.loadModel("src/test/resources/models/supervised.model.bin");

If we succeed to do this, it will save our resource to classify a new message without loading the model to memory again.

Best,

ashuezy commented 6 years ago

Its quite easy::

import java.util.ArrayList;
import java.util.List;

import org.apache.log4j.Logger;

import com.github.jfasttext.JFastText;
import com.github.jfasttext.JFastText.ProbLabel;
import com.xyz.constants.SentimentConstants;

public class TaxonomyUtil
{
    private static final Logger logger = Logger.getLogger(TaxonomyUtil.class);
    private static JFastText jft = new JFastText();

    public static void init()
    {
        jft.loadModel(PathUtil.getPath(SentimentConstants.TAXONOMY_FILE_PATH));
        System.out.println("Taxonomy Model -done");
    }

    public static ArrayList<String> getLabel(String entireData)
    {
        ArrayList<String> list = new ArrayList<String>();
        String label="";
        try
        {
            List<ProbLabel> predictionList = jft.predictProba(entireData, 1);
            if(predictionList.size()>0)
            {
                label = predictionList.get(0).label.replaceAll("__label__", "");
                list.add(label);
            }
        }
        catch(Exception e)
        {
            logger.error(e);
            e.printStackTrace();
        }
        return list;
    }

    public static void main(String args[])
    {
        init();
        System.out.println(getLabel("how are you"));
    }
}

Just call init at server startup one time Then call the getLabel method for every classification.

Rowing0914 commented 4 years ago

Thank you for your help!!