nok / sklearn-porter

Transpile trained scikit-learn estimators to C, Java, JavaScript and others.
BSD 3-Clause "New" or "Revised" License
1.28k stars 170 forks source link

How to read the multi-layer perceptrons model in Java written using python #6

Closed palaiya closed 7 years ago

palaiya commented 7 years ago

I am using the wrapper of scikit-learn Multilayer Perceptron in Python https://github.com/aigamedev/scikit-neuralnetwork to train the neural network and save it to a file. Now, I want to expose it on production to predict in real time. So, I was thinking to use Java for better concurrency than Python. Hence, my question is whether can we read the model using this library written using Python or above wrapper? The code below I am using for training the model and last three lines I want to port to Java to expose it on production

import pickle
import numpy as np
import pandas as pd
from sknn.mlp import Classifier, Layer
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

f = open("TrainLSDataset.csv")
data = np.loadtxt(f,delimiter = ',')

x = data[:, 1:]
y = data[:, 0]
X_train, X_test, y_train, y_test = train_test_split(x, y, test_size=0.3)

nn = Classifier(
    layers=[                    
        Layer("Rectifier", units=5),
        Layer("Softmax")],
    learning_rate=0.001,
    n_iter=100)

nn.fit(X_train, y_train)
filename = 'finalized_model.txt'
pickle.dump(nn, open(filename, 'wb')) 

**Below code i want to write in Java/GoLang for exposing it on Production** :

loaded_model = pickle.load(open(filename, 'rb'))
result = loaded_model.score(X_test, y_test)
y_pred = loaded_model.predict(X_test)
nok commented 7 years ago

Hello @palaiyacw,

the sklearn-porter module transpiles a subset of machine learning approaches which are implemented in the official scikit-learn project. Additional machine learning approaches or methods contributed by the community will be added in a more stable version.

Because of that I added the scikit-neuralnetwork project to the list of proposed classifiers.

Thanks and happy coding, Darius 🌵

palaiya commented 7 years ago

How long will it take to transpile scikit-neuralnetwork project to Java or GoLang?

palaiya commented 7 years ago

I tried to transpile the above code to java. As you have mentioned for multilayer perceptron scikit-learn>=0.18.0. I am using the following combination which satisfy your requirement. But i still get the following error:

Python : 2.7.6

Name: sklearn-porter
Version: 0.3.2

Name: scikit-learn
Version: 0.18.1

Name: numpy
Version: 1.11.3

Name: scipy
Version: 0.18.1
res = Porter(language='java').port(loaded_model) 
print(res)
Traceback (most recent call last):
  File "LeadScore.py", line 49, in <module>
    result1 = Porter(language='java').port(loaded_model) 
  File "/usr/local/lib/python2.7/dist-packages/sklearn_porter/Porter.py", line 60, in port
    cat, name = self.get_model_data(model)
  File "/usr/local/lib/python2.7/dist-packages/sklearn_porter/Porter.py", line 152, in get_model_data
    cat = Porter.is_supported_model(model)
  File "/usr/local/lib/python2.7/dist-packages/sklearn_porter/Porter.py", line 247, in is_supported_model
    raise ValueError(msg)
ValueError: The model is not an instance of a supported classifier or regressor.
nok commented 7 years ago

@palaiyacw,

Additional machine learning approaches or methods contributed by the community will be added in a more stable version.

In addition everything works as expected and all tests passed without any error. Please read the description again. You confuse the official implementation sklearn.neural_network.MLPClassifier with your requested project scikit-neuralnetwork.

Happy coding, Darius 🌵