python realtime service - internal error

sathyz commented 7 years ago

Following the steps in samples/python/tutorials/realtime/digit_classification.ipynb, I have created the model. Tests on DSVM passes.

Python 3.5.2 |Anaconda custom (64-bit)| (default, Jul  2 2016, 17:53:06) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import json
>>> import skdigits
>>> skdigits.init()
>>> skdigits.run("[[0.0,0.0,10.0,14.0,8.0,1.0,0.0,0.0,0.0,2.0,16.0,14.0,6.0,1.0,0.0,0.0,0.0,0.0,15.0,15.0,8.0,15.0,0.0,0.0,0.0,0.0,5.0,16.0,16.0,10.0,0.0,0.0,0.0,0.0,12.0,15.0,15.0,12.0,0.0,0.0,0.0,4.0,16
.0,6.0,4.0,16.0,6.0,0.0,0.0,8.0,16.0,10.0,8.0,16.0,8.0,0.0,0.0,1.0,8.0,12.0,14.0,12.0,1.0, 0.0]]")
8
>>>

I'm getting the following error on sending request to the service,

$ az ml service create realtime -f skdigits.py -d sklearn  -n skdigitsapp                                                                                         
Uploading dependencies.
 sklearn
 /anaconda/envs/py35/lib/python3.5/site-packages/azure/cli/command_modules/ml/service/azuremlutilities.py
Creating docker image...done.
Image available at : sparkscoreacr.azurecr.io/skdigitsapp
[Local mode] Running docker container.
Using default tag: latest
latest: Pulling from skdigitsapp
c62795f78da9: Already exists 
d4fceeeb758e: Already exists 
5c9125a401ae: Already exists 
0062f774e994: Already exists 
6b33fd031fac: Already exists 
81fdc302299d: Already exists 
cb7b074cdf3f: Already exists 
909391f68132: Already exists 
eb2dac95a3e7: Already exists 
bd3a5b3cf71b: Already exists 
5d0658498fa4: Already exists 
3a27844d7e55: Already exists 
30dcec6a5e25: Already exists 
93a4a29b64e8: Already exists 
0a3842698ae8: Already exists 
c1aea37fe470: Already exists 
3e9ded146371: Pull complete 
034b217c74fe: Pull complete 
Digest: sha256:83bf60a4ff1143b4624ddc3d646a14d4fa63699b4fa7919f6c0c70f337294141
Status: Downloaded newer image for sparkscoreacr.azurecr.io/skdigitsapp:latest
[Local mode] Success.
[Local mode] Scoring endpoint: http://127.0.0.1:32774/score
[Local mode] Usage: az ml service run realtime -n skdigitsapp [-d '{"input":"!! YOUR DATA HERE !!"}']
$ aml service run realtime -n skdigitsapp -d '{"input":"[[0.0,0.0,10.0,14.0,8.0,1.0,0.0,0.0,0.0,2.0,16.0,14.0,6.0,1.0,0.0,0.0,0.0,0.0,15.0,15.0,8.0,15.0,0.0,0.0,0.0,0.0,5.0,16.0,16.0,10.0,0.0,0.0,0.0,0.0,12.0,15.0,15.0,12.0,0.0,0.0,0.0,4.0,16.0,6.0,4.0,16.0,6.0,0.0,0.0,8.0,16.0,10.0,8.0,16.0,8.0,0.0,0.0,1.0,8.0,12.0,14.0,12.0,1.0, 0.0]]"}'
b'<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">\n<title>500 Internal Server Error</title>\n<h1>Internal Server Error</h1>\n<p>The server encountered an internal error and was unable to complete your request.  Either the server is overloaded or there is an error in the application.</p>\n'
$

ElliotWood commented 7 years ago

What do docker logs say? (e.g. docker logs )

ElliotWood commented 7 years ago

I just tested the digit_classification sample and got the same error. [500 Internal Server Error]. It looks like the underlying flask app can't deserialize the Numpy array, it simply passes a JSON string through.

Below is some sample code to convert the JSON string to an np.array, which should fix the error.

#%%writefile main.py
#The init and run functions will load and score your input using the saved model.
#They will also be used to generate the main.py script which will be part of your create service call.
def init():   
    # read in the model file
    from sklearn.externals import joblib
    global model
    model = joblib.load('sklearn/model.pkl')
    print("init done.")    

def run(input_array):
    import json

    import numpy as np
    if(type(input_array) == str):
        input_json = json.loads(input_array)
        input_array = np.array(input_json["input_array"])

    if (input_array.shape != (1, 64)):
        return 'Bad input: Expecting a json encoded list of lists of shape (1,64).'
    else:
        pred = model.predict(input_array)[0]
        print("init run.")  
        return json.dumps(str(pred))

Azure-Samples / Machine-Learning-Operationalization

python realtime service - internal error #10

Below is some sample code to convert the JSON string to an np.array, which should fix the error.