machinebox / issues

Machine Box issues, bugs and feature requests
2 stars 0 forks source link

classificationbox: saving model state #23

Closed laurenblackburn closed 6 years ago

laurenblackburn commented 6 years ago

When I download a model state from classification box the prediction, examples, and classes are not saved. For example - I created a test model, model3, with 2 classes, 448 examples, and 8 predictions:

$ curl -X GET http://localhost:8084/classificationbox/models/model3/stats
{
    "success": true,
    "predictions": 8,
    "examples": 448,
    "classes": [
        {
            "name": "class1",
            "examples": 224
        },
        {
            "name": "class2",
            "examples": 224
        }
    ]
}

I download the model with

$ curl -X GET http://localhost:8084/classificationbox/state/model3 > model3.classificationbox

When I close machinebox and reopen it to test my saved state, model3's stats are altered and it appears that only the name and ID have been saved:

$ curl -X GET http://localhost:8084/classificationbox/models
{
    "success": true,
    "models": []
}
$ curl -X POST -F 'file=@model3.classificationbox' http://localhost:8084/classificationbox/state
{
    "success": true,
    "id": "model3",
    "name": "test",
    "options": {}
}
$ curl -XGET http://localhost:8084/classificationbox/models
{
    "success": true,
    "models": [
        {
            "id": "model3",
            "name": "test"
        }
    ]
}
$ curl -X GET http://localhost:8084/classificationbox/models/model3/stats
{
    "success": true,
    "predictions": 0,
    "examples": 0,
    "classes": []
}

How do I save the classes and examples for each model?

dahernan commented 6 years ago

The classes are probably saved correctly, try to do curl -XGET http://localhost:8084/classificationbox/models/model3 or run another prediction, to see that the classes are there

What are not saved with the model are the stats, the stats volatile right now.

Can you confirm this behavior?

laurenblackburn commented 6 years ago

Confirmed that the classes appear with I do curl -XGET http://localhost:8084/classificationbox/models/model3

Does this mean that there is no way to tell how many examples you've trained each class over if you train over more than one instance of classifcationbox?

dahernan commented 6 years ago

You need to count it on the client side right now, I don't think we are going to support saving the stats, because you can monitor that in many ways

laurenblackburn commented 6 years ago

Okay, thanks for the clarification

dahernan commented 6 years ago

No problem, let us know any other problem or question