search

apple / turicreate

Turi Create simplifies the development of custom machine learning models.
BSD 3-Clause "New" or "Revised" License
11.2k stars 1.14k forks source link

Making Activity Classifier updatable #2258

Open mecid opened 5 years ago

mecid commented 5 years ago

Hello, I try to do my Activity Classifier updatable using this script.

import coremltools

spec = coremltools.utils.load_spec('RawSleepClassifier.mlmodel')

spec.description.trainingInput.extend([spec.description.input[0]])
spec.description.trainingInput.extend([spec.description.output[-1]])

builder = coremltools.models.neural_network.NeuralNetworkBuilder(spec=spec)
builder.make_updatable(['dense_layer1'])
builder.set_categorical_cross_entropy_loss(name='lossLayer', input='activityProbability', target='trueActivityLabel')

from coremltools.models.neural_network import SgdParams
builder.set_sgd_optimizer(SgdParams(lr=0.01, batch=32))

builder.set_epochs(10)

from coremltools.models import MLModel
mlmodel_updatable = MLModel(builder.spec)
mlmodel_updatable.save('SleepClassifier.mlmodel')

But I get the error:

RuntimeWarning: You will not be able to run predict() on this Core ML model. Underlying exception message was: Error compiling model: "Error reading protobuf spec. validator error: Model specification version for an updatable model must be '4' or above.".

I run MacOS 10.15 Beta 6 and Turi Create 5.7 with CoreML tools v3.0b3

TobyRoseman commented 5 years ago

It looks like TuriCreate is still exporting to Core ML using specificationVersion: 1. You do indeed need version 4 in order to have an updatable model. There are also lots of other improvements made in Core ML since the first version.

We should update TuriCreate to export models to Core ML using the most recent specification version.

mecid commented 5 years ago

@TobyRoseman so there is no way to generate updatable Activity Classifier with Turi Create 5.7? As I remember it works before.

TobyRoseman commented 5 years ago

@mecid - There is no way to generate an updatable Core ML model with Turi Create 5.7. I also don't think this has worked with any previous version of Turi Create.

mecid commented 5 years ago

@TobyRoseman can we expect it anytime soon?

hollance commented 5 years ago

What if you add spec.specificationVersion = 4 to the above script?

mecid commented 5 years ago

@hollance it works! Model works correctly. I will try to run on-device update task. Is it legal to change version like this? I mean I can set version to 5 or whatever and it also works.

hollance commented 5 years ago

@mecid I'm not sure what version of coremltools you're using, but I think in the latest version, builder.set_categorical_cross_entropy_loss() will automatically fill in that version number. If not, you can fill in anything you like here, but I would use 4 as that is the correct number. ;-)

mecid commented 5 years ago

@hollance so you mean by setting version to 4 and using builder.set_categorical_cross_entropy_loss() my Model packaging will be updated to latest version? I on the latest coreml tools beta version.

hollance commented 5 years ago

@mecid I was slightly mistaken, builder. make_updatable() will now automatically fill in the specificationVersion for you. Not sure why that didn't happen automatically in your script, as with the current beta version of coremltools it should do that. This means you shouldn't have to fill in specificationVersion yourself anymore.

mecid commented 5 years ago

@hollance I'm on CoreML tools 3.0b3 and Turi Create 5.7. Is it safe to use that model with manual setting version to 4? I fear that it can be broken.

hollance commented 5 years ago

It is safe. Note that coremltools 3.0b4 is the latest, not b3.

mecid commented 5 years ago

@hollance thanks, just checked it now, we already have b5.

hollance commented 5 years ago

Ah nice!

TobyRoseman commented 4 years ago

Created #2860 to track the more general issue (i.e. this issue for all toolkits that support exporting to Core ML).