Model Persistence

[JonnoFTW]

The models need to be periodically stored in a database using a BLOB (since nupic models can be converted to a string using cPickle).

Code might look something like this:

import cPickle as pickle
def store_model(guid):
    blob = pickle.dumps(model['model'])
    netinfo = pickle.dumps(model['model']._netinfo.save()) #this won't work, I can't tell how network saving actually works
    store_blob_in_db(guid, blob, netinfo)
    del blob, netinfo

We also need to take into account the fact that the OPF CLA model needs some extra serialization: https://github.com/numenta/nupic/blob/master/src%2Fnupic%2Fframeworks%2Fopf%2Fclamodel.py?ts=4#L1286-L1306 This data could probably be stored in a second BLOB column.

Basically, I propose the following behaviour (subject to feedback):

• Models should be persisted in a BLOB field in MySQL.
• All in memory models should be stored in the database every 5 minutes
• Models should be dumped to the database if they are unused for 5 minutes.
• If a GET, PUSH or reset API call is made to a model that is not in memory, the server will attempt to retrieve the model by its GUID from the database and then perform the requested action.
• DB schema to look like:

id: Integer, pk
guid: varchar,
model: BLOB
net_info: BLOB
created: timestamp
updated: timestamp

Other questions:

• Should models not in memory be included in the list of all models?

[rhyolight]

@oxtopus @vitaly-krugl Do either of you have any advice about saving OPF models into MySQL?

[vitaly-krugl]

@JonnoFTW, @rhyolight: assuming a BLOB can accommodate large models that can be upwards of a Gig or more (uncompressed): an OPF model has the save(...) method for saving a model to a specified directory in the filesystem. For example, see https://github.com/numenta/numenta-apps/blob/1a1e2bea770b94577a175f21e4441fa6212a7b05/htmengine/htmengine/model_checkpoint_mgr/model_checkpoint_mgr.py#L389-L392. You might then want to tar and compress the directory contents using a fast compression algorithm, such as LZ4, before sending it to the database.

For loading the model, you would get the tar/compressed image from the database, uncompress and un-tar it; then use ModelFactory.loadFromCheckpoint to load the model into memory like this: https://github.com/numenta/numenta-apps/blob/1a1e2bea770b94577a175f21e4441fa6212a7b05/htmengine/htmengine/model_checkpoint_mgr/model_checkpoint_mgr.py#L465