Closed ninoslavc closed 1 year ago
Hi @ninoslavc, I've checked this code and it works:
PATH = Path('models')
PATH.mkdir(parents=True, exist_ok=True)
learn.export(PATH/'Transformer.pkl')
learn2 = load_learner(PATH/'Transformer.pkl')
Please, try it and let me know if you still have the issue.
Hi @oguiza
Thank you. Unfortunately, I still have the same problem. Here is the full traceback:
TypeError Traceback (most recent call last)
Cell In[56], line 3
1 PATH = Path('models')
2 PATH.mkdir(parents=True, exist_ok=True)
----> 3 learn.export(PATH/'Transformer.pkl')
4 learn2 = load_learner(PATH/'Transformer.pkl')
File ~\AppData\Roaming\Python\Python39\site-packages\fastai\learner.py:428, in export(self, fname, pickle_module, pickle_protocol)
425 with warnings.catch_warnings():
426 #To avoid the warning that come from PyTorch about model not being checked
427 warnings.simplefilter("ignore")
--> 428 torch.save(self, self.path/fname, pickle_module=pickle_module, pickle_protocol=pickle_protocol)
429 self.create_opt()
430 if state is not None: self.opt.load_state_dict(state)
File ~\AppData\Roaming\Python\Python39\site-packages\torch\serialization.py:423, in save(obj, f, pickle_module, pickle_protocol, _use_new_zipfile_serialization)
421 if _use_new_zipfile_serialization:
422 with _open_zipfile_writer(f) as opened_zipfile:
--> 423 _save(obj, opened_zipfile, pickle_module, pickle_protocol)
424 return
425 else:
File ~\AppData\Roaming\Python\Python39\site-packages\torch\serialization.py:635, in _save(obj, zip_file, pickle_module, pickle_protocol)
633 pickler = pickle_module.Pickler(data_buf, protocol=pickle_protocol)
634 pickler.persistent_id = persistent_id
--> 635 pickler.dump(obj)
636 data_value = data_buf.getvalue()
637 zip_file.write_record('data.pkl', data_value, len(data_value))
TypeError: cannot pickle 'code' object
Unusually, I have that problem only with TST models.
Hi @ninoslavc, There's something else you can try: remove the callbacks before exporting the model. I've seen that callbacks can sometimes create issues.
PATH = Path('models')
PATH.mkdir(parents=True, exist_ok=True)
learn.remove_cbs(learn.cbs) # <--- INSERT THIS
learn.export(PATH/'Transformer.pkl')
learn2
Hi @oguiza,
Unfortunately, that did not help:
TypeError Traceback (most recent call last)
Cell In[30], line 4
2 PATH.mkdir(parents=True, exist_ok=True)
3 learn.remove_cbs(learn.cbs) # <--- INSERT THIS
----> 4 learn.export(PATH/'Transformer.pkl')
5 learn2
File ~\AppData\Roaming\Python\Python39\site-packages\fastai\learner.py:428, in export(self, fname, pickle_module, pickle_protocol)
425 with warnings.catch_warnings():
426 #To avoid the warning that come from PyTorch about model not being checked
427 warnings.simplefilter("ignore")
--> 428 torch.save(self, self.path/fname, pickle_module=pickle_module, pickle_protocol=pickle_protocol)
429 self.create_opt()
430 if state is not None: self.opt.load_state_dict(state)
File ~\AppData\Roaming\Python\Python39\site-packages\torch\serialization.py:423, in save(obj, f, pickle_module, pickle_protocol, _use_new_zipfile_serialization)
421 if _use_new_zipfile_serialization:
422 with _open_zipfile_writer(f) as opened_zipfile:
--> 423 _save(obj, opened_zipfile, pickle_module, pickle_protocol)
424 return
425 else:
File ~\AppData\Roaming\Python\Python39\site-packages\torch\serialization.py:635, in _save(obj, zip_file, pickle_module, pickle_protocol)
633 pickler = pickle_module.Pickler(data_buf, protocol=pickle_protocol)
634 pickler.persistent_id = persistent_id
--> 635 pickler.dump(obj)
636 data_value = data_buf.getvalue()
637 zip_file.write_record('data.pkl', data_value, len(data_value))
TypeError: cannot pickle 'code' object
Hi again @ninoslavc, I'm running out of options. The issue is that I cannot reproduce the issue. See the code working in Colab: gist I'm not sure if this will make a difference, but you could pass the path when you instantiate the learner. Pass it as path:
from tsai.models.TST import TST
X, y, splits = get_UCR_data('LSST', split_data=False)
tfms = [None, TSClassification()]
batch_tfms = TSStandardize(by_sample=True)
dls = get_ts_dls(X, y, splits=splits, tfms=tfms, batch_tfms=batch_tfms)
PATH = Path('models')
PATH.mkdir(parents=True, exist_ok=True)
learn = ts_learner(dls, TST, metrics=accuracy, path=PATH, cbs=[ShowGraph()])
learn.fit_one_cycle(10, 1e-2)
learn.export('Transformer.pkl')
Let me know if this doesn't work.
BTW, do you create any function/class that ends up in the learner in your script? Or are all functions/ classes in modules?
Hi @oguiza,
Thank you for your suggestions.
I did not change anything in the tutorial file - I used it as is. Adding a path did not help. Also, I tried to change cbs=ShowGraphCallback2()
with cbs=[ShowGraph()]
, but still got the error TypeError: cannot pickle 'code' object
.
I don't get how come that only TST models have that problem. All others work fine.
ok. Are you using loss_func=LabelSmoothingCrossEntropyFlat()
in your learner. If so, please, remove it, and try it again. I had a similar pickle issue last year. I'm just trying to isolate the issue. You may try removing certain components(like ShowGraph()) that are not necessary to run the code, and see if you can find what's causing the issue.
Hi @oguiza,
you are right, loss_func=LabelSmoothingCrossEntropyFlat()
caused the problem. After removing it, saving it to .pkl
works fine.
Thank you
I went through the tutorial: 07_Time_Series_Classification_with_Transformers.ipynb
Everything worked fine till I tried to export the TST model(s):
caused
TypeError: cannot pickle 'code' object
My configuration is following:
I tried to export the inception model at the beginning of the same tutorial, and there were no problems. I tried my own TST Transformers model and had the same problem. I have no problems exporting other tsai models in other tutorials.
How can this problem be solved? Also, I would suggest adding the part with inference on new data in this tutorial, like in others.