TypeError: Object of type 'Tensor' is not JSON serializable

[hieu-lam]

Hi,

We have successfully trained the model, but we got a problem while saving the model to a file. The exception is as below:

epoch 54 total loss 0.20950795884999707 0.00021984046049317634 aida-A micro F1: 0.9299655568312285 aida-B micro F1: 0.9415830546265329 msnbc micro F1: 0.9395562356541699 aquaint micro F1: 0.8797202797202797 ace2004 micro F1: 0.8853118712273641 clueweb micro F1: 0.7417355371900826 wikipedia micro F1: 0.7820427483174321 change learning rate to 0.0001 att_mat_diag tok_score_mat_diag entity2entity_mat_diag entity2entity_score_mat_diag knowledge2entity_mat_diag knowledge2entity_score_mat_diag type_emb cnn.weight cnn.bias score_combine.0.weight score_combine.0.bias score_combine.3.weight score_combine.3.bias save model to model/ Traceback (most recent call last): File "main.py", line 225, in ranker.train(conll.train, dev_datasets, config) File "../DCA/ed_ranker.py", line 1032, in train self.model.save(self.args.model_path) File "../DCA/abstract_word_entity.py", line 78, in save json.dump(config, f) File "/usr/lib64/python3.6/json/init.py", line 179, in dump for chunk in iterable: File "/usr/lib64/python3.6/json/encoder.py", line 430, in _iterencode yield from _iterencode_dict(o, _current_indent_level) File "/usr/lib64/python3.6/json/encoder.py", line 404, in _iterencode_dict yield from chunks File "/usr/lib64/python3.6/json/encoder.py", line 325, in _iterencode_list yield from chunks File "/usr/lib64/python3.6/json/encoder.py", line 437, in _iterencode o = _default(o) File "/usr/lib64/python3.6/json/encoder.py", line 180, in default o.class.name) TypeError: Object of type 'Tensor' is not JSON serializable

Previously we got the same issue as #7 , but we followed the instruction and got that fixed.

We hope you can assist us with this issue :-)

[YoungXiyuan]

Thank you for your interest in our work and sorry for my late reply.

This issue is mainly caused by the version compatibility problem in the Pytorch. The DCA framework currently does not work with Pytorch 0.4 or higher, you need to install Pytorch 0.3, preferably in a virtual environment so that you can have both versions installed at the same time.

And I also find some possible solutions for this issue if someone have to use Pytorch >= 0.4： （1）https://discuss.pytorch.org/t/typeerror-tensor-is-not-json-serializable/36065 （2）https://github.com/fossasia/visdom/issues/554

I suspect that tensors stored in variable config of "abstract_word_entity.py#Line75" does not convert to numpy arrays automatically. So you can convert them explicitly as the following manner in "abstract_word_entity.py#Line73-75":

for k, v in self.__dict__.items():
   if not hasattr(v, '__dict__'):
      if isinstance(v, torch.Tensor):
         config[k] = v.cpu().data.numpy()
      else:
         config[k] = v

The above suggestion should hopefully do exactly that automatically, but I haven't been able to take a chance to test it. Glad to hear you found a solution though.

[hieu-lam]

Hm... That doesn't work unfortunately. We've also tried config[k] = v.cpu().numpy() - but the exact same problem still appears.

[YoungXiyuan]

Can you print the data type of the variable v using print(type(v)) before the code line "config[k] = v" ?

I am not sure whether there are some lists of tensors stored in the variable v. While we are able to convert tensors to numpy arrays via "v.cpu().numpy()" or "v.cpu().data.numpy()", converting a list of tensors currently might not be supported.

[hieu-lam]

Hi, I'm sorry for my late reply, here is what we added to the code:

for k, v in config.items():
        print(k, type(v))

And here is the output (none of them is Tensor type, so I guess there must be a list of Tensor somewhere):

......
word_voca <class 'dict'>
entity_voca <class 'dict'>
training <class 'bool'>
_non_persistent_buffers_set <class 'list'>
emb_dims <class 'int'>
freeze_embs <class 'bool'>
hid_dims <class 'int'>
tok_top_n <class 'int'>
margin <class 'float'>
param_copy_switch <class 'bool'>
dr <class 'float'>
gamma <class 'float'>
tok_top_n4ment <class 'int'>
tok_top_n4ent <class 'int'>
tok_top_n4word <class 'int'>
tok_top_n4inlink <class 'int'>
order_learning <class 'bool'>
dca_method <class 'int'>
ent_unk_id <class 'int'>
word_unk_id <class 'int'>
ent_inlinks <class 'dict'>
use_local <class 'bool'>
use_local_only <class 'bool'>
freeze_local <class 'bool'>
saved_log_probs <class 'list'>
rewards <class 'list'>
actions <class 'list'>
order_saved_log_probs <class 'list'>
decision_order <class 'list'>
targets <class 'list'>
record <class 'bool'>
flag <class 'int'>
added_words <class 'list'>
added_ents <class 'list'>
Traceback (most recent call last):
  File "main.py", line 225, in <module>
    ranker.train(conll.train, dev_datasets, config)
  File "../DCA/ed_ranker.py", line 1032, in train
    self.model.save(self.args.model_path)
  File "../DCA/abstract_word_entity.py", line 84, in save
    json.dump(config, f)
  File "/home/trol/.conda/envs/dca/lib/python3.6/json/__init__.py", line 179, in dump
    for chunk in iterable:
  File "/home/trol/.conda/envs/dca/lib/python3.6/json/encoder.py", line 430, in _iterencode
    yield from _iterencode_dict(o, _current_indent_level)
  File "/home/trol/.conda/envs/dca/lib/python3.6/json/encoder.py", line 404, in _iterencode_dict
    yield from chunks
  File "/home/trol/.conda/envs/dca/lib/python3.6/json/encoder.py", line 325, in _iterencode_list
    yield from chunks
  File "/home/trol/.conda/envs/dca/lib/python3.6/json/encoder.py", line 437, in _iterencode
    o = _default(o)
  File "/home/trol/.conda/envs/dca/lib/python3.6/json/encoder.py", line 180, in default
    o.__class__.__name__)
TypeError: Object of type 'Tensor' is not JSON serializable

[hieu-lam]

Hi again,

We've looked into the specific element in those lists and found out that the "targets" is actually the cause - it's a list of Tensor.

for k, v in config.items():
                print(k, type(v))
                if(isinstance(v, list) and (len(v) > 0)):
                    print("list elem type: ",type(v[0]))

Output:

targets <class 'list'>
list elem type:  <class 'torch.Tensor'>

We have some questions regarding to this: is it important to save the "targets" list? Can we ignore it? Can we load the model properly again?

[YoungXiyuan]

Thank you for your provided information and sorry for my late reply.

We have spent some time inspecting the variable "targets" in the mulrel_ranker.py and find out that the variable "target" is associated with a deprecated ablation study experiment and should be useless for the current DCA framework.

I suggest you to reset the variable "targets" to an empty list or not to include it into the dict "config" in the code line #75 of abstract_word_entity.py, and I don't think those operations would affect the model loading during the evaluation phase.

Give it a try (:

[hieu-lam]

Thank you for your suggestion and sorry for my late reply. We've set the "targets" key to an empty list and now everything works just fine :-) We appreciate your assistance.