Inference Problem: TypeError: only integer scalar arrays can be converted to a scalar index

[jrfackler]

Hello,

When I try to do inference on the validation set: input1 = learn1.get_X_preds(X_valid)

I now get: TypeError: only integer scalar arrays can be converted to a scalar index

2 days ago it worked fine so something seems to have changed. I have tried it on 2 different machines with the same result.

Does anyone have any ideas?

my_setup()

os              : Linux-5.4.0-122-generic-x86_64-with-glibc2.31
python          : 3.9.13
tsai            : 0.3.4
fastai          : 2.7.10
fastcore        : 1.5.27
torch           : 1.12.0+cu116
device          : 1 gpu (['Quadro RTX 4000'])
cpu cores       : 8
threads per cpu : 1
RAM             : 29.37 GB
GPU memory      : [8.0] GB

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Input In [14], in <cell line: 19>()
     21 
     22 
---> 23 _, _, input1 = learn1.get_X_preds(X_valid, with_decoded=True)
     24 
     25 

File /usr/local/lib/python3.9/dist-packages/tsai/inference.py:17, in get_X_preds(self, X, y, bs, with_input, with_decoded, with_loss)
     15     print("cannot find loss as y=None")
     16     with_loss = False
---> 17 dl = self.dls.valid.new_dl(X, y=y, bs=bs)
     18 output = list(self.get_preds(dl=dl, with_input=with_input, with_decoded=with_decoded, with_loss=with_loss, reorder=False))
     19 if with_decoded and len(self.dls.tls) >= 2 and hasattr(self.dls.tls[-1], "tfms") and hasattr(self.dls.tls[-1].tfms, "decodes"):

File /usr/local/lib/python3.9/dist-packages/tsai/data/core.py:532, in NumpyDataLoader.new_dl(self, X, y, bs)
    530 assert X.ndim == 3, "You must pass an X with 3 dimensions [batch_size x n_vars x seq_len]"
    531 if y is not None and not is_listy(y): y = [y]
--> 532 return self.new(self.dataset.add_dataset(X, y=y), bs=min(bs, len(X)))

File /usr/local/lib/python3.9/dist-packages/fastai/data/core.py:97, in TfmdDL.new(self, dataset, cls, **kwargs)
     95 if not hasattr(self, '_n_inp') or not hasattr(self, '_types'):
     96     try:
---> 97         self._one_pass()
     98         res._n_inp,res._types = self._n_inp,self._types
     99     except Exception as e: 

File /usr/local/lib/python3.9/dist-packages/tsai/data/core.py:498, in _one_pass(self)
    496 @patch
    497 def _one_pass(self:TfmdDL):
--> 498     b = self.do_batch([self.do_item(0)])
    499     if self.device is not None: b = to_device(b, self.device)
    500     its = self.after_batch(b)

File /usr/local/lib/python3.9/dist-packages/fastai/data/load.py:168, in DataLoader.do_batch(self, b)
--> 168 def do_batch(self, b): return self.retain(self.create_batch(self.before_batch(b)), b)

File /usr/local/lib/python3.9/dist-packages/tsai/data/core.py:547, in NumpyDataLoader.create_batch(self, b)
    545 if hasattr(self, "split_idxs"): self.input_idxs = self.split_idxs[b]
    546 else: self.input_idxs = self.idxs
--> 547 return self.dataset[b]

File /usr/local/lib/python3.9/dist-packages/tsai/data/core.py:438, in TSDatasets.__getitem__(self, it)
    436 def __getitem__(self, it):
    437     if self.inplace:
--> 438         return tuple([ptl[it] for ptl in self.ptls])
    439     else:
    440         return tuple([typ(stack(ptl[it]))[...,self.sel_vars, self.sel_steps] if (i==0 and self.multi_index) else typ(stack(ptl[it])) \
    441                       for i,(ptl,typ) in enumerate(zip(self.ptls,self.typs))])

File /usr/local/lib/python3.9/dist-packages/tsai/data/core.py:438, in <listcomp>(.0)
    436 def __getitem__(self, it):
    437     if self.inplace:
--> 438         return tuple([ptl[it] for ptl in self.ptls])
    439     else:
    440         return tuple([typ(stack(ptl[it]))[...,self.sel_vars, self.sel_steps] if (i==0 and self.multi_index) else typ(stack(ptl[it])) \
    441                       for i,(ptl,typ) in enumerate(zip(self.ptls,self.typs))])

File /usr/local/lib/python3.9/dist-packages/tsai/data/core.py:278, in NoTfmLists.__getitem__(self, it)
    276 def __getitem__(self, it):
    277     if hasattr(self.items, 'oindex'): return self.items.oindex[self._splits[it]]
--> 278     else: return self.items[self._splits[it]]

TypeError: only integer scalar arrays can be converted to a scalar index

[jrfackler]

I am going to close this issue. I have been using the non-stable version of tsai for a long time without issue but all of a sudden the error above started to occur. I don't know the cause but when I switched to the stable version of tsai it works as expected again. I will just use the stable version.

[oguiza]

Hi @jrfackler, I've reopened this issue since it's something I'd like to investigate a bit further. Bear in mind that the latest version will become the stable version when I make a new pip release, which will happen soon. And I don't want to make it if there's a known issue like this. Could you provide a code snippet to reproduce this issue? I've tried it, but couldn't reproduce it.

[jrfackler]

Hi @oguiza,

I just emailed you a code snippet and the data files to reproduce the issue. Please let me know if you need anything else. Thanks!

[jrfackler]

Hi @oguiza,

Did you just update the stable version? It is causes that same error for me now. This there a way I could use the older version? Thanks.

[oguiza]

Hi @jrfackler, I created a new pip release (0.3.4) this morning, which is probably the reason why you see the issue again. I'm sorry I've been busy this week. I expect to have some more time next week. I'll look into the issue and try to fix it. In the meantime, you can just use:

pip install tsai==0.3.2

[jrfackler]

Thank you very much @oguiza! That version is working perfectly for me.

[oguiza]

Are you ok if I close this issue? The code snippet and data you sent to me do not reproduce this issue, but one with load_learner.

[jrfackler]

@oguiza, thank you very much for looking at this issue. This does seem to be caused by learn.export and the different versions of tsai so I think it okay to close the issue.

What I have found is that if I train the model again with the new version of tsai, then learn.export and load_learner work fine.

But, if I have an already trained model that used tsai version 0.3.2 then load_learner does not work with the new version of tsai and that model.

For already trained models, pip install tsai==0.3.2 works fine for load_learner.

I will also try the learn.save and learn.load method from the blog article for new training models.

Thanks again!

[yangtzech]

This happens to me. A model exported with learn.export in environment A is used to make predictions with learn = load_learner(PATH, cpu=False) and learn.get_X_preds(x_test,y_test) in environment B. Environment A is synced with https://github.com/timeseriesAI/tsai/commit/b075bfad924ab62a2e3f96594b77a42ac71ff136, while environment B is with https://github.com/timeseriesAI/tsai/commit/07aa6db6e38f5c7ab1791197cfd91e5d832f7eb2. But if I run the code that exports the model in environment B. The error does not exist.