tspdb crashing

[gsxipsarl]

Hi, I have been testing tspdb on few datasets and noticed it crashes sometimes. It originally happens because calling create_pindex on a dataset throws: PL/Python function "create_pindex", line 268, in create_index
PL/Python function "create_pindex", line 309, in update_model PL/Python function "create_pindex", line 140, in update_model PL/Python function "create_pindex", line 238, in fitModels PL/Python function "create_pindex", line 222, in fit PL/Python function "create_pindex", line 119, in _computeWeights PL/Python function "create_pindex", line 77, in matrixFromSVD

Looking into it I realized it happens because in line 55 of algorithms/svdWrapper.py, you are returnig a list instead of datafram. so I changed it to: return (np.array([]), np.array([]), np.array([]))

Then the error changes: ERROR: IndexError: invalid index to scalar variable. CONTEXT: Traceback (most recent call last): PL/Python function "create_pindex", line 18, in TSPD.create_index() PL/Python function "create_pindex", line 268, in create_index PL/Python function "create_pindex", line 309, in update_model PL/Python function "create_pindex", line 140, in update_model PL/Python function "create_pindex", line 238, in fitModels PL/Python function "create_pindex", line 222, in fit PL/Python function "create_pindex", line 123, in _computeWeights PL/Python function "create_pindex"

And this happens because in previous part returned empty dataframe. looking into why that line is executed, it is because k==0 (kSingularValue==0)

Then I traced back and it happens because in prediction_models/ts_svd_model.py fit function, this code: (self.sk, self.Uk, self.Vk) = svdMod.reconstructMatrix(self.kSingularValues, returnMatrix=False) if self.kSingularValues is None: self.kSingularValues= len(self.sk) will change kSingularValues to 0 (somehow self.sk is 0)

That's where I stopped and could not check further. If you want, I can send you my dataset to simulate the problem.

Thanks!

[AbdullahO]

Thanks avabarid for your feedback.

I suspect the issue is that the default method that selects the rank end up selecting sk =0, which is easily fixed by ensuring that the chosen rank is always >=1.

I fixed that issue in the latest commit. Let me know if that fixes things. If it does not, we can take a closer look at the data and try to replicate the error.

Thanks

[gsxipsarl]

Hi Abdullah,

Let me check and I will get back to you.

Thanks!

From: AbdullahO @.*** Sent: Sunday, May 9, 2021 8:48 AM To: AbdullahO/tspdb Cc: avabarid; Author Subject: Re: [AbdullahO/tspdb] tspdb crashing (#5)

Thanks avabarid for your feedback.

I suspect the issue is that the default method that selects the rank end up selecting sk =0, which is easily fixed by ensuring that the chosen rank is always >=1.

I fixed that issue in the latest commit. Let me know if that fixes things. If it does not, we can take a closer look at the data and try to replicate the error.

Thanks

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/AbdullahO/tspdb/issues/5#issuecomment-835714398 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ALOQH4GR3DFQFMB3WITMMYLTMYVYXANCNFSM44OHSKAQ . https://github.com/notifications/beacon/ALOQH4HTIH6N6OWI4OOHNDTTMYVYXA5CNFSM44OHSKA2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOGHH72XQ.gif

[gsxipsarl]

I think your latest update fixed it! thank you.