search

vc1492a / PyNomaly

Anomaly detection using LoOP: Local Outlier Probabilities, a local density based outlier detection method providing an outlier score in the range of [0,1].
Other
319 stars 37 forks source link

Fix/data #57

Closed romajain1 closed 1 month ago

romajain1 commented 4 months ago

There is a function for checking missing values in the data. But, for python 3.6+, throws error. Traceback (most recent call last): File "offline_training.py", line 133, in <module> m = loop.LocalOutlierProbability(real_data, extent=2, n_neighbors=20, use_numba=True, progress_bar=True).fit() File "lib/python3.10/site-packages/PyNomaly/loop.py", line 751, in fit if self.data is not None and self.Validate._missing_values( File "/lib/python3.10/site-packages/PyNomaly/loop.py", line 238, in _missing_values if np.any(np.isnan(obj.data)): File "/lib/python3.10/site-packages/pandas/core/generic.py", line 2102, in __array_ufunc__ return arraylike.array_ufunc(self, ufunc, method, *inputs, **kwargs) File "/lib/python3.10/site-packages/pandas/core/arraylike.py", line 404, in array_ufunc result = mgr.apply(getattr(ufunc, method)) File "/lib/python3.10/site-packages/pandas/core/internals/managers.py", line 352, in apply applied = b.apply(f, **kwargs) File "/lib/python3.10/site-packages/pandas/core/internals/blocks.py", line 366, in apply result = func(self.values, **kwargs) File "/lib/python3.10/site-packages/pandas/core/arrays/categorical.py", line 1665, in __array_ufunc__ raise TypeError( TypeError: Object with dtype category cannot perform the numpy op isnan

IroNEDR commented 3 months ago

@vc1492a I have just finished up our new test-pipeline. It makes sense to merge the PR https://github.com/vc1492a/PyNomaly/pull/58 first so we can have automated testing active on this PR before merge

vc1492a commented 3 months ago

@vc1492a I have just finished up our new test-pipeline. It makes sense to merge the PR #58 first so we can have automated testing active on this PR before merge

Thanks @IroNEDR. I have approved #58 and it's ready for a merge.

IroNEDR commented 2 months ago

@romajain1 can you give us more details on how you encountered this error? We have a test pipeline setup for Python 3.6 running on Ubntu and a unit test that actually evaluates the userwarning raised by the missing values function as well as several others tests that also execute this method under the hood without problems.

Can you provide us with the following information:

That would be helpful so we can better evaluate the changes

vc1492a commented 1 month ago

Closing due to inactivity.