earthgecko
commented
2 years ago Hi @water-e
From one user to another. You can do that using the adtk Pipeline Pipe object and using aCustomizedTransformer1D see https://adtk.readthedocs.io/en/stable/notebooks/demo.html#CustomizedTransformer and https://adtk.readthedocs.io/en/stable/notebooks/demo.html#Pipeline it is quite flexible and you can add whatever steps you like with CustomizedTransformer1D and CustomizedDetector1D.
The Pipe and Pipeline methods take a bit of work to get familiar and if it proves too difficult or complicated, there is no reason why you cannot just transform the series and anomalies output in your own code with your own subsequent function/s and then pass the result to the next adtk method you wish to use.
As for the existing ThresholdAD being broken for missing data I cannot say. I have not used ThresholdAD on anything really. However, if you have debugged it and fixed it may be useful to open another issue describing it fully and adding an example data set and the code snippet that can be used to reproduce the issue and the code snippet to fix the issue. There has not been any maintenance activity on this repo for some time, so you just documenting the issue may help someone that runs into it in the future.
water-e
The existing examples all seem to envision serial transforms feeding into a single detector which are then logically connected.
I don't see any where a simple detector is used for a preliminary detection, values are screened and the most egregious anomalies deleted, then the next detector is used. Is this possible? It is a common pattern of work. For example, a Seasonal decomposition is more robust if ridiculous values (ThresholdAD) have been removed (I think the existing ThresholdAD is broken for missing data, but making it work is only a one line fix).