search

robjhyndman / forecast

Forecasting Functions for Time Series and Linear Models
http://pkg.robjhyndman.com/forecast
1.1k stars 340 forks source link

tsoutlier unable to flag obvious outliers when total samples increased #907

Closed sanjmeh closed 10 months ago

sanjmeh commented 2 years ago

I have started to use tsoutliers and tsclean functions which are quite simple compared to tso of tsoutliers package. But after a few trials I am unable to understand why it is unable to detect obvious outliers when the window size of the time series is increased. I did a few trials and arrived at a point from where the window size has an impact.

It appears the algorithm takes in consideration the standard deviation of a large part of the time series and not just localised to its neigbours.

The case: I have longitute data for a moving vehicle. If you apply the tsclean on the lg data for the first 1000 values it successfully detects outliers and cleans them. But if we go to first 3000 values, the outliers at the same point (Apr 1st, 2:13 am) are missed. I have plotted both situations in a zoomable time series (using dygraphs) and also pasted the full data (in numeric format) below to reproduce the problem.

https://rpubs.com/sanjmeh/tsoutlier_works https://rpubs.com/sanjmeh/tsoutlier_fail

And here's the code to generate the two plots.

dy_format <- function(dy) dy %>% 
    dyOptions(drawPoints = T,pointSize = 2, strokeWidth = 2,fillGraph = T, strokePattern = c(1,2)) %>% 
    dyRangeSelector()

# successful detection
x2 |> head(1000) |> mutate(lg2=tsclean(lg)) |> dygraph() |> dy_format() |> dyAnnotation(x = asPosix('2022-04-01 02:13:58'),text = "Zoom here",width = 100,height = 20,attachAtBottom = T)

# missed detection
x2 |> head(3000) |> mutate(lg2=tsclean(lg)) |> dygraph() |> dy_format() |> dyAnnotation(x = asPosix('2022-04-01 02:13:58'),text = "Zoom here",width = 100,height = 20,attachAtBottom = T)

dput(x2$lg) c(77.9193799, 77.9170416, 77.9160433, 77.9124299, 77.91144, 77.9036733, 77.90266, 77.9017183, 77.9016733, 77.9016266, 77.90143, 77.9005983, 77.9005716, 77.9004983, 77.8971749, 77.8964766, 77.8958266, 77.8950799, 77.8946283, 77.8941066, 77.8935916, 77.8932016, 77.890315, 77.8898083, 77.8821533, 77.8815833, 77.8789533, 77.8783283, 77.8750383, 77.87416, 77.86766, 77.8667466, 77.863015, 77.8621083, 77.8603033, 77.8593816, 77.8546383, 77.8540833, 77.85398, 77.846285, 77.8454183, 77.8385766, 77.8377383, 77.8318733, 77.8311166, 77.8303516, 77.8299466, 77.8254566, 77.8246899, 77.8206783, 77.8198633, 77.8163633, 77.8154866, 77.8048316, 77.8039483, 77.7967416, 77.7965033, 77.7964933, 77.7847666, 77.7839633, 77.7779366, 77.7771166, 77.7763399, 77.7755166, 77.77476, 77.7743599, 77.7734183, 77.772475, 77.7695733, 77.7693099, 77.7690883, 77.7688566, 77.7635733, 77.7627233, 77.7603716, 77.759925, 77.756515, 77.75576, 77.7411783, 77.74049, 77.73573, 77.7352083, 77.734005, 77.7282383, 77.7265333, 77.72563, 77.71887, 77.7178716, 77.6856816, 77.6850216, 77.684235, 77.6823749, 77.6815316, 77.669785, 77.66903, 77.661705, 77.66095, 77.6559933, 77.6553699, 77.65276, 77.6520333, 77.644245, 77.643555, 77.6401616, 77.6395466, 77.6369633, 77.6368899, 77.63663, 77.6362899, 77.6356216, 77.634855, 77.6311116, 77.630995, 77.6307716, 77.6306166, 77.6305916, 77.6273316, 77.6269066, 77.6239083, 77.623345, 77.6229733, 77.6227466, 77.6227133, 77.6226966, 77.622675, 77.6226483, 77.6226216, 77.6225983, 77.6225733, 77.62253, 77.6225133, 77.6224916, 77.622485, 77.6224916, 77.6225116, 77.6225099, 77.6225066, 77.6225083, 77.62285, 77.62299, 77.62302, 77.622955, 77.6228216, 77.6225216, 77.619705, 77.619655, 77.6196233, 77.6194566, 77.6194016, 77.6193016, 77.619275, 77.619245, 77.61913, 77.61909, 77.6190683, 77.6186683, 77.6186433, 77.6185766, 77.618495, 77.61846, 77.6183966, 77.6182566, 77.618205, 77.6181249, 77.6181083, 77.61798, 77.6179583, 77.6179433, 77.617865, 77.6176599, 77.6172266, 77.6171933, 77.6171316, 77.6169616, 77.6133666, 77.6132966, 77.6132716, 77.61321, 77.6131949, 77.6131916, 77.6131733, 77.6130716, 77.6125883, 77.6124899, 77.612455, 77.6122533, 77.6120083, 77.6066633, 77.6061499, 77.6058033, 77.6056816, 77.6056816, 77.6054899, 77.6054416, 77.6054183, 77.6053816, 77.6053366, 77.605195, 77.6049383, 77.6045433, 77.6040233, 77.5935833, 77.5929383, 77.5859699, 77.58542, 77.5830366, 77.5825933, 77.5822816, 77.582055, 77.580975, 77.5809916, 77.5807249, 77.5808749, 77.5817616, 77.5818833, 77.583205, 77.58332, 77.5843516, 77.5845899, 77.5857433, 77.586075, 77.5865116, 77.5870366, 77.5876033, 77.588285, 77.5930466, 77.5937583, 77.6006466, 77.60135, 77.6019633, 77.6023866, 77.6026049, 77.6046583, 77.6048966, 77.606065, 77.6060833, 77.6060716, 77.6060883, 77.606095, 77.6061416, 77.6061566, 77.6059999, 77.6056849, 77.605, 77.6048166, 77.604525, 77.6044499, 77.6040933, 77.60385, 77.6034466, 77.602925, 77.6014783, 77.6010266, 77.600725, 77.6005083, 77.6004366, 77.6005, 77.6006783, 77.6011916, 77.6012433, 77.6012266, 77.6011316, 77.5999516, 77.5997516, 77.59962, 77.5996216, 77.5997233, 77.5995833, 77.59813, 77.5979716, 77.5978183, 77.5978433, 77.5980133, 77.59817, 77.5983683, 77.5983783, 77.59831, 77.598325, 77.5983799, 77.5983716, 77.5983783, 77.5986133, 77.598755, 77.60166, 77.60166, 77.6015216, 77.6012983, 77.600975, 77.6005533, 77.59904, 77.5987966, 77.5974333, 77.5971233, 77.5948333, 77.5947099, 77.5942116, 77.5939366, 77.591045, 77.5908133, 77.5901016, 77.589945, 77.5900516, 77.5901666, 77.5903433, 77.5903316, 77.5893766, 77.589135, 77.586505, 77.5864066, 77.5858616, 77.5857016, 77.585335, 77.5852883, 77.5847616, 77.5845866, 77.5836016, 77.5835966, 77.5837183, 77.5838883, 77.584165, 77.5846349, 77.5852966, 77.5904616, 77.5907583, 77.5909733, 77.5910683, 77.5962566, 77.7226666, 77.7227816, 77.7228466, 77.7228466, 77.7229149, 77.72306, 77.59805, 77.598135, 77.59808, 77.59745, 77.72827, 77.7287916, 77.729155, 77.5973366, 77.59702, 77.59711, 77.597405, 77.5977483, 77.5995249, 77.59984, 77.6010083, 77.6014933, 77.6031133, 77.6034333, 77.6092316, 77.6097766, 77.6151533, 77.6154583, 77.6155933, 77.6153966, 77.61556, 77.6207883, 77.6210816, 77.6224216, 77.6224566, 77.6224633, 77.62247, 77.62248, 77.6242783, 77.6244883, 77.6257683, 77.6260033, 77.6286499, 77.6288566, 77.6301699, 77.6302066, 77.63022, 77.6302499, 77.6316366, 77.6317133, 77.6320366, 77.6322383, 77.6328433, 77.6332366, 77.6342249, 77.6348083, 77.636135, 77.6368716, 77.63843, 77.6392599, 77.65762, 77.6584416, 77.659055, 77.6595566, 77.6599016, 77.66133, 77.6614683, 77.6614466, 77.6614116, 77.66063, 77.660675, 77.6613016, 77.6614883, 77.6640533, 77.6643366, 77.666725, 77.6669416, 77.66723, 77.6676666, 77.6694216, 77.6700683, 77.6707616, 77.6715766, 77.6723466, 77.673075, 77.6737233, 77.6742233, 77.675655, 77.6758866, 77.6765733, 77.67732, 77.6803683, 77.68107, 77.682945, 77.6835566, 77.688415, 77.6886583, 77.6887716, 77.6887966, 77.6888716, 77.6890116, 77.689305, 77.6897266, 77.69013, 77.69063, 77.6932716, 77.694045, 77.695585, 77.6964283, 77.7006616, 77.7012233, 77.7017499, 77.7028283, 77.70336, 77.7040283, 77.7047333, 77.7113216, 77.7119433, 77.7123916, 77.7127933, 77.7150066, 77.7155883, 77.7174449, 77.718155, 77.7202399, 77.7210433, 77.7214399, 77.7218649, 77.7222066, 77.72264, 77.7227833, 77.7227533, 77.7223733, 77.7334433, 77.7370633, 77.7376966, 77.7383966, 77.7392433, 77.7457433, 77.7466983, 77.7516133, 77.7525316, 77.7594716, 77.7603, 77.7658783, 77.7664383, 77.7668266, 77.767085, 77.7676333, 77.7678516, 77.7681699, 77.7687833, 77.7707883, 77.77139, 77.7743283, 77.7748066, 77.7780133, 77.7783816, 77.7789483, 77.7847816, 77.7853933, 77.7859549, 77.7864199, 77.7877549, 77.788355, 77.7890733, 77.7898316, 77.7929533, 77.7936266, 77.7968116, 77.7973433, 77.7978216, 77.8001666, 77.8005733, 77.8044216, 77.8049416, 77.807255, 77.8073183, 77.8080816, 77.8082883, 77.8088816, 77.8091916, 77.81004, 77.8106, 77.8128033, 77.8132216, 77.814205, 77.8147149, 77.8168133, 77.8173066, 77.8189333, 77.8195383, 77.8243066, 77.824955, 77.8270583, 77.8275199, 77.8297166, 77.8297166, 77.8297433, 77.8297683, 77.8299533, 77.8326216, 77.8331416, 77.833565, 77.833865, 77.8347233, 77.8351533, 77.8370116, 77.83772, 77.839195, 77.8397899, 77.8402633, 77.8406283, 77.840915, 77.8409833, 77.8410216, 77.8411049, 77.8419766, 77.8425266, 77.844205, 77.8443916, 77.8446666, 77.85073, 77.851425, 77.8521683, 77.862625, 77.8627316, 77.86286, 77.8629283, 77.8630233, 77.8664683, 77.8665149, 77.866905, 77.8670866, 77.8677683, 77.8728966, 77.8737933, 77.879505, 77.8798233, 77.880155, 77.8804183, 77.8814816, 77.8816616, 77.8824283, 77.8826516, 77.8829483, 77.8845433, 77.8848933, 77.8852933, 77.8854233, 77.8855466, 77.8858283, 77.8871316, 77.88722, 77.8874166, 77.8875516, 77.8898749, 77.8902283, 77.8906033, 77.8910616, 77.89322, 77.8937699, 77.8974716, 77.8977866, 77.8992866, 77.899475, 77.8998233, 77.9000649, 77.901015, 77.9010199, 77.9010633, 77.9016833, 77.9020283, 77.9028383, 77.90285, 77.90285, 77.90285, 77.90285, 77.90285, 77.9029483, 77.9029716, 77.9030133, 77.9053016, 77.9056666, 77.9060316, 77.9064933, 77.908035, 77.9084466, 77.9133816, 77.9138883, 77.9149133, 77.9155483, 77.9180333, 77.91864, 77.9204016, 77.9207499, 77.9209583, 77.9218716, 77.922135, 77.9255633, 77.9256866, 77.9261966, 77.9262833, 77.9267233, 77.9269, 77.928765, 77.9289833, 77.9291533, 77.9294166, 77.92976, 77.9340066, 77.9345133, 77.9462916, 77.9466466, 77.948555, 77.9488966, 77.9542933, 77.954695, 77.955, 77.9551166, 77.9550416, 77.954875, 77.9516833, 77.9514333, 77.9513333, 77.9513883, 77.9537149, 77.9540166, 77.95668, 77.9569133, 77.9574633, 77.9576766, 77.9698866, 77.9702216, 77.9718199, 77.9722766, 77.9728766, 77.9735633, 77.9743333, 77.9751633, 77.9785416, 77.9794033, 77.982405, 77.9831866, 77.9886783, 77.989345, 77.9899983, 77.9908, 77.990875, 77.9909749, 77.9971716, 77.9976516, 77.9990799, 77.9994083, 77.9995716, 78.0019366, 78.0024016, 78.00618, 78.0066366, 78.0113683, 78.0113966, 78.0114, 78.0114, 78.0114749, 78.0114749, 78.0114749, 78.011585, 78.0161333, 78.0166366, 78.0176233, 78.01805, 78.0198049, 78.0202533, 78.0220016, 78.0226866, 78.0246183, 78.02524, 78.0273216, 78.0277433, 78.02884, 78.0292033, 78.0310283, 78.0314516, 78.0337833, 78.03409, 78.03425, 78.0343649, 78.03434, 78.0341266, 78.0339149, 78.0337016, 78.0336233, 78.0335616, 78.0335566, 78.0335533, 78.033535, 78.0333266, 78.0329683, 78.0329483, 78.0329316, 78.0329183, 78.0326816, 78.0326033, 78.0323533, 78.031105, 78.0306616, 78.0260633, 78.0255583, 78.0242983, 78.0237849, 78.0178416, 78.017815, 78.0177733, 78.0177533, 78.0177233, 78.01712, 78.0169216, 78.0168383, 78.0164233, 78.01424, 78.0140183, 78.0136233, 78.0134466, 78.0131733, 78.0130066, 78.0129399, 78.0128133, 78.012775, 78.007015, 78.0063583, 78.0058083, 78.0054066, 78.0051316, 78.0050316, 78.0050833, 78.0053333, 78.0154283, 78.0157349, 78.0160066, 78.0162016, 78.0169033, 78.0169183, 78.0168883, 78.0168833, 78.016835, 78.0168533, 78.01906, 78.0190866, 78.0207, 78.0208033, 78.020975, 78.0209216, 78.02074, 78.0204183, 78.0199633, 78.01289, 78.0121616, 77.9975533, 77.99681, 77.9834933, 77.9827783, 77.9774983, 77.97743, 77.9773766, 77.9772983, 77.9772599, 77.9771866, 77.9722199, 77.9714516, 77.9593933, 77.9586066, 77.9494533, 77.9487, 77.9366316, 77.9359566, 77.9347049, 77.9341633, 77.93279, 77.9323533, 77.9197783, 77.9193483, 77.91214, 77.911715, 77.9085833, 77.9080683, 77.8999216, 77.8994216, 77.8882766, 77.8878966, 77.8870349, 77.8868566, 77.8862549, 77.8862133, 77.8864033, 77.8865733, 77.8876066, 77.8879016, 77.8905683, 77.8911083, 77.8928333, 77.8935433, 77.9047683, 77.9054599, 77.9076666, 77.908495, 77.9128233, 77.9137499, 77.9297583, 77.9305566, 77.9410633, 77.941955, 77.94291, 77.965415, 77.9661116, 77.9667033, 77.9834033, 77.9840183, 77.9845283, 77.98479, 77.9849466, 77.98495, 77.98472, 77.9833766, 77.9831566, 77.9827633, 77.9827033, 77.9827833, 77.98292, 77.9835133, 77.98383, 77.9844066, 77.9844533, 77.984325, 77.9841033, 77.9837616, 77.98327, 77.9824866, 77.9816216, 77.9691466, 77.96867, 77.9635583, 77.9630483, 77.961, 77.9606433, 77.9596433, 77.9595866, 77.95902, 77.9587833, 77.9578383, 77.95761, 77.9563333, 77.9560516, 77.9555633, 77.95547, 77.9552233, 77.9551166, 77.955105, 77.9551466, 77.9550583, 77.9549133, 77.9548016, 77.954835, 77.9543066, 77.9542, 77.9517183, 77.9515966, 77.9496433, 77.9495766, 77.9489166, 77.9488683, 77.9486333, 77.9484666, 77.946935, 77.94681, 77.9459233, 77.9456033, 77.9450533, 77.9408533, 77.93028, 77.9304233, 77.9305649, 77.93083, 77.9311983, 77.9317166, 77.9323783, 77.9337716, 77.9345, 77.9398383, 77.9403283, 77.9406816, 77.940915, 77.9409866, 77.9404499, 77.94034, 77.9394516, 77.9393633, 77.9382316, 77.9381866, 77.9381666, 77.938155, 77.93807, 77.9378683, 77.9378599, 77.9378516, 77.9378516, 77.9378516, 77.9378516, 77.9378516, 77.9379716, 77.9379699, 77.937625, 77.937625, 77.937625, 77.9375583, 77.9375583, 77.9375599, 77.9374583, 77.9373116, 77.936935, 77.9369183, 77.9363816, 77.9363216, 77.9363533, 77.9361716, 77.9356533, 77.9356183, 77.9358633, 77.9357983, 77.935305, 77.9351316, 77.9346866, 77.934675, 77.9342633, 77.9341883, 77.9338133, 77.9339133, 77.9341099, 77.9343983, 77.9400716, 77.9404499, 77.9432183, 77.9437216, 77.9499533, 77.9503966, 77.9521816, 77.9524516, 77.9529533, 77.9533516, 77.9538266, 77.9549533, 77.9552933, 77.9555049, 77.9556033, 77.9555816, 77.9553116, 77.9552733, 77.9554366, 77.955795, 77.9584666, 77.959, 77.9605616, 77.9610766, 77.9636233, 77.9639599, 77.9696533, 77.9699983, 77.9776633, 77.9780716, 77.9816716, 77.9821783, 77.98436, 77.98482, 77.9856216, 77.9862233, 77.9865283, 77.9882583, 77.98876, 77.9910833, 77.9917, 77.9989133, 77.9995133, 78.0024733, 78.0029733, 78.0083049, 78.0089066, 78.0095283, 78.0100949, 78.0136516, 78.0144516, 78.0152199, 78.015825, 78.0170333, 78.0175016, 78.0186583, 78.01915, 78.02026, 78.0209149, 78.0289766, 78.0296549, 78.04542, 78.0460449, 78.053285, 78.05396, 78.06405, 78.0647, 78.0696633, 78.0703933, 78.078105, 78.0787383, 78.0794616, 78.0797999, 78.0800383, 78.0838, 78.0843233, 78.0849833, 78.088365, 78.0885116, 78.0885699, 78.088605, 78.0887566, 78.0888316, 78.0889283, 78.090245, 78.09035, 78.0904033, 78.0904483, 78.0904833, 78.0909516, 78.090985, 78.0909983, 78.0910133, 78.0910749, 78.0936133, 78.0945649, 78.095495, 78.0962583, 78.0965116, 78.0971699, 78.0977833, 78.0983833, 78.102325, 78.103075, 78.1074999, 78.1081466, 78.1095183, 78.1102316, 78.1133116, 78.1139933, 78.1292333, 78.1299016, 78.1366949, 78.137255, 78.1375549, 78.1376733, 78.1381033, 78.138325, 78.1392133, 78.13921, 78.1392033, 78.1392033, 78.1392033, 78.1392033, 78.1392983, 78.1393383, 78.1428433, 78.1432, 78.1489016, 78.1492783, 78.153295, 78.1535833, 78.157895, 78.1582249, 78.1587433, 78.160175, 78.1606333, 78.16093, 78.1611016, 78.1611716, 78.1609833, 78.16109, 78.1626616, 78.162695, 78.1628133, 78.163585, 78.163835, 78.169345, 78.1696683, 78.1712166, 78.1716399, 78.1753583, 78.1756683, 78.1769466, 78.1771433, 78.1779399, 78.178, 78.1780266, 78.1782016, 78.1792816, 78.1794716, 78.1812133, 78.1814316, 78.1832416, 78.183415, 78.184165, 78.1844, 78.18479, 78.1849183, 78.1854966, 78.1857766, 78.1864566, 78.1866033, 78.1891566, 78.1894283, 78.1925583, 78.1927233, 78.194335, 78.1943716, 78.1943499, 78.1943749, 78.19441, 78.1944183, 78.1945533, 78.1947466, 78.1989783, 78.19918, 78.1994516, 78.1998966, 78.2003516, 78.2009383, 78.2018449, 78.2055783, 78.2064, 78.2069766, 78.20748, 78.2077983, 78.2080716, 78.2082016, 78.2081966, 78.2083066, 78.2087366, 78.2090083, 78.2114983, 78.2118533, 78.2132283, 78.2137433, 78.2209983, 78.2216666, 78.2251216, 78.2257966, 78.2311383, 78.2317566, 78.235695, 78.2363916, 78.2681516, 78.2688033, 78.2793333, 78.2799883, 78.2850966, 78.2856149, 78.289025, 78.2897633, 78.2911216, 78.2916366, 78.2918566, 78.2920316, 78.2920616, 78.291965, 78.2916366, 78.2907433, 78.2904833, 78.2904566, 78.2905416, 78.2907583, 78.2952783, 78.2957116, 78.2979233, 78.2982083, 78.2990399, 78.2994533, 78.30224, 78.3026766, 78.3039766, 78.304455, 78.3054949, 78.3057716, 78.3059816, 78.306045, 78.3061816, 78.3062999, 78.3069233, 78.3074116, 78.3165383, 78.3174333, 78.319215, 78.3198033, 78.3201866, 78.3221466, 78.3228216, 78.3276083, 78.328455, 78.32872, 78.3344466, 78.3349966, 78.3356483, 78.3363049, 78.3389133, 78.3391966, 78.3396316, 78.3425366, 78.3430916, 78.34529, 78.34589, 78.3500349, 78.3507833, 78.3525266, 78.3532316, 78.355345, 78.3561883, 78.3602783, 78.3606533, 78.361175, 78.3627616, 78.3633599, 78.364245, 78.3676783, 78.3684466, 78.3717366, 78.3720249, 78.3722383, 78.3722733, 78.3723333, 78.37235, 78.3724133, 78.3738866, 78.374595, 78.3750733, 78.3751866, 78.3751866, 78.3753116, 78.375325, 78.3753983, 78.37581, 78.376515, 78.3766816, 78.3768466, 78.3769133, 78.3769133, 78.376825, 78.3765866, 78.37585, 78.3755866, 78.3754316, 78.3753916, 78.375525, 78.3757066, 78.3774033, 78.37782, 78.3814266, 78.3817183, 78.38238, 78.382555, 78.3827099, 78.3826433, 78.3822633, 78.382, 78.3766016, 78.3764466, 78.3754633, 78.3752116, 78.3748566, 78.37441, 78.36082, 78.3603649, 78.353925, 78.3535383, 78.349115, 78.348845, 78.3486816, 78.3486466, 78.34878, 78.3489733, 78.3513083, 78.3514783, 78.3517533, 78.3517433, 78.3515116, 78.3511216, 78.35066, 78.3501749, 78.3494916, 78.34868, 78.3397966, 78.3388516, 78.3273033, 78.3263433, 78.3224166, 78.3214783, 78.3154316, 78.3144183, 78.3062583, 78.3052683, 78.301215, 78.3001766, 78.29402, 78.2929966, 78.29096, 78.2900299, 78.271105, 78.2702566, 78.2686499, 78.2678983, 78.266565, 78.266005, 78.2654683, 78.2649599, 78.2618633, 78.26145, 78.2506983, 78.2504116, 78.249555, 78.2493699, 78.24548, 78.2453416, 78.2443066, 78.24415, 78.2423516, 78.2420933, 78.2394083, 78.2390566, 78.2348083, 78.2345683, 78.2343099, 78.2342583, 78.2344166, 78.2346466, 78.2352633, 78.2357233, 78.2362333, 78.2367233, 78.237895, 78.2385766, 78.2392816, 78.2400433, 78.2418366, 78.2427533, 78.2446016, 78.2455733, 78.2674233, 78.2683416, 78.2789133, 78.2799216, 78.2987783, 78.2997016, 78.3006266, 78.30145, 78.3084399, 78.3094216, 78.3103866, 78.3113333, 78.31233, 78.313335, 78.3192816, 78.3203549, 78.32138, 78.3223699, 78.3318316, 78.3327683, 78.3347933, 78.3357399, 78.3550183, 78.3560466, 78.3680033, 78.36903, 78.3741116, 78.3750516, 78.3760216, 78.3769416, 78.3778, 78.3785533, 78.3792066, 78.37982, 78.3857183, 78.3862783, 78.3882683, 78.3888916, 78.3902733, 78.3910383, 78.4048699, 78.4057133, 78.4099183, 78.410825, 78.4127016, 78.4128716, 78.428665, 78.4296166, 78.4396099, 78.4405916, 78.4425249, 78.443545, 78.445675, 78.446725, 78.4722433, 78.4732833, 78.4900466, 78.4908616, 78.49105, 78.491195, 78.4913133, 78.491395, 78.49145, 78.4914433, 78.491395, 78.4913016, 78.4911616, 78.4910016, 78.4909083, 78.4908166, 78.490645, 78.4905033, 78.4904433, 78.4903399, 78.4900433, 78.4900016, 78.490015, 78.4900716, 78.4901616, 78.4904416, 78.4908983, 78.490945, 78.490965, 78.4909983, 78.49107, 78.4913533, 78.49181, 78.4919349, 78.4920666, 78.4921299, 78.49225, 78.4923533, 78.4924316, 78.492475, 78.4925133, 78.4925316, 78.4924716, 78.4922516, 78.4919283, 78.4876649, 78.4872149, 78.4863899, 78.4861233, 78.4858416, 78.4858316, 78.4858249, 78.4858733, 78.4859433, 78.4861883, 78.4861883, 78.48616, 78.48571, 78.4855666, 78.4854983, 78.4854783, 78.48523, 78.48485, 78.48481, 78.4848083, 78.4848333, 78.484835, 78.4848333, 78.484835, 78.4848433, 78.4848533, 78.4848516, 78.4848583, 78.48486, 78.48486, 78.4848616, 78.4848666, 78.4848633, 78.4848616, 78.4849233, 78.4849266, 78.484945, 78.48497, 78.4850366, 78.485045, 78.48507, 78.4851, 78.4851, 78.4851466, 78.4851516, 78.485255, 78.4852333, 78.4851666, 78.4851816, 78.485085, 78.4852116, 78.4860066, 78.4863216, 78.4876399, 78.4879116, 78.4882833, 78.488945, 78.4905983, 78.4909933, 78.4913816, 78.49157, 78.49226, 78.49247, 78.4924799, 78.492375, 78.49218, 78.490165, 78.4897183, 78.4889216, 78.4886883, 78.48738, 78.4871683, 78.4868, 78.48302, 78.4825133, 78.4815566, 78.4814899, 78.4814566, 78.4812583, 78.4812016, 78.4812933, 78.4817783, 78.4817366, 78.4812699, 78.48099, 78.4801483, 78.4798599, 78.4783283, 78.4780516, 78.4767449, 78.476585, 78.4764016, 78.4761916, 78.4756233, 78.475305, 78.4741583, 78.4740916, 78.47389, 78.4737316, 78.47296, 78.4729483, 78.4740416, 78.4742249, 78.4755749, 78.4756583, 78.4756866, 78.4755733, 78.4737249, 78.473375, 78.472265, 78.47211, 78.4717016, 78.4714233, 78.4711033, 78.4709916, 78.4705016, 78.47056, 78.4707699, 78.4708249, 78.4708383, 78.4709533, 78.4711066, 78.471285, 78.4713683, 78.4714116, 78.47147, 78.4715066, 78.4715383, 78.47169, 78.4718499, 78.47216, 78.4721583, 78.4717566, 78.471745, 78.4717033, 78.47165, 78.4716266, 78.4713833, 78.4712816, 78.4714166, 78.4715366, 78.4719516, 78.4718633, 78.47175, 78.4713733, 78.4703799, 78.4700183, 78.4698483, 78.4687366, 78.4686233, 78.4682733, 78.4681116, 78.4673633, 78.4672783, 78.4663916, 78.46642, 78.4672083, 78.4674499, 78.4695933, 78.4700816, 78.4704466, 78.4707799, 78.47165, 78.4718666, 78.4720783, 78.47214, 78.47206, 78.4717416, 78.4713516, 78.470045, 78.4698183, 78.468695, 78.4685883, 78.4685666, 78.4686583, 78.4686966, 78.468615, 78.46583, 78.4656566, 78.4657049, 78.4655616, 78.464905, 78.464565, 78.4611283, 78.460925, 78.4602916, 78.4599266, 78.4590649, 78.4588733, 78.458225, 78.45818, 78.45826, 78.4581983, 78.4586183, 78.4587983, 78.4590883, 78.4595666, 78.460095, 78.4609033, 78.4669616, 78.4679066, 78.4757133, 78.4765949, 78.4774699, 78.4782066, 78.478805, 78.479395, 78.4797866, 78.4799966, 78.4800766, 78.4799983, 78.4797399, 78.479435, 78.4788733, 78.4782916, 78.46859, 78.4679533, 78.4647133, 78.4641083, 78.4603016, 78.4597333, 78.4594783, 78.4560149, 78.4554933, 78.4535966, 78.453285, 78.4525783, 78.4524616, 78.4519133, 78.4516783, 78.4513716, 78.45096, 78.450465, 78.449845, 78.4453766, 78.4447566, 78.4425766, 78.441965, 78.4405, 78.439715, 78.43802, 78.4375416, 78.4341183, 78.43364, 78.4301183, 78.4298266, 78.4296116, 78.429525, 78.4295966, 78.4293916, 78.4275166, 78.427355, 78.4273716, 78.4274833, 78.4281516, 78.4281416, 78.4280666, 78.4281866, 78.4285883, 78.4288716, 78.43213, 78.4324, 78.43279, 78.4331116, 78.434395, 78.4349966, 78.43904, 78.439525, 78.4398916, 78.4401316, 78.4402583, 78.4402833, 78.4401433, 78.4398333, 78.439345, 78.4379033, 78.4373233, 78.436625, 78.4358883, 78.4349933, 78.4340666, 78.4256083, 78.4246666, 78.4218816, 78.4210133, 78.4201966, 78.4194616, 78.4188416, 78.418355, 78.4179533, 78.4176783, 78.4174949, 78.4174666, 78.4175683, 78.4177833, 78.4181666, 78.4186599, 78.4205516, 78.4209283, 78.42158, 78.42184, 78.4222866, 78.4223033, 78.4219933, 78.4218616, 78.4217466, 78.4215483, 78.42106, 78.4206533, 78.4184566, 78.4178716, 78.4129783, 78.4125966, 78.4097833, 78.4093649, 78.40646, 78.4060166, 78.4021633, 78.4016766, 78.3963816, 78.3959833, 78.3950816, 78.394865, 78.3945849, 78.3945166, 78.394685, 78.39484, 78.3956433, 78.39603, 78.3979866, 78.3984999, 78.3995499, 78.3996983, 78.3997933, 78.3997633, 78.3995583, 78.3929533, 78.3925783, 78.3911816, 78.390725, 78.3875233, 78.3869083, 78.3828833, 78.382445, 78.382025, 78.3817033, 78.3803816, 78.3802599, 78.3802983, 78.3822066, 78.3820966, 78.3818866, 78.3814566, 78.38085, 78.370785, 78.3701366, 78.3688516, 78.368145, 78.3643116, 78.36356, 78.3604583, 78.3598066, 78.3592216, 78.3587616, 78.3584033, 78.3581066, 78.35785, 78.3577766, 78.3566966, 78.35675, 78.3584899, 78.3585783, 78.3611233, 78.361145, 78.3610349, 78.3608583, 78.3602816, 78.3598583, 78.3588816, 78.3582766, 78.3569083, 78.3561333, 78.3544199, 78.3535466, 78.3525783, 78.3516616, 78.34971, 78.3487516, 78.31392, 78.3129383, 78.3119866, 78.311185, 78.3104583, 78.3071133, 78.3065866, 78.3061433, 78.3058283, 78.3040566, 78.3036616, 78.3016966, 78.3014316, 78.2997216, 78.2995933, 78.2993833, 78.2990616, 78.2973683, 78.296825, 78.294165, 78.2935383, 78.2915716, 78.2908116, 78.2869333, 78.2861966, 78.2846733, 78.2838433, 78.2793333, 78.2785566, 78.2762716, 78.2754183, 78.2709583, 78.2700266, 78.2671516, 78.266245, 78.2654533, 78.2647933, 78.2641866, 78.2637883, 78.2610633, 78.2607416, 78.258025, 78.2576566, 78.2571716, 78.256705, 78.2561233, 78.255745, 78.2549066, 78.252865, 78.2518933, 78.2511566, 78.250485, 78.2465083, 78.2459483, 78.2454549, 78.242055, 78.2419883, 78.241945, 78.24188, 78.2412733, 78.2411583, 78.24109, 78.2410433, 78.2410433, 78.2410433, 78.2408966, 78.2408583, 78.240495, 78.2399183, 78.2387816, 78.2381716, 78.234895, 78.2343616, 78.2337583, 78.2334399, 78.2331416, 78.2329549, 78.231845, 78.23146, 78.224065, 78.2237116, 78.22232, 78.22216, 78.22218, 78.2222199, 78.2221916, 78.222055, 78.2217366, 78.2213666, 78.2187616, 78.21846, 78.2182866, 78.2181883, 78.2181966, 78.2183833, 78.2196283, 78.2197916, 78.219735, 78.2195116, 78.2191466, 78.2179833, 78.21739, 78.2155483, 78.2150933, 78.2147249, 78.2145583, 78.2144866, 78.2145633, 78.2147749, 78.2151366, 78.215605, 78.2161733, 78.2168416, 78.2197983, 78.2206366, 78.2223983, 78.223375, 78.22531, 78.2262833, 78.2303683, 78.2311299, 78.2316933, 78.2322216, 78.2327116, 78.2330633, 78.23323, 78.23332, 78.2332583, 78.233415, 78.2337366, 78.2341333, 78.2346783, 78.2353216, 78.2371816, 78.2377316, 78.2390616, 78.239425, 78.2423716, 78.2426233, 78.2430433, 78.243165, 78.2430016, 78.2428616, 78.2425933, 78.2422416, 78.241775, 78.2412333, 78.2371816, 78.2366283, 78.236175, 78.23581, 78.2349283, 78.2343616, 78.2275283, 78.2270683, 78.2230966, 78.2224233, 78.2217449, 78.2212683, 78.2157199, 78.2152133, 78.21184, 78.2112049, 78.209175, 78.20859, 78.206775, 78.2064349, 78.2053316, 78.2051716, 78.205125, 78.2051216, 78.2052683, 78.2054966, 78.2094166, 78.209705, 78.2124016, 78.2127683, 78.2139933, 78.214495, 78.22002, 78.220575, 78.2282533, 78.2288, 78.2312566, 78.2317183, 78.2328416, 78.2335033, 78.2368733, 78.2376466, 78.2408383, 78.2415766, 78.2451416, 78.2459733, 78.2545983, 78.2554116, 78.2597666, 78.2604133, 78.2609516, 78.2613366, 78.2615516, 78.2616949, 78.2619083, 78.2645033, 78.2649, 78.2655116, 78.2661799, 78.2669849, 78.2678416, 78.2708366, 78.27157, 78.2721649, 78.2726716, 78.27304, 78.2731733, 78.273585, 78.2735966, 78.27358, 78.2735783, 78.2735783, 78.27396, 78.2741266, 78.27506, 78.275115, 78.2747116, 78.2749333, 78.2760433, 78.2764016, 78.2795066, 78.2799933, 78.28554, 78.2861366, 78.2889266, 78.28954, 78.2902983, 78.2911383, 78.2920833, 78.3011449, 78.3021233, 78.306715, 78.3077066, 78.3144266, 78.3152666, 78.316945, 78.3177383, 78.3190083, 78.3195633, 78.320055, 78.3204716, 78.3210083, 78.3211466, 78.3211783, 78.3210699, 78.3208966, 78.3206233, 78.3202483, 78.3198199, 78.3136116, 78.3133533, 78.3132549, 78.313265, 78.3134133, 78.316865, 78.31716, 78.3181116, 78.3184966, 78.3226733, 78.3230583, 78.32326, 78.3238383, 78.3237966, 78.3237066, 78.3237816, 78.3240149, 78.3243266, 78.3315733, 78.3316133, 78.3316116, 78.3316, 78.33158, 78.3315116, 78.3314916, 78.3314899, 78.3314899, 78.3314433, 78.3314533, 78.3314533, 78.3314533, 78.3314533, 78.3293933, 78.3315183, 78.3312683, 78.3312683, 78.3315316, 78.3315399, 78.3315683, 78.3316599, 78.3317133, 78.3318116, 78.3319066, 78.3319483, 78.3319916, 78.3356416, 78.3362016, 78.337375, 78.33793, 78.3387866, 78.3395933, 78.3425483, 78.3430849, 78.3435516, 78.3475733, 78.348105, 78.3516066, 78.3522166, 78.3592866, 78.3599766, 78.363295, 78.3638566, 78.3647066, 78.3650266, 78.3659783, 78.3660683, 78.3663366, 78.3663183, 78.3661883, 78.3658966, 78.3654566, 78.3648616, 78.3619266, 78.3612, 78.3604483, 78.3597116, 78.3593833, 78.35913, 78.3589916, 78.359055, 78.3592316, 78.363055, 78.3634966, 78.3643566, 78.364855, 78.3664266, 78.3668066, 78.3682933, 78.36845, 78.368855, 78.36912, 78.3704283, 78.3707533, 78.3728999, 78.3731516, 78.373865, 78.3739833, 78.3751, 78.3752366, 78.3755283, 78.37572, 78.3768283, 78.3769033, 78.376735, 78.3768433, 78.3770633, 78.3775383, 78.3800766, 78.3805366, 78.38072, 78.380775, 78.3796699, 78.3795316, 78.3792699, 78.3787766, 78.37835, 78.3774983, 78.3697466, 78.3689866, 78.3683866, 78.3679049, 78.3675283, 78.3671966, 78.3640316, 78.3636433, 78.3625049, 78.3624583, 78.3622383, 78.36205, 78.3616599, 78.3612916, 78.3597683, 78.3593983, 78.3587266, 78.3581799, 78.3540833, 78.3538349, 78.3536516, 78.3538466, 78.35374, 78.3528483, 78.35288, 78.3529966, 78.3532916, 78.3536683, 78.3541466, 78.3547716, 78.3554866, 78.3562949, 78.3571633, 78.3694183, 78.3702033, 78.3717566, 78.37243, 78.3735916, 78.3740966, 78.3746533, 78.3750966, 78.375625, 78.375795, 78.37595, 78.3758983, 78.37563, 78.3753949, 78.3749216, 78.3747883, 78.3748933, 78.37509, 78.375465, 78.3760183, 78.3862183, 78.386665, 78.3886183, 78.3889216, 78.3934283, 78.3939099, 78.3960083, 78.3966783, 78.3974066, 78.3981333, 78.4006183, 78.4013783, 78.4019266, 78.4023649, 78.4030216, 78.4119899, 78.4128133, 78.4145116, 78.41541, 78.4268866, 78.427785, 78.4285566, 78.429295, 78.4312266, 78.4319683, 78.4328233, 78.4337083, 78.4346266, 78.43569, 78.4377083, 78.438675, 78.4426033, 78.4432333, 78.446665, 78.4476283, 78.4486016, 78.453465, 78.4544266, 78.4580016, 78.4588883, 78.4678916, 78.46867, 78.4693549, 78.4699183, 78.47036, 78.4706983, 78.4709483, 78.4710133, 78.4709683, 78.4707733, 78.4704233, 78.4700099, 78.4694733, 78.4688883, 78.46356, 78.4629066, 78.4618366, 78.4614066, 78.46108, 78.4608383, 78.4598433, 78.4596433, 78.4587733, 78.4582266, 78.4575383, 78.45677, 78.4559099, 78.4549883, 78.454065, 78.45303, 78.4519999, 78.4509683, 78.4500566, 78.4491766, 78.448325, 78.4475316, 78.4468516, 78.4462733, 78.4384, 78.4377316, 78.4368683, 78.4360149, 78.4350433, 78.4340966, 78.4221649, 78.4210799, 78.420025, 78.4190633, 78.4181349, 78.4173016, 78.415035, 78.4145133, 78.41412, 78.4139633, 78.4135033, 78.4133666, 78.4134716, 78.4133966, 78.4132233, 78.4128349, 78.4123383, 78.4116716, 78.4110416, 78.41025, 78.4093583, 78.40656, 78.405875, 78.4052316, 78.4047583, 78.4045333, 78.40432, 78.404365, 78.4045316, 78.404895, 78.40548, 78.406, 78.4085, 78.4091716, 78.4097616, 78.4103833, 78.4108966, 78.4157483, 78.4167683, 78.421425, 78.4220083, 78.42278, 78.424905, 78.4257533, 78.4265083, 78.4270633, 78.4275383, 78.4280116, 78.42826, 78.4284316, 78.42854, 78.42849, 78.4272899, 78.4269166, 78.4252533, 78.42462, 78.4241483, 78.4237783, 78.4236316, 78.4235716, 78.4237783, 78.4237066, 78.4234449, 78.4210483, 78.4205366, 78.4201466, 78.4194583, 78.4175816, 78.41667, 78.4147966, 78.4142116, 78.4096, 78.4093349, 78.4092749, 78.4092583, 78.4091016, 78.4064333, 78.4061283, 78.4058083, 78.4055583, 78.4053766, 78.404535, 78.4045433, 78.4076983, 78.4079216, 78.4136416, 78.4141233, 78.4178033, 78.4184166, 78.4202266, 78.4207416, 78.4239, 78.4242416, 78.4243933, 78.4244716, 78.4250516, 78.4252466, 78.42558, 78.4260716, 78.4318266, 78.4324199, 78.4342166, 78.4343333, 78.4343899, 78.4339616, 78.4339633, 78.4339599, 78.4339399, 78.4338833, 78.43385, 78.4339816, 78.4341949, 78.4430133, 78.4433183, 78.4507516, 78.4511416, 78.4515383, 78.4520766, 78.4527483, 78.4534999, 78.4571083, 78.457715, 78.4582166, 78.458655, 78.4589866, 78.4592316, 78.4595116, 78.4595316, 78.459755, 78.45993, 78.4608383, 78.4612316, 78.4617066, 78.4622566, 78.4709199, 78.4714866, 78.4765949, 78.4771483, 78.4788966, 78.47953, 78.4864816, 78.4870216, 78.48866, 78.489305, 78.4928999, 78.493535, 78.4963, 78.4968566, 78.5001216, 78.5003283, 78.5019216, 78.5022066, 78.5041633, 78.5045983, 78.50761, 78.5079883, 78.5097666, 78.5100483, 78.5120383, 78.5124583, 78.51396, 78.5143649, 78.5146233, 78.5147549, 78.5147466, 78.5146516, 78.5123566, 78.5121366, 78.5114616, 78.511115, 78.5070216, 78.5065183, 78.50555, 78.50487, 78.5041416, 78.5033816, 78.5010799, 78.5005816, 78.50028, 78.5001083, 78.5000733, 78.500085, 78.5003783, 78.5003616, 78.5002049, 78.499845, 78.49929, 78.4987066, 78.4964016, 78.4958766, 78.4954783, 78.49516, 78.4946116, 78.4945, 78.4940483, 78.4937266, 78.49015, 78.490055, 78.4900083, 78.48998, 78.48998, 78.48998, 78.48998, 78.4899766, 78.4899033, 78.4886333, 78.488405, 78.4883683, 78.488475, 78.4894599, 78.489595, 78.4898049, 78.4900633, 78.49197, 78.492335, 78.4926616, 78.49328, 78.4936483, 78.494095, 78.4947066, 78.4954, 78.49834, 78.4990033, 78.49977, 78.5003366, 78.5008983, 78.5019666, 78.5026033, 78.50386, 78.5043133, 78.5060733, 78.5064033, 78.5065699, 78.5065983, 78.5064866, 78.5062633, 78.505915, 78.5055249, 78.5043749, 78.5040983, 78.50408, 78.5041333, 78.5072249, 78.5072166, 78.5070283, 78.506815, 78.5058216, 78.5055433, 78.5, 78.49986, 78.4997983, 78.4998966, 78.5000966, 78.5003966, 78.5007816, 78.501285, 78.501835, 78.5024666, 78.5032316, 78.504055, 78.5179933, 78.5188316, 78.5197866, 78.5207733, 78.5320283, 78.5329933, 78.5339166, 78.5347, 78.5354266, 78.5360366, 78.5376366, 78.5380216, 78.5383333, 78.5385833, 78.5394916, 78.5398283, 78.5434783, 78.5438633, 78.5468, 78.5470533, 78.5470983, 78.5470666, 78.5469933, 78.5466666, 78.5459966, 78.5458883, 78.5459166, 78.546045, 78.5462766, 78.5465916, 78.5470099, 78.54787, 78.554095, 78.5550733, 78.5568983, 78.557595, 78.5579866, 78.5584166, 78.5651816, 78.5655133, 78.5663933, 78.5665316, 78.5666533, 78.5665866, 78.5663566, 78.5661633, 78.5657233, 78.5657216, 78.5657633, 78.5657733, 78.5657733, 78.5657416, 78.56574, 78.5657283, 78.5657183, 78.5656516, 78.5656216, 78.5656233, 78.5656216, 78.5655483, 78.5655466, 78.5655466, 78.5655683, 78.5655666, 78.5655649, 78.5655416, 78.565515, 78.56511, 78.5651366, 78.565635, 78.5657783, 78.5665216, 78.5667, 78.5670983, 78.5675166, 78.5680266, 78.5770649, 78.577765, 78.5781866, 78.5785416, 78.5787316, 78.5788633, 78.5788583, 78.5785933, 78.5731666, 78.5726916, 78.5704066, 78.5700116, 78.5698233, 78.5697283, 78.5697933, 78.5716366, 78.5716333, 78.5716233, 78.5715966, 78.5714833, 78.5713383, 78.5712266, 78.5711566, 78.5711883, 78.5712066, 78.5712133, 78.5716916, 78.5717166, 78.571705, 78.5716783, 78.5716783, 78.5716783, 78.5718133, 78.571865, 78.57189, 78.5719283, 78.5719466, 78.571975, 78.5719833, 78.5719849, 78.5719716, 78.5719366, 78.5717966, 78.5718366, 78.5718933, 78.5719666, 78.5719699, 78.5720549, 78.572125, 78.5721433, 78.572205, 78.57225, 78.5737866, 78.5737999, 78.5736916, 78.5695983, 78.5695349, 78.5694933, 78.5693466, 78.5690583, 78.5686899, 78.5681366, 78.5509233, 78.550225, 78.5494383, 78.5484783, 78.54385, 78.5430933, 78.5424416, 78.5420366, 78.5418016, 78.54136, 78.5412166, 78.5400616, 78.5398383, 78.538865, 78.538535, 78.5377416, 78.5377283, 78.5379216, 78.5384649, 78.5384283, 78.5380683, 78.5378349, 78.534655, 78.53415, 78.5321699, 78.5319, 78.5317733, 78.531805, 78.532, 78.5324016, 78.5332366, 78.5334083, 78.53344, 78.5333766, 78.5333016, 78.5334016, 78.5340449, 78.534025, 78.5338866, 78.5337466, 78.5330283, 78.5328933, 78.5329366, 78.533295, 78.53358, 78.5351333, 78.5354766, 78.5368016, 78.537025, 78.5379366, 78.5379483, 78.5378833, 78.5377133, 78.5367183, 78.5366533, 78.536705, 78.5365083, 78.5346083, 78.5345283, 78.5350716, 78.5351133, 78.5351983, 78.535375, 78.5356949, 78.5361233, 78.5384783, 78.5390766, 78.5395466, 78.5398066, 78.5399516, 78.5401966, 78.5401, 78.539765, 78.5397483, 78.53981, 78.5399733, 78.5402249, 78.54059, 78.541085, 78.5416183, 78.5423433, 78.5429966, 78.5458316, 78.5465383, 78.546965, 78.5475016, 78.5478549, 78.5480799, 78.5482066, 78.5487749, 78.5487566, 78.5483283, 78.548355, 78.5485033, 78.548325, 78.5467916, 78.5467833, 78.5467533, 78.546725, 78.5466999, 78.5466683, 78.5466149, 78.5464983, 78.5464133, 78.5464133, 78.5464133, 78.5464133, 78.5464133, 78.5464133, 78.5464133, 78.546185, 78.546185, 78.5463783, 78.546645, 78.5466416, 78.5456816, 78.5457249, 78.5463233, 78.546535, 78.5469066, 78.5474166, 78.549115, 78.5498049, 78.550655, 78.55145, 78.5556366, 78.556385, 78.5598566, 78.5602383, 78.5614316, 78.5620283, 78.5663066, 78.566735, 78.5698066, 78.570385, 78.5753683, 78.575985, 78.5772766, 78.57784, 78.5786216, 78.57916, 78.5951, 78.5961033, 78.5967916, 78.5975833, 78.6015216, 78.602185, 78.60554, 78.6056433, 78.6056583, 78.6056583, 78.6057866, 78.6057866, 78.605955, 78.6059683, 78.6059799, 78.60606, 78.6060933, 78.6144316, 78.6151533, 78.6220366, 78.6228416, 78.6245316, 78.625395, 78.6261116, 78.6266383, 78.6277083, 78.6283716, 78.6291966, 78.6301116, 78.636315, 78.637155, 78.641465, 78.6418983, 78.6421533, 78.6432399, 78.6437366, 78.6442333, 78.6445266, 78.64469, 78.6450466, 78.6452833, 78.6456216, 78.6460866, 78.64902, 78.6495966, 78.650995, 78.6515383, 78.6613199, 78.66211, 78.66296, 78.663205, 78.663355, 78.66347, 78.663605, 78.6637266, 78.66392, 78.66412, 78.6645683, 78.6649583, 78.6663883, 78.6668316, 78.6710983, 78.6715133, 78.6716666, 78.6725849, 78.6729633, 78.6733633, 78.6738616, 78.6825549, 78.6832733, 78.6837316, 78.6842216, 78.6879766, 78.6884533, 78.6988049, 78.6991916, 78.7095783, 78.7100183, 78.7123866, 78.7127616, 78.7179383, 78.718425, 78.71948, 78.7197966, 78.7199333, 78.719905, 78.7197833, 78.7182183, 78.717815, 78.7162166, 78.715985, 78.7147333, 78.7147699, 78.7149433, 78.7229366, 78.7232283, 78.7241349, 78.7241349, 78.7240666, 78.7237933, 78.7234533, 78.72286, 78.717715, 78.7171899, 78.7168633, 78.7166333, 78.7159166, 78.715735, 78.7158099, 78.7159716, 78.716355, 78.7163716, 78.721205, 78.7218766, 78.7225133, 78.7231066, 78.723415, 78.7247016, 78.7248366, 78.7275783, 78.7277099, 78.7283516, 78.7285316, 78.7288866, 78.7293666, 78.7343166, 78.7350816, 78.7354633, 78.7407533, 78.7410483, 78.741255, 78.7413783, 78.7437866, 78.7438149, 78.744045, 78.7442283, 78.744575, 78.744815, 78.7484466, 78.7490116, 78.754015, 78.7543716, 78.7547066, 78.7549816, 78.7552233, 78.7552466, 78.75868, 78.7594366, 78.7703499, 78.7711433, 78.7719849, 78.7725966, 78.773095, 78.7733866, 78.7736266, 78.7743266, 78.77453, 78.7760883, 78.7763016, 78.7773583, 78.777705, 78.7786816, 78.7789833, 78.77949, 78.783435, 78.7839749, 78.7841999, 78.7907316, 78.7909516, 78.792715, 78.7931216, 78.7997683, 78.8002433, 78.8007416, 78.80137, 78.8020533, 78.8082716, 78.8092183, 78.8095916, 78.8100966, 78.81044, 78.8111166, 78.8156683, 78.8165283, 78.8259166, 78.8266366, 78.8347216, 78.835745, 78.83669, 78.8375333, 78.8382716, 78.839045, 78.8396416, 78.840065, 78.8454116, 78.8459666, 78.8507616, 78.85151, 78.8599083, 78.860755, 78.880595, 78.88143, 78.8982483, 78.8989783, 78.915715, 78.9164699, 78.9196466, 78.9203683, 78.9209449, 78.9213733, 78.9220966, 78.9226066, 78.9278, 78.9280083, 78.9280083, 78.92957, 78.9297733, 78.9300883, 78.9301683, 78.9301683, 78.93035, 78.930395, 78.9308633, 78.9309099, 78.9309316, 78.9308816, 78.930775, 78.9260983, 78.9257766, 78.9256783, 78.9256149, 78.9270416, 78.9270383, 78.9268033, 78.9265216, 78.9264266, 78.9264583, 78.9263683, 78.9258466, 78.9258516, 78.9257333, 78.9256016, 78.9245966, 78.9243533, 78.9237133, 78.9237266, 78.9242633, 78.92433, 78.9257583, 78.9257816, 78.9263216, 78.9264933, 78.9268066, 78.932205, 78.93264, 78.9402533, 78.940735, 78.943315, 78.9437683, 78.9509949, 78.9511166, 78.9513399, 78.9575083, 78.9575633, 78.95844, 78.958495, 78.9596583, 78.9596133, 78.9596083, 78.9596066, 78.9596283, 78.9597533, 78.9604083, 78.96049, 78.9609683, 78.9611383, 78.9625833, 78.9626333, 78.96266, 78.9627016, 78.9638266, 78.9641766, 78.96564, 78.9659099, 78.9671183, 78.9678683, 78.972465, 78.97317, 78.9734016, 78.9782583, 78.9787083, 78.9815483, 78.981925, 78.9887533, 78.9891816, 78.992415, 78.9930066, 78.9943466, 78.9950749, 78.99663, 78.997285, 78.997905, 78.99837, 79.0016483, 79.0024133, 79.0031616, 79.003555, 79.0038199, 79.00481, 79.0050016, 79.0062266, 79.0066049, 79.0088066, 79.0092449, 79.0162883, 79.016695, 79.02305, 79.0230716, 79.023055, 79.0230283, 79.02301, 79.0229849, 79.0228766, 79.0228366, 79.0228233, 79.0228233, 79.0228233, 79.0228233, 79.0228966, 79.0230033, 79.0230799, 79.0230983, 79.0231183, 79.0231366, 79.0231533, 79.023285, 79.0233, 79.0233383, 79.0233516, 79.0233933, 79.02345, 79.02938, 79.02955, 79.02962, 79.0296549, 79.02969, 79.0296966, 79.0297183, 79.0307249, 79.0312116, 79.0422316, 79.0427266, 79.0464066, 79.0468516, 79.0476083, 79.04789, 79.04788, 79.0477716, 79.0475849, 79.0474083, 79.0453816, 79.0445366, 79.04425, 79.0440949, 79.0435933, 79.043545, 79.043525, 79.0435183, 79.0435166, 79.043525, 79.0435383, 79.0435583, 79.04358, 79.043605, 79.0579316, 79.0585266, 79.0590016, 79.067665, 79.0681583, 79.0684416, 79.0685033, 79.06853, 79.0685766, 79.0686049, 79.0687849, 79.0688149, 79.0688633, 79.0689566, 79.0698966, 79.0701116, 79.0733349, 79.07407, 79.0790516, 79.0798416, 79.0902499, 79.0910533, 79.0952616, 79.0961583, 79.1014983, 79.1023183, 79.1081299, 79.1090033, 79.1142649, 79.1151033, 79.1209016, 79.1217383, 79.1261933, 79.12713, 79.139645, 79.140475, 79.1432433, 79.1441383, 79.1514816, 79.1524333, 79.1746899, 79.1756433, 79.1765733, 79.1770016, 79.1777633, 79.1784, 79.1788216, 79.1791516, 79.1793166, 79.18108, 79.1811966, 79.182315, 79.182385, 79.18269, 79.1828633, 79.185075, 79.1852066, 79.1857583, 79.1857533, 79.18521, 79.18531, 79.18601, 79.1861933, 79.186485, 79.1864883, 79.1864316, 79.1863299, 79.1862699, 79.1861283, 79.185965, 79.1858716, 79.1856866, 79.1855116, 79.1837633, 79.1828366, 79.1826616, 79.1825883, 79.1825683, 79.1826, 79.1826483, 79.1826983, 79.1828233, 79.1829049, 79.1831283, 79.1900616, 79.1910183, 79.1918283, 79.2059999, 79.2069833, 79.21317, 79.2142199, 79.220255, 79.2212866, 79.22637, 79.22734, 79.2376766, 79.2386766, 79.2540883, 79.2551066, 79.2810233, 79.2819883, 79.2828966, 79.2829933, 79.2830433, 79.28309, 79.2831333, 79.2831616, 79.2831766, 79.282875, 79.2828366, 79.2826883, 79.2826683, 79.27503, 79.2739683, 79.268415, 79.2683916, 79.2683916, 79.2683916, 79.2683916, 79.2683916, 79.2683916, 79.2680316, 79.26785, 79.2669283, 79.2669433, 79.2671249, 79.26719, 79.26736, 79.2673749, 79.2673749, 79.2673749, 79.2673749, 79.2677566, 79.267835, 79.2679033, 79.26794, 79.268045, 79.2680949, 79.268175, 79.2691083, 79.2691749, 79.2692433, 79.2692816, 79.2693016, 79.2693216, 79.26934, 79.26941, 79.2694116, 79.2692983, 79.2692933, 79.2692966, 79.2693216, 79.2693383, 79.2693583, 79.2693866, 79.2695216, 79.2696333, 79.2701033, 79.2701033, 79.2703033, 79.270385, 79.270425, 79.2708566, 79.2714733, 79.2715266, 79.2715266, 79.2717149, 79.2717149, 79.2717149, 79.271485, 79.2714566, 79.2714283, 79.2714033, 79.2713783, 79.2713783, 79.2713783, 79.2713783, 79.2713783, 79.2713783, 79.2713783, 79.2713783, 79.2713783, 79.27128, 79.2712983, 79.2713416, 79.271405, 79.2713883, 79.2714016, 79.2714266, 79.2714266, 79.2714266, 79.2714266, 79.2714266, 79.2714266)

I am not pasting the full data.table with time column to avoid cluttering. But I can send it across if you need to do further investigation. Thanks for the wonderful package @robjhyndman

Best Sanjay

sanjmeh commented 2 years ago

I am adding another chart, showing doubtful results of tsclean. Screen Shot 2022-05-26 at 11 10 25 AM

robjhyndman commented 2 years ago

With automatic procedures, there are always series for which they won't work. In this case, the automatic trend estimate on the longer series is not particularly good, and so the threshold for outlier detection is quite high, and it misses them. On the other hand, the application to the shorter series is picking up things that are not outliers.

In this case, you might be better off looking for outliers in the differenced series. e.g., this finds the 10 most outlying points in the differences:

outliers <- x2$lg |> diff() |> abs() |> order() |> tail(10)
plot(difflg)
points(outliers,difflg[outliers],col='red',pch=19)
sanjmeh commented 2 years ago

Thanks but actually the difflg works well for accelerating vehicles (as lat/lon changes gradually) but not for brakings. When a vehicle stops the diff value drops to 0 (previous lat/lon to new lat/lon diff is 0) and this happens quiclkly and several times over the timeseries so this is not an outlier. I would still think the tsoutlier algo must be able to see what a human can instantly spot on a time series chart. The spikes.

If you see this one time series plot you will see what I am talking.

Please zoom into the corners where green dots are exposed (green is original data & red is after running tsclean). The tsoutliers (red) dot trend line is actually missing the vehicle's turns at many places. Seems like tsoutliers is working like a smoothening algorithm instead of an outlier detection algo. BTW could you share the algo you use intsclean() or tsoutliers()

Many thanks

robjhyndman commented 2 years ago

The code is open source, so you can see exactly what it does. It is also described at https://robjhyndman.com/hyndsight/tsoutliers/.

If the algorithm is not doing what you want, don't use it.

sanjmeh commented 2 years ago

Thanks, I will look into it and decide how to move forward.