search

christophsax / seasonal

R interface to X-13ARIMA-SEATS
www.seasonal.website
120 stars 27 forks source link

Errors with long time series #313

Open StaffanBetner opened 9 months ago

StaffanBetner commented 9 months ago

Reprex:

library(seasonal)

structure(c(9542, 8881, 10298, 9261, 9201, 8644, 8439, 8546, 
            10305, 9846, 9008, 9094, 9968, 9319, 10087, 9309, 9615, 8347, 
            8143, 7943, 9571, 9072, 8366, 8565, 8751, 8714, 9706, 9011, 9147, 
            9025, 9035, 8617, 10553, 10352, 8831, 9665, 9827, 9268, 10461, 
            9773, 10008, 9766, 9734, 9792, 10989, 10631, 9898, 9960, 10477, 
            9427, 10179, 9331, 9290, 8953, 8999, 8990, 10611, 9821, 9262, 
            9732, 10269, 9343, 10055, 9804, 9378, 9176, 8947, 8931, 10020, 
            9721, 9596, 9842, 9888, 8646, 10317, 10116, 10458, 9259, 9120, 
            9014, 10978, 10324, 10348, 10881, 11209, 10248, 11739, 10785, 
            10632, 9967, 10200, 9988, 11447, 11190, 10683, 10951, 11362, 
            10619, 11602, 10796, 10965, 10603, 10480, 10288, 11776, 11263, 
            10457, 11394, 11612, 11546, 12109, 10967, 11090, 10046, 10165, 
            10222, 12067, 11455, 10583, 11300, 11536, 10624, 11545, 10965, 
            10787, 9827, 9570, 9470, 10716, 10433, 10200, 10961, 11178, 10149, 
            11761, 11037, 11090, 10672, 10798, 10171, 11873, 11067, 10467, 
            11321, 11689, 10728, 12682, 11246, 11687, 10645, 10204, 10044, 
            11779, 11049, 10581, 11945, 12651, 11496, 12627, 11435, 11668, 
            10374, 10655, 10210, 11804, 11109, 10528, 11447, 11837, 10982, 
            11966, 11490, 11273, 10324, 10402, 10115, 11891, 11671, 10434, 
            11815, 12962, 11863, 12850, 11639, 11034, 10449, 10437, 10317, 
            12081, 11426, 10752, 11179, 12070, 10277, 11868, 10782, 10992, 
            10162, 10232, 9766, 10972, 10512, 10367, 10832, 11097, 9698, 
            10756, 9998, 10489, 9127, 8667, 8211, 9387, 9154, 8754, 9609, 
            9979, 9508, 10672, 9335, 9449, 9015, 9540, 9471, 11020, 10070, 
            9472, 10146, 10570, 9787, 10374, 10165, 9989, 9642, 9799, 9216, 
            10590, 10209, 9116, 10381, 11034, 10016, 11368, 10984, 10745, 
            10279, 10252, 9842, 11031, 10467, 9988, 11327, 11326, 10515, 
            11541, 10749, 10872, 10103, 9950, 9473, 10971, 10500, 9923, 11060, 
            11279, 10974, 11888, 11064, 10810, 10249, 10360, 10222, 11706, 
            11230, 10598, 11263, 11664, 10474, 11965, 11377, 11704, 10614, 
            10289, 10205, 12019, 11306, 10423, 11209, 11933, 10620, 12158, 
            11637, 11550, 10972, 11407, 10712, 11949, 11106, 10443, 11471, 
            11816, 11165, 12091, 11612, 11738, 11104, 10764, 10457, 11711, 
            11113, 10692, 11627, 12465, 10930, 12683, 11643, 11624, 10458, 
            10903, 10953, 12003, 11864, 11067, 11883, 12013, 10773, 12125, 
            11691, 11538, 10349, 10423, 10389, 11633, 11185, 10772, 11573, 
            12086, 10865, 12148, 11510, 11803, 11099, 11301, 11173, 12207, 
            11894, 11115, 11842, 11938, 11755, 12215, 11892, 11473, 10406, 
            10478, 10152, 11546, 11142, 9822, 11443, 11732, 10104, 11893, 
            11158, 11301, 10484, 10707, 10483, 11961, 11245, 10477, 11259, 
            12212, 11340, 12569, 12060, 11599, 9976, 9851, 10116, 11375, 
            11360, 10495, 11347, 11881, 10460, 12116, 11044, 10710, 10482, 
            10529, 10363, 11896, 11318, 10554, 11522, 11852, 11392, 12618, 
            11788, 11832, 10839, 11050, 10918, 12274, 11526, 10999, 11657, 
            11917, 10408, 12061, 11525, 11736, 10821, 10791, 10847, 12037, 
            11950, 11312, 11903, 12599, 11306, 12733, 11894, 11856, 10704, 
            10904, 11074, 12354, 11740, 11032, 11686, 12438, 10867, 12816, 
            11908, 11606, 11019, 11102, 11065, 12125, 11917, 11114, 12192, 
            12322, 11448, 12257, 11265, 11211, 10694, 10916, 10886, 12115, 
            11296, 10511, 11530, 11889, 10807, 11864, 11256, 11195, 10241, 
            9993, 10368, 12005, 11021, 10191, 11239, 11981, 10393, 12138, 
            10582, 11097, 10772, 11044, 11133, 11302, 10220, 10595, 12340, 
            12821, 11116, 12445, 11327, 11345, 10484, 10617, 10530, 11718, 
            11098, 10655, 11360, 11850, 10550, 11632, 10839, 10791, 10467, 
            10644, 10631, 11388, 10291, 9670, 10869, 11849, 10523, 11365, 
            11063, 11205, 10584, 10731, 10394, 11764, 10740, 10012, 11499, 
            11605, 10260, 11639, 11212, 10889, 10652, 10592, 10477, 11289, 
            10474, 10201, 12119, 12364, 10764, 12097, 11008, 11055, 10783, 
            10679, 10912, 12058, 10858, 10409, 11612, 11834, 11114, 12143, 
            10900, 10786, 10461, 10679, 10774, 12193, 11158, 10725, 11541, 
            11521, 10614, 11927, 11103, 10724, 10535, 10793, 10997, 11953, 
            11011, 10683, 11138, 12170, 10718, 12317, 11395, 11246, 11019, 
            11108, 10820, 12157, 11409, 10669, 11495, 11634, 10533, 12037, 
            11142, 11140, 10700, 11078, 10740, 12060, 11116, 10251, 11451, 
            11793, 11064, 12805, 12000, 11378, 10940, 11283, 10944, 12531, 
            11412, 10667, 11322, 11696, 10346, 12469, 12137, 11970, 11329, 
            11584, 11403, 12705, 11480, 10685, 11566, 11699, 11131, 12352, 
            11577, 11217, 11031, 11174, 10913, 12255, 11364, 10940, 11711, 
            11303, 10611, 11934, 11445, 11674, 10393, 10642, 10910, 11972, 
            11014, 10439, 11559, 11761, 10964, 11969, 11315, 11340, 10735, 
            11007, 11052, 12191, 10922, 10275, 11421, 11529, 10653, 11878, 
            11743, 11578, 10657, 11041, 11041, 11957, 11096, 10615, 11621, 
            11834, 10570, 12479, 11348, 11308, 10752, 11455, 10947, 12237, 
            11271, 10850, 11569, 12065, 10905, 12180, 11267, 11377, 10984, 
            11343, 11170, 12174, 10953, 10662, 11713, 12004, 11302, 12538, 
            12032, 11845, 10964, 11637, 11443, 12293, 11199, 10472, 11145, 
            12134, 11036, 12441, 11774, 12012, 11093, 11487, 11344, 12292, 
            11562, 10886, 11444, 11851, 10643, 12469, 11546, 11733, 10900, 
            10875, 10943, 11899, 10827, 10521, 11418, 11122, 10583, 11934, 
            11152, 11242, 10574, 11029, 10869, 11859, 11186, 10362, 11065, 
            11562, 10829, 11848, 11287, 11164, 10667, 10849, 10846, 11820, 
            10819, 10334, 10843, 11042, 10488, 11783, 11090, 10989, 10532, 
            10754, 10575, 11479, 10392, 9921, 11155, 10864, 10124, 11785, 
            11101, 11182, 10484, 10579, 10454, 11300, 10556, 10098, 10931, 
            11058, 9820, 11525, 10263, 10005, 10148, 10529, 10203, 10261, 
            9530, 9271, 10384, 10345, 9810, 11234, 10674, 10645, 10110, 10110, 
            9612, 10352, 9723, 9270, 9794, 10578, 9384, 11276, 10442, 10919, 
            9768, 9957, 9681, 10197, 9761, 9124, 9768, 10082, 9579, 10700, 
            9770, 10291, 9058, 9326, 9580, 10532, 9853, 9214, 9970, 9513, 
            9080, 9753, 9545, 9066, 7905, 7703, 8243, 10504, 11104, 10963, 
            11814, 12511, 11941, 13332, 12476, 12407, 11366, 11010, 10936, 
            11296, 10591, 10072, 10815, 10889, 10209, 11826, 11295, 11474, 
            10365, 10700, 10166, 10418, 9927, 9867, 10587, 10582, 9958, 10700, 
            10401, 10547, 9987, 9810, 9493, 9708, 8789, 8052, 8919, 9539, 
            8972, 10694, 9902, 10204, 9367, 9331, 9134, 9561, 8921, 8515, 
            9295, 8998, 8946, 9821, 9996, 10020, 8923, 9270, 8952, 9127, 
            8484, 8096, 8422, 8698, 8107, 9836, 9459, 9905, 8959, 8992, 8597, 
            8870, 8357, 8021, 8491, 8478, 7902, 9412, 9019, 9137, 8484, 8664, 
            8369, 8587, 8228, 7576, 8151, 8417, 8067, 9061, 8895, 8996, 8459, 
            8301, 7934, 8302, 7430, 6851, 7281, 8172, 7798, 9202, 8546, 8822, 
            8080, 8172, 7988, 8090, 7684, 7416, 7898, 7774, 7271, 8481, 8188, 
            8472, 7875, 7877, 7470, 7766, 7296, 6940, 7451, 7317, 7258, 8640, 
            8375, 8474, 7875, 8060, 7760, 8029, 7623, 7270, 7539, 8070, 7175, 
            8312, 7936, 8357, 7819, 7889, 7472, 7686, 6698, 6334, 7326, 7458, 
            7306, 8295, 8029, 8178, 7474, 7540, 7232, 7489, 7077, 6681, 7020, 
            6748, 6527, 7813, 7486, 7730, 7367, 7273, 7050, 7140, 6812, 6378, 
            6696, 6931, 6661, 7804, 7351, 7696, 7204, 7101, 7070, 7106, 6707, 
            6566, 6895, 7083, 6778, 7678, 7672, 7672, 7432, 7388, 7191, 7039, 
            6703, 6370, 6900, 7282, 7003, 8266, 7980, 7986, 7860, 7527, 7393, 
            7462, 6779, 6448, 6952, 6989, 6985, 8369, 8049, 8342, 7856, 7639, 
            7573, 7801, 6950, 6594, 7226, 7409, 7009, 8586, 8397, 8606, 7912, 
            8126, 7887, 7937, 7543, 7107, 7427, 7740, 7314, 9059, 8869, 8745, 
            8125, 8372, 8059, 8119, 7951, 7436, 7591, 7735, 7788, 9227, 8955, 
            8938, 8075, 8159, 7838, 7683, 7164, 6754, 7462, 6777, 6389, 8613, 
            8955, 9456, 9007, 9172, 8494, 8908, 8035, 7526, 8395, 8642, 8166, 
            10059, 9950, 10593, 10069, 10011, 9556, 9860, 9546, 8636, 8873, 
            9691, 9297, 11393, 11318, 11495, 10822, 10779, 10344, 10807, 
            10081, 9454, 9911, 10534, 10633, 12541, 11788, 11955, 11820, 
            11478, 10744, 11320, 10927, 10484, 10767, 10820, 10454, 12561, 
            12237, 12760, 11445, 11290, 11127, 11151, 10847, 10042, 10639, 
            10631, 10470, 12480, 12266, 12453, 11344, 11301, 10881, 10748, 
            10197, 9603, 10223, 10931, 10246, 12423, 12290, 12074, 11048, 
            10515, 10158, 10506, 9643, 8973, 9972, 10513, 10276, 12093, 11648, 
            11570, 10662, 10818, 10088, 10405, 9693, 9270, 9647, 9818, 9831, 
            11588, 11695, 11384, 10155, 10015, 9710, 9654, 9523, 8706, 9193, 
            9591, 9403, 10957, 10967, 10817, 9879, 9591, 8846, 8963, 9085, 
            8432, 8883, 9232, 8810, 10286, 10248, 10330, 9613, 9232, 8855, 
            8520, 8460, 8012, 8570, 8503, 8682, 10347, 10448, 10235, 9308, 
            9181, 8730, 8971, 8747, 8285, 8755, 9075, 8494, 10492, 10121, 
            10294, 9400, 9186, 8913, 9099, 8495, 8112, 8463, 8695, 8430, 
            9941, 9814, 9980, 8837, 8673, 8329, 8594, 8234, 7584, 7985, 8720, 
            8665, 9789, 9920, 9852, 8973, 9000, 8784, 8882, 8662, 7950, 8108, 
            8640, 8426, 10220, 10314, 10083, 8879, 9071, 8670, 8843, 8582, 
            7912, 8320, 8625, 8250, 10017, 10253, 9750, 9282, 9059, 8393, 
            8800, 8465, 8101, 8173, 8737, 8167, 9895, 10311, 9802, 8742, 
            8582, 8030, 8563, 8353, 7878, 8442, 8470, 8080, 9883, 10078, 
            9848, 8919, 9017, 8369, 8623, 8015, 7464, 7977, 8088, 8365, 9621, 
            9602, 9460, 8684, 8673, 8212, 8472, 7926, 7455, 7661, 8349, 8189, 
            10100, 10093, 9588, 8554, 8568, 8471, 8428, 8329, 7847, 7985, 
            8684, 8462, 10394, 10329, 9914, 9027, 8705, 8364, 8726, 8542, 
            8056, 8081, 9061, 8459, 10862, 10789, 10692, 9347, 9382, 8820, 
            9375, 9068, 8555, 8493, 9633, 9641, 11471, 11766, 11251, 10174, 
            10111, 9484, 10239, 9821, 9353, 9720, 9928, 9834, 11957, 12123, 
            11427, 10203, 10011, 9686, 10009, 9734, 8925, 8969, 9863, 9957, 
            11714, 11756, 11229, 9773, 9894, 9963, 10318, 10004, 9381, 9502, 
            10085, 9927, 12068, 12024, 11290, 10346, 9899, 9405, 9309, 9611, 
            8649, 8747, 9356, 9485, 10941, 10985, 10484, 9315, 9185, 9156, 
            9135, 8894, 8088, 8063, 9320, 8548, 10483, 10406, 10052, 8940, 
            8690, 8408, 8579, 8654, 7621, 7921, 8834, 8633, 10371, 10609, 
            9999, 9266, 9075, 8695, 9032, 8800, 8115, 8721, 9222, 9152, 10881, 
            11157, 10565, 9469, 9498, 9264, 9250, 9068, 8425, 8533, 9108, 
            9352, 10472, 10427, 10334, 9451, 9373, 9091, 9122, 8843, 8199, 
            8501, 9201, 8903, 10312, 10414, 10203, 9284, 9132, 8823, 8840, 
            8620, 7888, 8043, 9367, 8705, 10438, 10536, 10155, 9158, 9213, 
            8891, 8881, 8520, 8109, 7901, 8976, 8740, 10024, 10259, 9564, 
            8641, 8723, 8149, 8202, 7799, 7009, 7546, 8403, 8418, 9586, 9146, 
            8914, 8240, 8111, 7922, 8090, 7590, 6920, 7005, 7924, 8074, 9483, 
            9064, 8704, 8010, 7983, 7991, 7806, 7502, 6939, 6577, 7635, 7485, 
            9151, 8795, 8947, 7876, 7690, 7571, 7444, 7266, 6716, 6646, 8037, 
            7835, 9257, 9272, 8970, 8137, 8172, 7664, 7611, 7608, 6831, 6861, 
            8293, 8260, 9409, 9502, 8785, 8241, 8408, 7740, 7740, 7176, 6633, 
            6877, 7971, 7795, 8947, 8971, 8577, 8016, 8104, 7837, 7772, 7023, 
            6401, 6651, 7856, 7579, 9136, 8900, 8208, 7808, 7674, 7713, 7771, 
            7147, 6488, 6468, 7615, 7565, 8862, 8598, 8336, 8046, 7629, 7679, 
            7494, 7024, 6394, 6538, 7715, 7896, 8959, 8717, 8727, 7848, 7957, 
            7778, 7614, 7300, 6896, 6482, 8103, 8005, 9503, 9264, 9107, 8337, 
            8452, 8258, 7952, 7537, 6878, 7067, 8583, 8191, 9610, 9809, 9152, 
            8689, 8703, 8624, 8680, 7679, 7050, 7180, 8429, 8441, 9966, 9750, 
            9554, 9189, 9208, 8650, 8574, 7844, 7428, 7666, 9027, 9065, 10863, 
            10248, 10345, 9575, 9585, 9492, 9360, 8537, 7817, 8166, 9431, 
            9300, 10838, 10242, 10614, 9864, 10210, 9888, 9442, 9106, 8520, 
            8568, 10205, 10194, 11594, 11380, 11127, 10496, 10663, 10436, 
            10274, 9836, 8768, 8965, 10734, 10285, 11638, 11640, 11155, 10288, 
            10701, 10334, 9917, 9237, 8900, 8908, 10845, 10406, 11738, 11369, 
            10905, 10671, 10539, 10187, 10094, 9162, 8395, 8537, 10250, 9638, 
            11289, 11220, 10611, 10284, 10156, 10100, 9423, 8904, 7965, 8158, 
            9296, 9511, 10900, 10694, 10241, 9975, 9607, 9285, 8383, 8317, 
            7914, 8134, 8745, 8690, 9922, 9616, 9403, 9130, 9032, 8735, 8225, 
            7958, 7018, 6948, 8001, 7963, 8774, 8939, 8455, 8066, 8551, 8116, 
            7613, 7360, 6693, 6766, 7750, 7288, 8256, 8491, 8248, 8060, 8165, 
            7883, 7245, 6844, 5970, 6302, 7274, 7227, 7937, 8155, 8074, 7522, 
            8129, 7930, 7463, 6893, 6186, 6238, 7188, 7080, 7981, 7967, 8071, 
            7738, 7832, 7595, 7207, 7007, 6139, 6368, 7306, 7441, 8001, 7931, 
            8186, 7815, 8000, 7768, 7490, 7267, 6770, 6466, 7677, 7288, 8237, 
            8217, 8437, 7765, 8103, 7928, 7526, 7427, 6497, 6364, 7807, 7483, 
            8448, 8799, 8745, 8088, 8317, 8274, 8038, 8058, 6898, 6860, 8049, 
            7826, 8619, 8767, 8656, 8528, 9186, 8481, 8355, 8231, 7215, 7244, 
            8400, 8117, 8978, 8912, 8794, 8936, 9150, 8772, 8252, 7927, 7374, 
            7316, 8277, 7797, 8953, 8961, 9207, 8760, 9026, 8968, 8424, 8230, 
            7299, 7444, 8543, 8434, 9714, 9248, 9486, 9095, 9538, 9302, 8692, 
            8767, 7725, 7369, 8777, 8356, 9518, 9353, 9739, 9132, 9688, 9373, 
            8834, 8922, 8009, 7720, 9089, 8852, 9332, 9791, 9734, 9391, 10137, 
            9477, 9024, 8796, 7826, 7852, 8930, 8385, 9779, 9681, 9768, 9862, 
            10341, 10016, 9470, 9021, 8225, 8323, 9765, 9228, 10281, 10023, 
            10113, 10140, 10301, 9996, 9497, 9423, 8501, 8373, 9252, 8517, 
            9746, 9631, 9957, 9761, 10224, 9978, 9473, 8975, 8258, 7998, 
            9153, 9282, 9629, 9688, 10049, 9800, 10274, 10224, 9474, 9207, 
            8268, 8129, 9500, 8717, 9916, 10034, 10230, 9716, 10506, 10199, 
            9426, 9081, 8167, 8101, 9122, 8804, 10034, 10277, 10319, 10180, 
            10560, 10095, 9641, 9304, 8403, 8168, 9460, 8776, 9501, 9982, 
            10224, 9981, 10455, 9964, 9938, 9614, 8563, 8412, 9677, 9349, 
            10246, 10130, 10758, 10262, 10455, 10121, 10020, 9365, 8537, 
            8505, 9491, 9110, 9997, 9945, 10571, 9895, 10503, 10147, 9555, 
            9381, 8580, 8241, 9378, 8722, 10077, 10227, 10498, 10154, 10431, 
            10418, 9624, 9401, 8549, 8353, 9620, 8821, 9811, 10064, 10278, 
            10014, 10478, 10241, 9338, 9351, 8431, 8076, 9677, 8954, 9646, 
            9713, 10408, 9877, 10119, 9959, 9323, 9174, 8194, 8033, 9071, 
            8989, 10067, 9823, 10322, 10216, 10325, 10082, 9419, 9316, 8492, 
            8141, 8917, 8546, 9194, 8816, 9580, 9531, 9232, 9369, 8374, 8147, 
            7506, 7522, 8278, 7929, 8727, 8386, 9319, 8960, 8966, 8607, 8099, 
            8214), tsp = c(1851, 2023.75, 12), class = "ts") -> 
  births_sweden

seas(births_sweden)
#> Error: X-13 run failed
#> 
#> Errors:
#> - Problem reading
#>   C:\Users\staff\AppData\Local\Temp\Rtmpi2WE5Z\x13b29c6ce828af/iofile.dta.
#>   Check your input file and format.
#> - Problem reading
#>   C:\Users\staff\AppData\Local\Temp\Rtmpi2WE5Z\x13b29c6ce828af/iofile.dta.
#>   Check your input file and format.
#> - Time series could not be read due to previously found errors
#> - Specify series before user-defined adjustments
#> - Need to specify a series to identify outliers
#> 
#> Notes:
#> - Correct input errors in the order they are detected since the
#>   first one or two may be responsible for the others (especially
#>   if there are errors in the SERIES or COMPOSITE spec).

Created on 2023-12-26 with reprex v2.0.2

eddelbuettel commented 9 months ago

I think this is limitation in the x13 binary that is not as well documented as it could be. The earliest year for which this works for me is

seas(window(births_sweden, start=1942)
eddelbuettel commented 9 months ago

Issue #309, filed only weeks ago, touched the same topic and refers to the reference manual stating that 780 is the maximum number of observations. (And oddly length(window(births_sweden, start=1942)) is higher than that but that reference table 2.2 on program limits has some extra language.)

PS Same manual in its current version

StaffanBetner commented 9 months ago

Thanks! I looked into the source code and traced it to srslen.prm. pobs = pyr1*psp, pyr1 = maximum number of years in the series (set to 65), psp = maximum length of seasonal period (set to 12). The maximum number of years seems to me like a fairly arbitrary number.

StaffanBetner commented 9 months ago

It doesn't fully work to change. The maximum value (of years in the series) that I can compile x13binary with is 136, but the highest that actually works to run is 131. The longest span that actually works is seas(window(births_sweden, start=c(1895,5))), which is 1542 values, compared to 131*12 = 1572. I think I am satisfied for today with bending the capabilities of X-13ARIMA-SEATS.