search

rjdverse / rjdemetra

R interface to JDemetra+ v 2.x
https://rjdverse.github.io/rjdemetra
52 stars 16 forks source link

Differences between results JDemetra+ app and RJDemetra #121

Closed VabelGit closed 1 year ago

VabelGit commented 1 year ago

#R code.txt Dear JDemetra & RJDemetra Team,

I hope this message finds you well. I would like to bring to your attention a notable difference in the results obtained when using the same dataset and function in the TRAMO-SEATS option between the JDemetra and RJDemetra applications. Interestingly, these discrepancies do not seem to arise when using the X13ARIMA function.

To illustrate this issue, let me provide an example: When employing the FULL method in TRAMO-SEATS with no other variations in both versions, I obtained identical results. Similarly, when using a single pre-defined outlier, the results matched. However, the problem emerged when I introduced two or more outliers into the analysis. The values started to diverge significantly, and this discrepancy grew more pronounced as I included additional outliers.

To help you investigate this matter further, I have attached the Excel file that was used along with the corresponding R code.

Your assistance in resolving this issue would be greatly appreciated.

Thank you for your attention to this matter.

Best regards,

David DataSet.xlsx

[FULL TRAMO SEATS JDEMETRA.pdf] #R code.txt (https://github.com/jdemetra/rjdemetra/files/12550501/FULL.TRAMO.SEATS.JDEMETRA.pdf)

AQLT commented 1 year ago

Hi, This is not a difference between JDemetra and RJDemetra: the two models weren't exactly the same. Indeed, the variables were not in the same orders in JDemetra and in RJDemetra and it might cause small discrepancies (see below, changing the order of the variables you get the same results). I think this comes from the Cholesky decomposition which depends on the order of the variables (what do you think @palatej?). In X-13 you have the same behavior:

library("RJDemetra")
Arm_ts <- structure(c(1.85613376004978, 1.01599277516231, 0.409969676625379, 
                      -0.19363865971248, 0.73179011766618, 0.157756635632296, 0.934719470678138, 
                      -1.15596845635705, 1.22914353394814, -0.786294642015085, 0.0967992231783827, 
                      -1.78206604635777, -1.70159663013218, 0.519568692089048, -0.703398064216543, 
                      0.957732968671932, -0.0332950528576226, 0.798351556026831, -0.128207870126831, 
                      -0.572134617428688, -0.333324861086971, -0.974786096681401, -1.95241202902086, 
                      -0.599099039762621, 0.83228810791031, 0.904928768573658, 0.799461205072395, 
                      -0.372899458437729, 0.0366828797622636, -0.876093251672868, -0.436428113064847, 
                      0.85692537769209, 0.364886205748785, -0.344118999769091, -0.248982553984446, 
                      -0.361943811625181, 0.0574898628200714, -1.24117734600735, -0.063720789372023, 
                      1.20783373821279, 0.810358319906532, 1.61319742994802, -0.753722875399416, 
                      -0.660949855897501, -1.87449056451071, -1.16679021616578, -0.471303036593827, 
                      0.161523095317605, 0.0177425874510403, 1.01352730486053, 0.52021123589414, 
                      1.95473225279899, 0.087728200437802, 0.369068612785834, 0.733138714289939, 
                      -0.101887784702298, 0.793328377478823, -0.750900696592949, -0.191750331498976, 
                      -0.169233138062976, 0.289123237713423, 1.28571486490122, -0.411464457592012, 
                      0.0128650162931184, -1.1196834925454, -0.177577741563763, 0.143523734914483, 
                      0.266592557926542, 0.266613167183809, -1.83874440098109, -0.504730649361504, 
                      -2.11067519632259, 2.55276155067324, 1.26473294919839, 1.48133304286294, 
                      -1.17592462107388, -0.0329736753999503, 0.837391448809532, 0.599928483699016, 
                      0.500172227742847, 0.307815104328697, -1.25404210250686, -2.60402673944506, 
                      -1.34130373768747, 0.328766479932466, 1.97518927205851, 1.34127097734929, 
                      -0.544529548185877, -0.146557968437064, -0.360409359176235, 2.64091407678773, 
                      0.805949594720995, -0.102471277274376, -2.08456650598801, -2.29958058878334, 
                      -1.16289670439897, -0.268741106875892, 1.66121771046285, 0.751697108683653, 
                      1.07754505902223, 1.2104592996658, 0.76204777380612, 0.970680035224445, 
                      -0.894096824640605, -0.736949709456911, -1.20125123837885, -1.60765882907886, 
                      -1.62120689624645, 0.36136471997403, 1.15774756476895, 0.984714166023884, 
                      0.425911177844397, -0.453442123051457, 0.468103330216606, -0.106853296343133, 
                      0.637808690512947, -0.126613000133398, -1.47409548830109, -0.170649971429288, 
                      -0.141421192477743, 0.906311429356918, -0.174428961261781, 0.193363266360818, 
                      0.215490987727812, -0.0576503346482114, 0.511544967027658, -0.0893051038286308, 
                      1.07292407931165, -0.416385484171812, -1.70453675767043, -2.54115002354957, 
                      -0.115028256002963, 0.630459327956451, 0.582138714380344, -0.887629926589435, 
                      0.0448169925573954, 0.514546188275698, 0.129766167728341, 1.56779800795833, 
                      0.509039226255878, 0.771958409332313, -3.22523651130626, -3.42499698399035, 
                      -1.33365513532674, 1.64235993156724, 1.9497112171101, 0.823763759002964, 
                      0.373067174570566, 0.707090188871843, 0.339507754188135, 0.784283587732792, 
                      0.287511019629615, 0.105306750866549, -2.11859382198391, -1.86324537976993, 
                      -3.58780416263862, -6.84538481570071, -9.5111675150999, -6.1526481219663, 
                      6.02756208333594, 5.13753412587066, 5.2690768435309, 2.1722694474584, 
                      2.18174389041654, 2.67441869192068, 1.94416295487176, 2.16776774349453, 
                      1.64736595303104, -0.366420363184284, -0.0615202505502321, -1.94635379988511, 
                      3.04822305283389, 0.85526560539111, 1.42640540312568, -0.653947770024238, 
                      1.41581293034992, 1.44831479450186, 1.11085308461217, 0.719698423828838, 
                      -1.11856376423412, -0.894192763401195, -1.98299971731053, -0.691717920511474, 
                      -1.13192309637223, 0.806864717166167, 0.434958470215575, 1.11118643651412, 
                      -0.93452700556162, -0.0724856742285274, -0.773815619067679, 0.0230321431899024, 
                      0.249538332224179, 1.27955573363863, 0.813856021152547, -1.03173469451229
), dim = c(195L, 1L), dimnames = list(NULL, "Armenia"), tsp = c(2007.08333333333, 
                                                                2023.25, 12), class = "ts")

# Model 2
Spec2 <- tramoseats_spec(
  outlier.enabled = FALSE, 
  usrdef.outliersEnabled = TRUE, 
  usrdef.outliersType = c("AO", "LS"), 
  usrdef.outliersDate = c("2020-03-01", "2020-02-01"))

mod_Arm2 <- tramoseats(Arm_ts, Spec2)
summary(mod_Arm2$regarima)
#> y = regression model + arima (0, 0, 2, 1, 0, 0)
#> 
#> Model: RegARIMA - TRAMO/SEATS
#> Estimation span: from 2-2007 to 4-2023
#> Log-transformation: no
#> Regression model: no mean, no trading days effect, no leap year effect, no Easter effect, outliers(2)
#> 
#> Coefficients:
#> ARIMA: 
#>          Estimate Std. Error T-stat Pr(>|t|)    
#> BPhi(1)  -0.21181    0.07209 -2.938   0.0037 ** 
#> Theta(1)  0.71541    0.05030 14.222   <2e-16 ***
#> Theta(2)  0.72929    0.05023 14.519   <2e-16 ***
#> ---
#> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#> 
#> Regression model: 
#>             Estimate Std. Error T-stat Pr(>|t|)    
#> AO (3-2020)  -3.8662     0.7166 -5.395 1.97e-07 ***
#> LS (2-2020)   0.1324     0.2728  0.485    0.628    
#> ---
#> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#> 
#> 
#> Residual standard error: 1.142 on 189 degrees of freedom
#> Log likelihood = -303.6, aic = 619.2, aicc = 619.7, bic(corrected for length) = 0.4001

# Model 6
spec6 <- tramoseats_spec(outlier.enabled = FALSE, 
                        usrdef.outliersEnabled = TRUE, 
                        usrdef.outliersType = c("AO", "LS", "LS", "LS", "TC", "LS"), 
                        usrdef.outliersDate = c("2020-03-01", "2021-02-01", "2020-02-01", "2020-08-01","2021-05-01","2020-05-01"))
mod_Arm6 <- tramoseats(Arm_ts, spec6)
#> Warning in tramoseats.SA_spec(Arm_ts, spec6): [decomposition.Model
#> decomposition: Parameters cut off]
summary(mod_Arm6$regarima)
#> y = regression model + arima (0, 0, 3, 1, 0, 0)
#> 
#> Model: RegARIMA - TRAMO/SEATS
#> Estimation span: from 2-2007 to 4-2023
#> Log-transformation: no
#> Regression model: no mean, no trading days effect, no leap year effect, no Easter effect, outliers(6)
#> 
#> Coefficients:
#> ARIMA: 
#>          Estimate Std. Error  T-stat Pr(>|t|)    
#> BPhi(1)  -0.33609    0.07211  -4.661 5.83e-06 ***
#> Theta(1)  0.38647    0.05722   6.755 1.61e-10 ***
#> Theta(2)  0.28123    0.05947   4.729 4.32e-06 ***
#> Theta(3) -0.67614    0.05690 -11.883  < 2e-16 ***
#> ---
#> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#> 
#> Regression model: 
#>             Estimate Std. Error  T-stat Pr(>|t|)    
#> AO (3-2020)  -2.5321     0.3153  -8.031 8.97e-14 ***
#> LS (2-2021)  -2.4710     0.3648  -6.773 1.45e-10 ***
#> LS (2-2020)  -7.1393     0.6041 -11.819  < 2e-16 ***
#> LS (8-2020)  -3.1197     0.7912  -3.943 0.000112 ***
#> TC (5-2021)   3.3933     0.4495   7.549 1.64e-12 ***
#> LS (5-2020)  12.7895     0.9888  12.934  < 2e-16 ***
#> ---
#> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#> 
#> 
#> Residual standard error: 0.7791 on 184 degrees of freedom
#> Log likelihood = -232.4, aic = 486.7, aicc = 488.2, bic(corrected for length) = -0.2289

Spec2_x13_1 <- regarima_spec_x13(
  outlier.enabled = FALSE, 
  usrdef.outliersEnabled = TRUE, 
  usrdef.outliersType = c("LS", "AO"), 
  usrdef.outliersDate = c("2020-02-01", "2020-03-01"))
Spec2_x13_2 <- regarima_spec_x13(
  outlier.enabled = FALSE, 
  usrdef.outliersEnabled = TRUE, 
  usrdef.outliersType = c("AO", "LS"), 
  usrdef.outliersDate = c("2020-03-01", "2020-02-01"))
mod1_x13 <- summary(regarima(Arm_ts, Spec2_x13_1))
mod2_x13 <- summary(regarima(Arm_ts, Spec2_x13_2))
cbind(rbind(mod1_x13$coefficients$arima,
            mod1_x13$coefficients$regression)[,1],
      rbind(mod2_x13$coefficients$arima,
            mod2_x13$coefficients$regression)[,1][c(1:5,7,6)])
#>                   [,1]       [,2]
#> Phi(1)       0.2395841  0.2396029
#> BPhi(1)     -0.2300540 -0.2300525
#> Theta(1)     0.8775714  0.8776135
#> Theta(2)     0.8057022  0.8057235
#> Mean         0.6378764  0.6379218
#> LS (2-2020)  0.7636256  0.7636859
#> AO (3-2020) -3.5525028 -3.5522040
VabelGit commented 1 year ago

Hi, you are right, using the same order as JDetretra+ gives me the same results. Also, I found that in some cases, this happens too when using the X13 method.

I think (maybe is possible, IDK), that the JDemetra+ could have the option to specify the order of the outliers to review different possible combinations. In any case, this is only a suggestion.

Thanks for the clarification.

VabelGit commented 1 year ago

Hi ¡

I´m using the 4.3.1 version of R and the NbDemetra 2.2.4.

As I mentioned in GitHub, would be nice that JDemetra allow to specify the order of the outliers.

Thanks 😊

From: Tanguy Barthelemy @.> Sent: Thursday, September 7, 2023 10:30 AM To: jdemetra/rjdemetra @.> Cc: VabelGit @.>; Mention @.> Subject: Re: [jdemetra/rjdemetra] Differences between results JDemetra+ app and RJDemetra (Issue #121)

Hi @VabelGit https://github.com/VabelGit ,

With the same specifications as mentioned in the GUI, I get the same results. What version of R and RJDemetra are you using?

Also in your code you define more outliers than in the GUI (2nd scenario).

Here is the code that reproduces your two scenarios:

library("RJDemetra") library("readxl")

Arm <- read_xlsx("./data/DataSet.xlsx") Arm_ts <- ts(Arm[, -1L], start = c(2007L, 2L), frequency = 12L)

Model 1 : AUTOMATIC

Spec1 <- tramoseats_spec(spec = "RSA3") mod_Arm1 <- tramoseats(Arm_ts, Spec1) mod_Arm1$regarima

Model 2

Spec2 <- tramoseats_spec( outlier.enabled = FALSE, usrdef.outliersEnabled = TRUE, usrdef.outliersType = c("LS", "AO"), usrdef.outliersDate = c("2020-02-01", "2020-03-01"))

mod_Arm2 <- tramoseats(Arm_ts, Spec2) mod_Arm2$regarima

— Reply to this email directly, view it on GitHub https://github.com/jdemetra/rjdemetra/issues/121#issuecomment-1710358488 , or unsubscribe https://github.com/notifications/unsubscribe-auth/BCNLQ3TEA3VBD5CZS6FPMFTXZHR6HANCNFSM6AAAAAA4PBNCPI . You are receiving this because you were mentioned. https://github.com/notifications/beacon/BCNLQ3QSLQSGZGJ7PF7K42LXZHR6HA5CNFSM6AAAAAA4PBNCPKWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTTF6H75Q.gif Message ID: @. @.> >

-- Aviso legal: El contenido de este mensaje y los archivos adjuntos son confidenciales y de uso exclusivo de la Universidad Nacional de Colombia. Se encuentran dirigidos sólo para el uso del destinatario al cual van enviados. La reproducción, lectura y/o copia se encuentran prohibidas a cualquier persona diferente a este y puede ser ilegal. Si usted lo ha recibido por error, infórmenos y elimínelo de su correo. Los Datos Personales serán tratados conforme a la Ley 1581 de 2012 y a nuestra Política de Datos Personales que podrá consultar en la página web  www.unal.edu.co http://www.unal.edu.co/. Las opiniones, informaciones, conclusiones y cualquier otro tipo de dato contenido en este correo electrónico, no relacionados con la actividad de la Universidad Nacional de Colombia, se entenderá como personales y de ninguna manera son avaladas por la Universidad.

TanguyBarthelemy commented 11 months ago

Ok, thank you for your interesting comment 💯!