search

srlanalytics / bdfm

Bayesian dynamic factor model estimation and predictive statistics including nowcasting and forecasting
MIT License
5 stars 6 forks source link

weird stuff #54

Closed christophsax closed 5 years ago

christophsax commented 5 years ago

I got some weird results auf of bdfm but cannot build a reprex for some reason. If you look at the inputs below, can you tell me why bdfm retuns the following:

Browse[1]> Est$Ys
       [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11] [,12] [,13]
  [1,]  NaN  NaN  NaN  NaN  NaN  NaN  NaN  NaN  NaN   NaN   NaN   NaN   NaN
  [2,]  NaN  NaN  NaN  NaN  NaN  NaN  NaN  NaN  NaN   NaN   NaN   NaN   NaN
  [3,]  NaN  NaN  NaN  NaN  NaN  NaN  NaN  NaN  NaN   NaN   NaN   NaN   NaN
  [4,]  NaN  NaN  NaN  NaN  NaN  NaN  NaN  NaN  NaN   NaN   NaN   NaN   NaN
  [5,]  NaN  NaN  NaN  NaN  NaN  NaN  NaN  NaN  NaN   NaN   NaN   NaN   NaN
  [6,]  NaN  NaN  NaN  NaN  NaN  NaN  NaN  NaN  NaN   NaN   NaN   NaN   NaN
  [7,]  NaN  NaN  NaN  NaN  NaN  NaN  NaN  NaN  NaN   NaN   NaN   NaN   NaN
  [8,]  NaN  NaN  NaN  NaN  NaN  NaN  NaN  NaN  NaN   NaN   NaN   NaN   NaN
  [9,]  NaN  NaN  NaN  NaN  NaN  NaN  NaN  NaN  NaN   NaN   NaN   NaN   NaN

Input

Browse[1]> B
          [,1]        [,2]        [,3]       [,4]        [,5]        [,6]
[1,] 0.1400570  0.28342898  0.02463268 0.09876378  0.41493088 -0.08225953
[2,] 0.2427822 -0.06140643 -0.08899870 0.19425675 -0.04059895 -0.10247326
[3,] 0.1358646 -0.01709368  0.16005657 0.02191113 -0.04848549  0.01263135
           [,7]        [,8]        [,9]
[1,] 0.04155414  0.05197034 -0.04299829
[2,] 0.16206179  0.09270511  0.02016617
[3,] 0.03053021 -0.03768047  0.12379524
Browse[1]> 
Browse[1]> Jb
9 x 15 sparse Matrix of class "dgCMatrix"

 [1,] 1 . . . . . . . . . . . . . .
 [2,] . 1 . . . . . . . . . . . . .
 [3,] . . 1 . . . . . . . . . . . .
 [4,] . . . 1 . . . . . . . . . . .
 [5,] . . . . 1 . . . . . . . . . .
 [6,] . . . . . 1 . . . . . . . . .
 [7,] . . . . . . 1 . . . . . . . .
 [8,] . . . . . . . 1 . . . . . . .
 [9,] . . . . . . . . 1 . . . . . .
Browse[1]> 
Browse[1]> q
          [,1]       [,2]       [,3]
[1,] 14018.384 -3911.3251  -729.5430
[2,] -3911.325 14068.3176  -713.8509
[3,]  -729.543  -713.8509 10361.2285
Browse[1]> 
Browse[1]> H
               [,1]        [,2]         [,3]
 [1,] -0.1143946088 -0.16499381 -0.061151420
 [2,]  0.0038247616  0.09023861 -0.113355065
 [3,] -0.4466763019 -0.35558773 -0.157309826
 [4,] -0.1408597502 -0.21964144  0.306970119
 [5,] -0.0943984992 -0.10749433 -0.100578793
 [6,] -0.0633560241 -0.09445751  0.214317138
 [7,]  0.3748361250  0.26531102  0.177311520
 [8,] -0.0747134933 -0.15460347 -0.052510850
 [9,] -0.4171408818 -0.22224975 -0.172704390
[10,] -0.2197146863 -0.18166046  0.035243910
[11,]  0.0304629364 -0.18320742  0.532106884
[12,] -0.1427637682 -0.14947988 -0.011947067
[13,] -0.0726245109 -0.11646064 -0.035470863
[14,] -0.4534462202  0.51306689  0.156004324
[15,] -0.4503976334  0.51678906  0.153024418
[16,] -0.1411228734 -0.16502487 -0.028488998
[17,] -0.1663797693 -0.20014563 -0.078525110
[18,]  0.1200498541  0.12373650  0.084360004
[19,]  0.0563724157  0.15904004 -0.236848561
[20,]  0.0503939171  0.10869034  0.128632086
[21,]  0.0006579469  0.06502818 -0.507188098
[22,]  0.2100304664  0.26192716  0.042044222
[23,] -0.0699741040  0.02934624 -0.005812104
[24,]  0.0607570828  0.18796254 -0.483950140
Browse[1]> 
Browse[1]> head(R)
         [,1]     [,2]     [,3]     [,4]     [,5]     [,6] [,7] [,8] [,9] [,10]
[1,] 4047.516    0.000    0.000    0.000    0.000    0.000    0    0    0     0
[2,]    0.000 8931.522    0.000    0.000    0.000    0.000    0    0    0     0
[3,]    0.000    0.000 3320.686    0.000    0.000    0.000    0    0    0     0
[4,]    0.000    0.000    0.000 7302.149    0.000    0.000    0    0    0     0
[5,]    0.000    0.000    0.000    0.000 9512.611    0.000    0    0    0     0
[6,]    0.000    0.000    0.000    0.000    0.000 9099.507    0    0    0     0
     [,11] [,12] [,13] [,14] [,15] [,16] [,17] [,18] [,19] [,20] [,21] [,22]
[1,]     0     0     0     0     0     0     0     0     0     0     0     0
[2,]     0     0     0     0     0     0     0     0     0     0     0     0
[3,]     0     0     0     0     0     0     0     0     0     0     0     0
[4,]     0     0     0     0     0     0     0     0     0     0     0     0
[5,]     0     0     0     0     0     0     0     0     0     0     0     0
[6,]     0     0     0     0     0     0     0     0     0     0     0     0
     [,23] [,24]
[1,]     0     0
[2,]     0     0
[3,]     0     0
[4,]     0     0
[5,]     0     0
[6,]     0     0
Browse[1]> head(Y)
     A191RL1Q225SBEA W068RCQ027SBEA USSLIND     PCEDG     PCEND    UMCSENT
[1,]              NA             NA      NA        NA        NA         NA
[2,]              NA             NA      NA -153.9913  92.86770   30.14165
[3,]       -48.11321             NA      NA -212.8959  72.33531 -268.83014
[4,]              NA             NA      NA -263.4715 -56.52035 -134.52641
[5,]              NA             NA      NA -107.2555 -23.28103  -21.04352
[6,]      -374.03044       171.5241      NA  124.6114 -58.88443  180.01014
        UNRATE JTSJOL     INDPRO CSUSHPINSA     HSN1F TSIFRGHT FRGSHPUSM649NCIS
[1,]        NA     NA         NA         NA        NA       NA               NA
[2,]   2.05965     NA  -25.50708         NA -126.2572       NA               NA
[3,]   2.05965     NA  -81.29109         NA -184.5785       NA               NA
[4,] 362.75592     NA -359.64318         NA -344.4773       NA               NA
[5,] 362.75592     NA -423.04068         NA  326.7322       NA               NA
[6,]  62.17570     NA -228.58764         NA  224.0656       NA               NA
     CAPUTLG2211S   IPG2211S DGORDER AMTMNO MNFCTRIRSA RETAILIRSA WHLSLRIRSA
[1,]           NA         NA      NA     NA         NA         NA         NA
[2,]    171.62829  179.99908      NA     NA         NA         NA         NA
[3,]    113.30408  118.41151      NA     NA         NA         NA         NA
[4,]   -123.43476 -114.09116      NA     NA         NA         NA         NA
[5,]   -125.99941 -119.33716      NA     NA         NA         NA         NA
[6,]    -40.02244  -33.15787      NA     NA         NA         NA         NA
     CPILFESL      ICSA TWEXB T10Y3M
[1,]       NA        NA    NA     NA
[2,] 491.1365  27.10702    NA     NA
[3,]       NA  95.03807    NA     NA
[4,]       NA        NA    NA     NA
[5,] 314.0447 369.30948    NA     NA
[6,]       NA -29.49387    NA     NA
Browse[1]>
christophsax commented 5 years ago

Pushed a sanity check to master. When I run the following in a fresh R session, it fails, like below. But it works fine in a reprex. So this returns sometimes NaN, sometimes not. Don't get it.

> library(bdfm)
> logs <- c("W068RCQ027SBEA", "PCEDG", "PCEND", "JTSJOL", "INDPRO", "CSUSHPINSA", "HSN1F", "TSIFRGHT", "IPG2211S", "DGORDER", "AMTMNO", "CPILFESL", "ICSA")
> diffs <- setdiff(colnames(econ_us), c("A191RL1Q225SBEA", "USSLIND"))
> 
> m <- dfm(data = econ_us, factors = 3, pre_differenced = "A191RL1Q225SBEA", store_idx = "A191RL1Q225SBEA",
+          logs = logs, diffs = diffs)
Error in bdfm(Y = Y, m = m, p = p, Bp = Bp, lam_B = lam_B, Hp = Hp, lam_H = lam_H,  : 
  !all(is.na(Est$Ys)) is not TRUE
srlanalytics commented 5 years ago

This looks like the standard us_gdp.R example, right? Odd that it errors... Every time I've run it it's been fine. However, based on what you've said I have a pretty good idea what's going on:

NaN's typically come from the Rcpp code (C++ doesn't support NA; NaN is the numeric (double) alternative to an actual value). This happens when simulated draws for parameters and factors are not stationary --- one or the other explodes and results in NaN values. The reason you might get it some times and not others is because we're drawing for factors and parameters, and it could be the case that sometimes these draws converge to nice stationary distributions and sometimes they don't. For a truly reproducible example we need to set the RNG seed in the C++ code. This is something we'll have to introduce at some point... I looked into it and it wasn't obvious to me how to go about it since we're doing so many simulations in different places in the code.

However, I still think it's odd than any run of the us_gdp.R example would not have converged... I've run it probably 50 times and never had a problem. Is there anything particularly different about this example or the environment?

srlanalytics commented 5 years ago

Quick update here --- I moved the error message into the C++ code so it will break earlier of draws are not stationary, and it returns the error "Draws non-stationary" which is hopefully more informative.

christophsax commented 5 years ago

However, I still think it's odd than any run of the us_gdp.R example would not have converged... I've run it probably 50 times and never had a problem. Is there anything particularly different about this example or the environment?

When I opened the issue, I got the problem every time, so it was probably something on top of the seed. Since it is gone now, and we cannot reproduce, I guess we close this. Setting the seed in cpp would be valuable if we run into similar situations.