Matlab calls wrong "estimate" routine

IRIS-Solutions-Team / IRIS-Toolbox

[IrisToolbox] for Macroeconomic Modeling

Other

92 stars 42 forks source link

Matlab calls wrong "estimate" routine #374

Open guzmangt opened 1 year ago

guzmangt commented 1 year ago

Hi Jaromir,

I am trying to estimate a model, using R2022a and IRIS_20221026 as follows:

[Estar,ps_obj,Cov,Hess,mest,V,Delta,PDelta] = estimate(m,d,estrange, E,...
    'NoSolution', 'Penalty', 'Filter', filteropt, 'MaxIter', 10000, 'MaxFunEvals', 1e+6, 'TolFun', 1e-12, 'TolX', 1e-12, 'Solver', 'fmin');

For some reason, matlab calls the estimate routine from some matlab toolbox:

which estimate
C:\Program Files\MATLAB\R2022a\toolbox\econ\econ\@statespace\estimate.m  % statespace method

However, the IRIS toolbox is at the top of the matlab path:

path

        MATLABPATH

    my_path\IRIS_Tbx_20221026\Legacy
    my_path\IRIS_Tbx_20221026\DataManagement
    my_path\IRIS_Tbx_20221026\Extend
    my_path\IRIS_Tbx_20221026\Reporting
    my_path\IRIS_Tbx_20221026\Shortcuts
    my_path\IRIS_Tbx_20221026\ShrinkageEstimation
    my_path\IRIS_Tbx_20221026\Statistics
    my_path\IRIS_Tbx_20221026\StructuralModeling
    my_path\IRIS_Tbx_20221026\TimeSeriesModeling
    my_path\IRIS_Tbx_20221026\Utilities
    my_path\IRIS_Tbx_20221026\dates
    my_path\IRIS_Tbx_20221026\dbase
    my_path\IRIS_Tbx_20221026\dates\DaterConstructors
    my_path\IRIS_Tbx_20221026

...

I'm not sure whether this is a matlab bug, or whether it has to do with how the IRIS toolbox paths are added to the matlab path. Have you encountered this before? Do you have tips on how to circumvent this? I tried calling model.estimate(...) but that gave me an error.

Best, Guzmán

jaromir-benes commented 1 year ago

Hi - if "m" is a Model object, then Matlab will always call the correct estimate function. Can you verify that immediately before you run the line with the estimate function, "m" exists as a Model object? (Put a debug point immediately before that, and verify on the command line). Best J

On Wed, May 3, 2023 at 3:38 PM guzmangt @.***> wrote:

Hi Jaromir,

I am trying to estimate a model, using R2022a and IRIS_20221026 as follows:

[Estar,ps_obj,Cov,Hess,mest,V,Delta,PDelta] = estimate(m,d,estrange, E,... 'NoSolution', 'Penalty', 'Filter', filteropt, 'MaxIter', 10000, 'MaxFunEvals', 1e+6, 'TolFun', 1e-12, 'TolX', 1e-12, 'Solver', 'fmin');

For some reason, matlab calls the estimate routine from some matlab toolbox:

which estimate C:\Program @.***\estimate.m % statespace method

However, the IRIS toolbox is at the top of the matlab path:

path
  MATLABPATH
my_path\IRIS_Tbx_20221026\Legacy my_path\IRIS_Tbx_20221026\DataManagement my_path\IRIS_Tbx_20221026\Extend my_path\IRIS_Tbx_20221026\Reporting my_path\IRIS_Tbx_20221026\Shortcuts my_path\IRIS_Tbx_20221026\ShrinkageEstimation my_path\IRIS_Tbx_20221026\Statistics my_path\IRIS_Tbx_20221026\StructuralModeling my_path\IRIS_Tbx_20221026\TimeSeriesModeling my_path\IRIS_Tbx_20221026\Utilities my_path\IRIS_Tbx_20221026\dates my_path\IRIS_Tbx_20221026\dbase my_path\IRIS_Tbx_20221026\dates\DaterConstructors my_path\IRIS_Tbx_20221026

...

I'm not sure whether this is a matlab bug, or whether it has to do with how the IRIS toolbox paths are added to the matlab path. Have you encountered this before? Do you have tips on how to circumvent this? I tried calling model.estimate(...) but that gave me an error.

Best, Guzmán

— Reply to this email directly, view it on GitHub https://github.com/IRIS-Solutions-Team/IRIS-Toolbox/issues/374, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGCVKKXZ7CFOHHP7I3MWWNTXEJNULANCNFSM6AAAAAAXUNVTMU . You are receiving this because you are subscribed to this thread.Message ID: @.***>

guzmangt commented 1 year ago

Thank you so much for the quick reply!

I do believe "m" is a model object:

K>> disp(m)
  1×1 Linear Model

                   FileName: "po_nairu2_Bela.model"
                    Comment: "% po_nairu2.model                                                         %"
               LinearStatus: 1
               GrowthStatus: 0
                NumVariants: 1
          NumVariantsSolved: 1
    NumMeasurementEquations: 13
     NumTransitionEquations: 27
       SizeTransitionMatrix: [32 28]
             NumExportFiles: 0
                   UserData: []

Furthermore, the access() function seems to be working with it fine:

K>> access(m,"transition-variables")

ans = 

  1×27 string array

  Columns 1 through 12

    "UNR_HAT"    "UNR_BAR"    "UNR_SFT"    "Y_HAT"    "Y_BAR"    "Y_SFT"    "LAX_HAT"    "LAX_BAR"    "LAX_SFT"    "AHN_HAT"    "AHN_BAR"    "AHN_SFT"

  Columns 13 through 25

    "TFP_HAT"    "TFP_BAR"    "LAN_"    "LAN_SFT"    "K_"    "K_SFT"    "TFP_"    "TFP_SFT"    "dY"    "dY_BAR"    "d4Y_BAR"    "PIE_HAT"    "PIE_BAR"

  Columns 26 through 27

    "WIE_HAT"    "WIE_BAR"

K>> access(m,"parameter-values")

ans = 

  struct with fields:

    alpha1: 0.5000
    alpha2: 0.7000
    gamma1: 0.7000
    gamma2: 0.5000
    gamma4: 0
    gamma6: 0
      iota: 0.6319
     kappa: 0.5000
     beta1: 0.7000
     beta2: 0.5000
      phi0: 0.0163
       phi: 0.7000
     beta3: 0.7000
     beta4: 0.5000
     wdisc: 0.0027
     omega: 1

jaromir-benes commented 1 year ago

...and what exactly happens when you run the estimate(m, ...) function?

On Wed, May 3, 2023 at 4:11 PM guzmangt @.***> wrote:

Thank you so much for the quick reply!

I do believe "m" is a model object:

K>> disp(m) 1×1 Linear Model
               FileName: "po_nairu2_Bela.model"
                Comment: "% po_nairu2.model                                                         %"
           LinearStatus: 1
           GrowthStatus: 0
            NumVariants: 1
      NumVariantsSolved: 1
NumMeasurementEquations: 13
 NumTransitionEquations: 27
   SizeTransitionMatrix: [32 28]
         NumExportFiles: 0
               UserData: []
Furthermore, the access() function seems to be working with it fine:

K>> access(m,"transition-variables")

ans =

1×27 string array

Columns 1 through 12
"UNR_HAT"    "UNR_BAR"    "UNR_SFT"    "Y_HAT"    "Y_BAR"    "Y_SFT"    "LAX_HAT"    "LAX_BAR"    "LAX_SFT"    "AHN_HAT"    "AHN_BAR"    "AHN_SFT"
Columns 13 through 25
"TFP_HAT"    "TFP_BAR"    "LAN_"    "LAN_SFT"    "K_"    "K_SFT"    "TFP_"    "TFP_SFT"    "dY"    "dY_BAR"    "d4Y_BAR"    "PIE_HAT"    "PIE_BAR"
Columns 26 through 27
"WIE_HAT"    "WIE_BAR"
K>> access(m,"parameter-values")

ans =

struct with fields:
alpha1: 0.5000
alpha2: 0.7000
gamma1: 0.7000
gamma2: 0.5000
gamma4: 0
gamma6: 0
  iota: 0.6319
 kappa: 0.5000
 beta1: 0.7000
 beta2: 0.5000
  phi0: 0.0163
   phi: 0.7000
 beta3: 0.7000
 beta4: 0.5000
 wdisc: 0.0027
 omega: 1
— Reply to this email directly, view it on GitHub https://github.com/IRIS-Solutions-Team/IRIS-Toolbox/issues/374#issuecomment-1533099861, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGCVKKSU3DC3OPMEPKWE66DXEJRP5ANCNFSM6AAAAAAXUNVTMU . You are receiving this because you commented.Message ID: @.***>

guzmangt commented 1 year ago

It estimates something wildly different (and which does not look very reasonable) than what I get with R2019a and IRIS_20151016

jaromir-benes commented 1 year ago

If it runs, then it calls the "right" estimate function.

The differences may come from two things:

1) There's a different default setting for how the kalman filter treats unit root variables. Sometimes it performs much better, sometimes much worse

Try to use the old default method by setting

... = estimate(m, ..., "filter", {"unitRootInitials", "fixedUnknown"}, ...)

2) You may be setting up different prior distributions. The prior distribution functions were a bit different back in 2015 than they're now. Copy-paste your prior specifications from before and now, I'll have a quick look.

On Wed, May 3, 2023 at 4:17 PM guzmangt @.***> wrote:

It estimates something wildly different (and which does not look very reasonable) than what I get with R2019a and IRIS_20151016

— Reply to this email directly, view it on GitHub https://github.com/IRIS-Solutions-Team/IRIS-Toolbox/issues/374#issuecomment-1533109634, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGCVKKQZA2THIVO7M2JQRBTXEJSGVANCNFSM6AAAAAAXUNVTMU . You are receiving this because you commented.Message ID: @.***>

guzmangt commented 1 year ago

Method 1 doesn't change anything, unfortunately.

Regarding the priors, I change things like E.alpha1 = {NaN, 0, 1, logdist.beta(P.alpha1,0.15)}; to E.alpha1 = {NaN, 0, 1, distribution.Beta.fromMeanStd(P.alpha1,0.15)}; following some of the example files.

jaromir-benes commented 1 year ago

...and the data you use in this estimation are exactly the same as in the older version?

On Wed, May 3, 2023 at 4:58 PM guzmangt @.***> wrote:

Method 1 doesn't change anything, unfortunately.

Regarding the priors, I change things like E.alpha1 = {NaN, 0, 1, logdist.beta(P.alpha1,0.15)}; to E.alpha1 = {NaN, 0, 1, distribution.Beta.fromMeanStd(P.alpha1,0.15)}; following some of the example files.

— Reply to this email directly, view it on GitHub https://github.com/IRIS-Solutions-Team/IRIS-Toolbox/issues/374#issuecomment-1533194179, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGCVKKTYP3KMDSZSWEUL5UDXEJXDDANCNFSM6AAAAAAXUNVTMU . You are receiving this because you commented.Message ID: @.***>

guzmangt commented 1 year ago

To the decimal, I've checked it multiple times to be 100% sure. All the inputs (data, parameters, settings, etc.) are exactly the same (except where I've needed to adapt them, like in the case of the priors). I have run out of things I can come up with to check.

jaromir-benes commented 1 year ago

Plot the prior distributions and visually check how far the final estimates are far from/close to their prior modes (not prior means, but modes). J

On Wed, May 3, 2023 at 5:58 PM guzmangt @.***> wrote:

To the decimal, I've checked it multiple times to be 100% sure. All the inputs (data, parameters, settings, etc.) are exactly the same (except where I've needed to adapt them, like in the case of the priors). I have run out of things I can come up with to check.

— Reply to this email directly, view it on GitHub https://github.com/IRIS-Solutions-Team/IRIS-Toolbox/issues/374#issuecomment-1533301300, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGCVKKT7JCOKP2QDIHRCEFDXEJ6EFANCNFSM6AAAAAAXUNVTMU . You are receiving this because you commented.Message ID: @.***>

guzmangt commented 11 months ago

Dear Jaromir,

I have gotten to a point in the code in which the input data and the priors are all equal, and most estimates come out very close--except for one parameter.

I am not sure this is related, but I have noticed a problem with one of the observable variables I am feeding the model which starts about 25 quarters later than the rest of the variables:

When using the 20151016 toolbox and MATLAB 2019a, the model produces a series for that variable for the non-observed period that looks pretty reasonable, whereas the 20221026 and MATLAB 2022a combination (which is the combination I was asking about above) estimates a very low level for the variable at the (non-observed) beginning of the sample.

Are you aware of any settings in the estimate function that I might be able to tweak to revert to the behavior of the old toolbox?

Best, Guzmán