Error during etimation - Githubissues

rayhan1993 commented 6 years ago

Hi, I experienced this error when I was running the optimization. While the estimation of parameter was done, error messages came out for other output tab. Not sure what is going on. Thanks

java.lang.NullPointerException at ec.tss.dfm.DfmResults.calcSmoothedSignal(DfmResults.java:332) at ec.tss.dfm.DfmResults.calcSignalProjections(DfmResults.java:965) at ec.tss.dfm.DfmResults.getSignalProjections(DfmResults.java:956) at be.nbb.demetra.dfm.output.FitSignalDataView.toCollection(FitSignalDataView.java:253) at be.nbb.demetra.dfm.output.FitSignalDataView.updateChart(FitSignalDataView.java:217) at be.nbb.demetra.dfm.output.FitSignalDataView.lambda$new$1(FitSignalDataView.java:134) at java.beans.PropertyChangeSupport.fire(Unknown Source) at java.beans.PropertyChangeSupport.firePropertyChange(Unknown Source) at java.beans.PropertyChangeSupport.firePropertyChange(Unknown Source) at java.awt.Component.firePropertyChange(Unknown Source) at be.nbb.demetra.dfm.output.FitSignalDataView.setDfmResults(FitSignalDataView.java:200) at be.nbb.demetra.dfm.output.FitSignalDataItemFactory$1.getView(FitSignalDataItemFactory.java:51) at be.nbb.demetra.dfm.output.FitSignalDataItemFactory$1.getView(FitSignalDataItemFactory.java:47) at ec.ui.view.tsprocessing.ComposedProcDocumentItemFactory.getView(ComposedProcDocumentItemFactory.java:90) at ec.ui.view.tsprocessing.ProcDocumentViewFactory$View.getView(ProcDocumentViewFactory.java:63) at ec.ui.view.tsprocessing.DefaultProcessingViewer.showComponent(DefaultProcessingViewer.java:357) at ec.ui.view.tsprocessing.DefaultProcessingViewer.lambda$new$0(DefaultProcessingViewer.java:77) at java.beans.VetoableChangeSupport.fireVetoableChange(Unknown Source) at java.beans.VetoableChangeSupport.fireVetoableChange(Unknown Source) at org.openide.explorer.ExplorerManager$1AtomicSetSelectedNodes.checkAndSet(ExplorerManager.java:274) at org.openide.explorer.ExplorerManager$1AtomicSetSelectedNodes.run(ExplorerManager.java:299) at org.netbeans.modules.openide.util.DefaultMutexImplementation.readAccess(DefaultMutexImplementation.java:193) at org.netbeans.modules.openide.util.DefaultMutexImplementation$1R.run(DefaultMutexImplementation.java:1037) at org.netbeans.modules.openide.util.DefaultMutexImplementation.doWrapperAccess(DefaultMutexImplementation.java:1054) at org.netbeans.modules.openide.util.DefaultMutexImplementation.readAccess(DefaultMutexImplementation.java:183) at org.openide.util.Mutex.readAccess(Mutex.java:260) at org.openide.explorer.ExplorerManager.setSelectedNodes(ExplorerManager.java:304) at org.openide.explorer.ExplorerManager$1SetExploredContextAndSelection.run(ExplorerManager.java:399) at org.netbeans.modules.openide.util.DefaultMutexImplementation.readAccess(DefaultMutexImplementation.java:193) at org.netbeans.modules.openide.util.DefaultMutexImplementation$1R.run(DefaultMutexImplementation.java:1037) at org.netbeans.modules.openide.util.DefaultMutexImplementation.doWrapperAccess(DefaultMutexImplementation.java:1054) at org.netbeans.modules.openide.util.DefaultMutexImplementation.readAccess(DefaultMutexImplementation.java:183) at org.openide.util.Mutex.readAccess(Mutex.java:260) at org.openide.explorer.ExplorerManager.setExploredContextAndSelection(ExplorerManager.java:420) at org.openide.explorer.view.BeanTreeView.selectionChanged(BeanTreeView.java:149) at org.openide.explorer.view.TreeView.callSelectionChanged(TreeView.java:833) at org.openide.explorer.view.TreeView$TreePropertyListener.run(TreeView.java:1475) at org.netbeans.modules.openide.util.DefaultMutexImplementation$1Exec.run(DefaultMutexImplementation.java:973) at org.openide.nodes.Children$ProjectManagerDeadlockDetector.execute(Children.java:1938) at org.netbeans.modules.openide.util.DefaultMutexImplementation.postRequest(DefaultMutexImplementation.java:979) at org.netbeans.modules.openide.util.DefaultMutexImplementation.postRequest(DefaultMutexImplementation.java:942) at org.netbeans.modules.openide.util.DefaultMutexImplementation.postReadRequest(DefaultMutexImplementation.java:308) at org.openide.util.Mutex.postReadRequest(Mutex.java:371) at org.openide.explorer.view.TreeView$TreePropertyListener.valueChanged(TreeView.java:1444) at javax.swing.tree.DefaultTreeSelectionModel.fireValueChanged(Unknown Source) at javax.swing.tree.DefaultTreeSelectionModel.notifyPathChange(Unknown Source) at javax.swing.tree.DefaultTreeSelectionModel.setSelectionPaths(Unknown Source) at javax.swing.tree.DefaultTreeSelectionModel.setSelectionPath(Unknown Source) at javax.swing.JTree.setSelectionPath(Unknown Source) at javax.swing.plaf.basic.BasicTreeUI.selectPathForEvent(Unknown Source) at javax.swing.plaf.basic.BasicTreeUI$Handler.handleSelection(Unknown Source) at javax.swing.plaf.basic.BasicTreeUI$Handler.mousePressedDND(Unknown Source) at javax.swing.plaf.basic.BasicTreeUI$Handler.mousePressed(Unknown Source) at java.awt.AWTEventMulticaster.mousePressed(Unknown Source) at java.awt.AWTEventMulticaster.mousePressed(Unknown Source) at java.awt.AWTEventMulticaster.mousePressed(Unknown Source) at java.awt.AWTEventMulticaster.mousePressed(Unknown Source) at java.awt.AWTEventMulticaster.mousePressed(Unknown Source) at java.awt.AWTEventMulticaster.mousePressed(Unknown Source) at java.awt.Component.processMouseEvent(Unknown Source) at javax.swing.JComponent.processMouseEvent(Unknown Source) at java.awt.Component.processEvent(Unknown Source) at java.awt.Container.processEvent(Unknown Source) at java.awt.Component.dispatchEventImpl(Unknown Source) at java.awt.Container.dispatchEventImpl(Unknown Source) at java.awt.Component.dispatchEvent(Unknown Source) at java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source) at java.awt.LightweightDispatcher.processMouseEvent(Unknown Source) at java.awt.LightweightDispatcher.dispatchEvent(Unknown Source) at java.awt.Container.dispatchEventImpl(Unknown Source) at java.awt.Window.dispatchEventImpl(Unknown Source) at java.awt.Component.dispatchEvent(Unknown Source) at java.awt.EventQueue.dispatchEventImpl(Unknown Source) at java.awt.EventQueue.access$500(Unknown Source) at java.awt.EventQueue$3.run(Unknown Source) at java.awt.EventQueue$3.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source) at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source) at java.awt.EventQueue$4.run(Unknown Source) at java.awt.EventQueue$4.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source) at java.awt.EventQueue.dispatchEvent(Unknown Source) at org.netbeans.core.TimableEventQueue.dispatchEvent(TimableEventQueue.java:159) at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source) at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source) at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source) at java.awt.EventDispatchThread.pumpEvents(Unknown Source) at java.awt.EventDispatchThread.pumpEvents(Unknown Source) at java.awt.EventDispatchThread.run(Unknown Source)

Liedo commented 6 years ago

Could you post or send me the workfile?

What I would need to reproduce your error is a file with extension xml together with an accomanying folder the the same name. Also mention the name of the model (you probably have different models in your workspace) that is giving you the problem. This way I will be able to access your model and see -- a) the data you are using and whether it is compatible with the model, and -- b) the estimation options you are using (EM algorithm or numerical optimization).
Also, please mention the version of JDemetra+ and the plugin you are using

rayhan1993 commented 6 years ago

Hi, unfortunately I am unable to upload the data due to company firewall :(

I probably need help just with the factor loadings

I am currently using v2.2.1, dynamic factor model nowcasting plugin
I ran a 3 common factor VAR lag 3 model (named workbase), using EM optimization (preliminary and final, both 100 iterations)
The data is compatible because I can run it with a uniform factor loading of Monthly data
I believe the issue is with factor loading. Certain combinations of factor loadings seem to work. May I ask how should I be deciding the factor loading whether to use M,Q or YoY, the guide seems to suggest this is totally up to the users discretionary)

Thank you.

Liedo commented 6 years ago

What the options Q and YoY are doing is to place restrictions on the factor loading structure. If your data is uniform: they are all noisy indicators of some underlying concepts, it could work by setting M and selecting a number of factors equal to the number of concepts. But I have no idea of the kind of application you have in mind. If it is an application related to economics, I am more likely to be able to help you. If you are mixing montly growth rates with year-on-year growth rates, I suggest the following:

select M for those series that refer to (or are correlated with) monthly growth rates or first differences
select YoY for the variables that represent year-on-year growth rates or series that even if they represent a monthly growth rate, they have a strong expectations component (it is very soft data). In this case, you can also use QoQ. Typical examples are some of the business surveys and confidence indicators published by Eurostat (by the way, we are investgating alternative loading structure to better accomodate expectations in a model consistent way, but this will not be publicly available anytime soon)

Liedo commented 6 years ago

If you want, you can send your example by email to me and I will try to give you some tips. Otherwise, this paper provides a more coherent explanation of how build models with the kind of heterogeneous (e.g. mixed frequencies, bad measurements, expectations, different growth rates) data that we typically encounter in real time applications https://www.nbb.be/doc/ts/publications/wp/wp331en.pdf

rayhan1993 commented 6 years ago

Hi Liedo,

Unfortunately I cant email out attachments as well. I have moved past the estimate part but experienced an error for simulation. After running for a while, a pop up box suddenly appears and says:

ec.tstoolkit.timeseries.TsException: Invalid id Hopefully this can be resolved just by looking at the error message. Thanks once again.

Ray

Liedo commented 6 years ago

This looks like a different problem (@maggima, do you have any clue?). In this case, it looks like you are looking for a time series that does not exist. It is possible that when you updated your input data, you changed the name of at least one of the series by mistake.

I would suggest re-doing your model from scratch and pay some extra attention to your input data files. Here you have a very brief explanation of how JDemetra+ loads your data. Understanding this may help you to avoid the kind of problems that appear when you change the location of your input data, etc..... https://github.com/nbbrd/jdemetra-nowcasting/wiki/Load-Data

rayhan1993 commented 6 years ago

Hi Liedo,

Is there a way to export all the data from output and forecast simulation into excel/csv format? Thank you.

Ray

Liedo commented 5 years ago

Hi Rayhan, I responded to your question in a separate issue, which I opened myself on your behalf. This way I can close this one, which is unrelated. Thank you for all your comments and inquiries

nbbrd / jdemetra-nowcasting

Error during etimation #8