Dimensionality problem with FluctuationsTwoAsset.ipynb

[kossacoff]

Hi all,

I can't run TwoAsset.ipynb because there is a problem in FluctuationsTwoAsset.ipynb (I think in the SGU_solver function). The invertibility condition is violated and the the command output is this:

SGU_solver The number of states is 62 Elapsed time is 1.2540298000000005 seconds. Use Schmitt-Grohe-Uribe algorithm AE[xprime uprime] = B[x u] A = (dF/dxprimek dF/duprime), B = -(dF/dx dF/du) Computing Jacobian: F1 = DF/DXprime F3 = DF/DX Total number of parallel blocks: 11. Block number: 0 done. Block number: 1 done. Block number: 2 done. Block number: 3 done. Block number: 4 done. Block number: 5 done. Block number: 6 done. Block number: 7 done. Block number: 8 done. Block number: 9 done. Block number: 10 done. Elapsed time is 261.0893924 seconds. Computing Jacobian F2 - DF/DYprime Total number of parallel blocks: 12. Block number: 0 done. Block number: 1 done. Block number: 2 done. Block number: 3 done. Block number: 4 done. Block number: 5 done. Block number: 6 done. Block number: 7 done. Block number: 8 done. Block number: 9 done. Block number: 10 done. Block number: 11 done. Elapsed time is 536.816385 seconds. nk is equal to 59 Warning: no local equilibrium exists, critical eigenvalue shifted to: 0.9777074908967675 Warning: invertibility condition violated

I haven't touched/changed nothing in the original code, but I'm using the sequential implementation. If I use the other function, my notebook just run and run, with no answer or output... I'm just trying to understand the code to use it for my master degree thesis, but I can't understand why this is happening (notice that nk, which must be equal to the number of states -62-, is lower -59-).

Hope someone can help me. Thanks in advance!

[llorracc]

@kossacoff, thanks for bringing this to our attention.

These things typically happen because of subtle incompatibilities in requirements (like python 3.7 vs 3.8).

@camriddell is working on trying to fix these kinds of problems right now. He's away for a couple of weeks but perhaps can look into this when he returns.

[llorracc]

@kossacoff,

Did you make sure to create an environment that has the right versions of the underlying software?

You need to do pip install -r requirements.txt and you need to make sure your version of python is the right one.

I've just rerun both the OneAsset and TwoAsset models and was able to get them to work, with the right environment (@camriddell, I can export the environment for you).

This repo is scheduled to become one of our official REMARKs soon, which should launch in a preconfigured environment, as a way of getting around these kinds of configuration problems.

[kossacoff]

Hi, @llorracc. Thanks for your answers.

I'm not so fluent in Python yet, but yesterday I learned a lot about environments, trying to create one as you recommended. In fact, I create an environment with Python 3.6.12 (following runtime.txt, is this the correct Python version?) and install the requirements in requirements.txt. Then I exported to my notebook. But I still have the same problem if I run the code with the sequential SGU_solver; now, if I run the alternative solver function, the problem appears in the multiprocessing modul/library. I think the problem is in these lines:

Inside [line 1062] the for bl in range(0, blocks): loop...

    range_= range(bl*packagesize, min(packagesize*(bl+1), mpar['numstates']))
    cc = np.zeros((mpar['numcontrols'], 1))
    ss = np.zeros((mpar['numstates'], 1))
    p1 = Process(target = FF_1_3, args = (range_, Xss, Yss, Gamma_state, indexMUdct, indexVKdct, par, mpar, grid, targets, Copula, P_H, aggrshock, Fb, packagesize, bl, ss, cc, out_DF1,  out_DF3, out_bl))
    procs1.append(p1)
    p1.start()
    print(f'Block number: {str(bl)}')    

It returns the following error: BrokenPipeError: [Errno 32] Broken pipe.

Any idea how to solve it? Thanks again!

[llorracc]

I'm guessing you are running on a Windows machine (not Mac or linux)?

See if you can figure out how to fix it by looking at what you get when you Google "BrokenPipeError: [Errno 32] Broken pipe python windows"

[kossacoff]

Yes, I'm running on a Windows. Yesterday I've been searching about that but I couldn't find something helpful, I think I'll continue looking how to solve it... by the way, I saw there is a .gitignore document in the main folder. Is it necessary to install it like the requirements text? Thanks again!

[llorracc]

@camriddell, @alanlujan91, @MridulS: Does any of you know anything about this "BrokenPipeError: [Errno 32] Broken pipe" problem that crops up when this repo is run on Windows?

PS. This is slated to become a regular REMARK, so we need to fix this problem anyway.

[kossacoff]

Finally, I could run the code on Windows writing an specific environment and using the sequential SGU solver function. The problem is still in the alternative SGU solver function, in the multiprocessing module, since the Process class is used. I found this, maybe is useful to understand how to solve the problem:

• Broken Pipe Error in Python
• Multiprocessing module

Hopefully, I think is enough to create the correct environment and use the sequential solver if this repo is run on Windows.

Thank you very much for your time and help, @llorracc!

[llorracc]

@kossacoff, delighted to hear you got it running!

Could you help us out by giving an explicit code snippet for creating the proper environment for Windows?

[llorracc]

P.S. It might run without modification if you were to use the Windows subsystem for linux for everything. Or if you were to run it in a VirtualBox virtual machine as described in econ-ark-tools

[kossacoff]

Sure. I sequentially run the following lines in the Conda prompt before running the codes in Jupyter Notebook:

conda create --name blenv python=3.6.12 -y conda activate blenv conda install pip pip install -r C:...\requirements.txt conda env export > \Users...\blenv.yaml pip install --user ipykernel python -m ipykernel install --user --name=blenv

Once I had the kernel available in Jupyter, I changed the default kernel to the one generated above for all the codes and run.

[llorracc]

Thanks, I've confirmed that your steps work on my own captive Windows machine that I keep for testing such things.

@camriddell, or @MridulS or @alanlujan91, how do we need to change the requirements.txt file (or what do we need to replace it with) to robustify this REMARK?