Tracking the test case to bifurcate the sampling error

[jlperla]

This is all about finding the MWE to see what changed.

• On julia 1.6 instantiate new HMCExmaples branch, make sure it will work with a simple short RBC kalman run. If not, then I can help debug it
• Then with print_level = 1 https://github.com/HighDimensionalEconLab/HMCExamples.jl/blob/main/src/turing_models.jl#L126 should do an output of every sample from turing and then show in https://github.com/HighDimensionalEconLab/HMCExamples.jl/blob/main/src/turing_models.jl#L131
  • So I think the basic approach is to just run something like

    julia --project --threads auto -O1 bin/fit_FVGQ_1_kalman.jl --results_path ./results/main_rbc_1_FVGQ --overwrite_results true --num_samples 1000 --print_level 1

    and then maybe pipe the output to a file? with > output.txt or just copy/paste the output at the end.

• With this, we shhoud be able to see all of the theta that are proposed and then see a println("Perturbation failed / Infinite variance with retcode $(sol.retcode)")

Then with this

• Switch over entirely to the DSSM package itself, as HMCExamples is no longer required.
• Pick one that looks like it probably shouldn't have been rejected (we can come back to thte list) and then switch over to the DSSM entirely and make sure we can manually call generate_perturbation and generate_perturbation_derivatives! on the theta and get it to return a retcode != Success.

---

Keep that test code around because we will want to check soemthing equivalent on old code.

• Then switch to the old branch of DSSM back when things "worked" in january, instantiate the pacakges, etc.
• Now take the theta that you thought shouldn't have been rejected and that you were able to manually trigger a failure on with the DSSM for the later version, and try the same thing with the old DSSM code.
• If that also triggers an error, maybe try a different theta from your list of rejected thetas from HMCExamples.

Once we find the theta which (1) triggers an error in the new DSSM; and (2) doesn't trigger in the new DSSM, we can take the next step in bifurcating the differences.

[jlperla]

Closing since we can reproduce the wrong gradients.