try harder to find a finite initial point and an initial stepsize direction

[tpapp]

Transformations are great, but it is occasionally not feasible or practical to make the whole of ℝⁿ valid (cf #102).

The default warmup should try harder for the initial point, eg making N=100 random draws in the [-2,2]ⁿ hypercube (should be customizable).

Similarly, initial stepsize search should give up early if not being able to find finite steps with a particular momentum, and try another momentum.

[chenwilliam77]

Hi @tpapp,

Has there been any progress on this issue? Being able to specify a restricted location for the initial point would be very helpful for my current use case because during the warm up phase, the initial point chosen seems to be a very very low probability point, and the leapfrog integrator seems to have a difficult time "escaping" from this low probability region. In particular, some of the parameters are stuck very close to the upper bound I imposed on them, and short of making this upper bound much closer to the true valuing generating the pseudo-data I made, I don't seem to get a reasonable parameter choice. While it's not a problem to restrict the parameter space for the initial point, it'd be sub-optimal to make tight restrictions on the parameter space for the actual estimation in real applications, where I don't a priori know where the "true" parameters are.

Best,

William

[tpapp]

No, I did not make progress on this. In my experience the difficulty happens with models that are ill-behaved for some reason so finding a good initial point helps little.

You can always use eg Optim.jl and then pass the initial point manually, skipping the optimization step.

[tpapp]

I am closing this because this can (and should be) be addressed outside DynamicHMC. Ideally with a more NUTS-friendly model, but if all else fails then a quasi-random or similar exploration of a hypercube (but that fails quickly in higher dimensions).