Choice of optimizer

[thodson-usgs]

Adam

• Fastest
• better for low N, but can fail for large N

• Adagrad

• Default for pymc
• Slower then ADAM
• But more reliable

  mean_field = pm.fit(method='advi', n=200_000, obj_optimizer=pm.adagrad_window(learning_rate=.001, n_win=20))