mnwhite
commented
1 year ago I'm not sure the article really explains or justifies the method well. It begins by saying that the function is expensive to compute, but then proposes a method that samples the parameter space in a semi-disorganized way. The article also says that the derivative "isn't known", not that it's not differentiable. It's not clear whether it's meant for R^N problems and the graphics are just there as readable examples, or if it's just for optimization on R-- the author never says. Sentences like this also make me very suspicious of whether the author knows what they're talking about (emphasis added): "After a certain number of iterations, we’re destined to arrive at a global minima, unless the function’s shape is very bizarre..." In higher dimensional parameter spaces, the amount of queried points needed to generate an updated surrogate function would be very large.
More generally, I don't think many structural papers use a grid search for their parameters. A variety of search and optimization methods are used, but a grid search would only be done in 1 or 2 dimensions as an initial check on parameters about which the econometrician knows very little.
On Thu, Jun 22, 2023 at 10:35 AM Sebastian Benthall < @.***> wrote:
cc @alanlujan91 https://github.com/alanlujan91
This article outlines Bayesian Optimization, a method of computing $x$ that optimizes $f(x)$ when $f(x)$ is hard to sample and non-differentiable.
It looks like this might be an improvement on our current grid-based way of doing structural estimation.
[I learned about this technique in a conversation with other colleagues about surrogate models; this accomplishes some of what surrogate models accomplish, but the algorithm is simpler and requires, I believe, somewhat less infrastructural overhead because it depends on Gaussian Processes rather than full fledged neural network architecture. ]
— Reply to this email directly, view it on GitHub https://github.com/econ-ark/HARK/issues/1291, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADKRAFO5IRKNIBJS65MSE6DXMRJZZANCNFSM6AAAAAAZQJ2RQY . You are receiving this because you are subscribed to this thread.Message ID: @.***>
sbenthall
llorracc
cc @alanlujan91
This article outlines
Bayesian Optimization, a method of computing $x$ that optimizes $f(x)$ when $f(x)$ is hard to sample and non-differentiable.https://towardsdatascience.com/the-beauty-of-bayesian-optimization-explained-in-simple-terms-81f3ee13b10f
It looks like this might be an improvement on our current grid-based way of doing structural estimation.
[I learned about this technique in a conversation with other colleagues about surrogate models; this accomplishes some of what surrogate models accomplish, but the algorithm is simpler and requires, I believe, somewhat less infrastructural overhead because it depends on Gaussian Processes rather than full fledged neural network architecture. ]