-
`File ~\Python\PyTorch\RL\utils\optim.py:61, in Sophia.hutchinson(self, p, grad)
59 def hutchinson(self, p, grad):
60 u = torch.randn_like(grad)
---> 61 hessian_vector_product = t…
-
Hello, there was an error when I used the Sophia optimizer to train GPT3 with Megatron. The error point is that `grad` cannot be substituted into the optimizer with `require_grad = True` state to calc…
-
Some functions have sparse Jacobian or sparse Hessian and it can be usefull to obtain them as sparse matrices rather than accessing to the values through vector-jacobian or vector-hessian products fu…
-
Hey! :wave: I'm trying to use Hessian approximation, but setting it like (with JuMP)
```julia
m = Model(()->MadNLP.Optimizer(hessian_approximation=MadNLP.CompactLBFGS));
```
has no effect, it stil…
-
I'd like to apply Newton's method on a convex problem. When I try `jaxopt.ScipyMinimize` with `method="trust-exact"` I get the error:
```
ValueError: Hessian matrix is required for trust region ex…
-
It would be useful for JuMP to have a method to compute jacobian-vector and jacobian-matrix products. The current approach for computing jacobians can be interpreted as a jacobian-matrix products with…
-
as discussed by @lucashervier, it would be good to have a way to properly benchmark our inverse hessian vector product calculator on different parameter dimensions -- mnist, cifar, imagenet convolutio…
-
Hi @FFroehlich !
Many thanks again for your implementation!
I just had an idea, what one might want to do in the case of using Fides on a really large-scale system, where complete factorization o…
-
Hi all, I have isolated a relatively lightweight example where I find that HMC is *significantly* outperforming PT. I tested most variations supported in Pigeons: Slice Sampler & AutoMALA, fixed, vari…
-
**Mistake description**
Hi, on the Argyris page we can read:
"On each vertex: point evaluations,
point evaluations of derivatives in coordinate directions, and point evaluations of components of Ja…