-
At the moment I don't know if there is wide user interest, but feel free to contribute here.
See initial discussion with @timholy in https://github.com/JuliaDiff/AbstractDifferentiation.jl/pull/134
-
Hi,
I came across your implementation of the Influence Function and am interested in separately grabbing the Hessian that is needed for the calculations. Is there a way for me to grab it without ha…
-
Pearlmutter method only gives "good" value of hessian vector product in the first two iterations in conjugate gradient loop
-
It would be amazing to have a direct op to compute jacobians and hessians w.r.t. model parameters like we have for objax.Grad. I suppose that these would require an unreduced loss value (i.e. raise an…
-
Hello, there was an error when I used the Sophia optimizer to train GPT3 with Megatron. The error point is that `grad` cannot be substituted into the optimizer with `require_grad = True` state to calc…
-
Hi!
Thank you making the source code of your work available. I tried to use the library for an application involving a 3D network architecture, and ran into the following issue:
```
********** …
-
`File ~\Python\PyTorch\RL\utils\optim.py:61, in Sophia.hutchinson(self, p, grad)
59 def hutchinson(self, p, grad):
60 u = torch.randn_like(grad)
---> 61 hessian_vector_product = t…
-
Hi @FFroehlich !
Many thanks again for your implementation!
I just had an idea, what one might want to do in the case of using Fides on a really large-scale system, where complete factorization o…
-
### 🚀 The feature, motivation and pitch
Currently, the `torch.lobpcg` matrix-free eigensolver only supports fully materialized dense or sparse tensors as input. This makes it impossible to, for examp…
-
**Mistake description**
Hi, on the Argyris page we can read:
"On each vertex: point evaluations,
point evaluations of derivatives in coordinate directions, and point evaluations of components of Ja…