aleximmer / Laplace

Laplace approximations for Deep Learning.
https://aleximmer.github.io/Laplace
MIT License
432 stars 61 forks source link

Adding prior precision in `KronLaplace` is broken #171

Closed wiseodd closed 3 weeks ago

wiseodd commented 2 months ago

Traceback:

Traceback (most recent call last):
  File "/Users/agustinuskristiadi/Projects/Laplace/examples/huggingface_example.py", line 186, in <module>
    print(f'[LoRA-LLM] The predictive tensor is of shape: {lora_la(X_test).shape}.')
                                                           ^^^^^^^^^^^^^^^
  File "/Users/agustinuskristiadi/Projects/Laplace/laplace/baselaplace.py", line 774, in __call__
    f_mu, f_var = self._glm_predictive_distribution(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/agustinuskristiadi/miniforge3/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/agustinuskristiadi/Projects/Laplace/laplace/baselaplace.py", line 887, in _glm_predictive_distribution
    f_var = self.functional_variance(Js)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/agustinuskristiadi/Projects/Laplace/laplace/baselaplace.py", line 1309, in functional_variance
    return self.posterior_precision.inv_square_form(Js)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/agustinuskristiadi/Projects/Laplace/laplace/baselaplace.py", line 1294, in posterior_precision
    return self.H * self._H_factor + self.prior_precision
           ~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
  File "/Users/agustinuskristiadi/Projects/Laplace/laplace/utils/matrix.py", line 70, in __add__
    raise ValueError('Can only add Kron to Kron.')
ValueError: Can only add Kron to Kron.