search

Blealtan / efficient-kan

An efficient pure-PyTorch implementation of Kolmogorov-Arnold Network (KAN).
MIT License
3.49k stars 306 forks source link

RuntimeError: false INTERNAL ASSERT FAILED #4

Open csinva opened 1 month ago

csinva commented 1 month ago

Getting this error when running test_simple_math.py. Any idea how to resolve it?

  File "/home/chansingh/test/kan.py", line 131, in curve2coeff
    solution = torch.linalg.lstsq(
               ^^^^^^^^^^^^^^^^^^^
RuntimeError: false INTERNAL ASSERT FAILED at "../aten/src/ATen/native/BatchLinearAlgebra.cpp":1539, please report a bug to PyTorch. torch.linalg.lstsq: (Batch element 0): Argument 6 has illegal value. Most certainly there is a bug in the implementation calling the backend library.

(pytorch is up-to-date, version '2.3.0+cu121', python 3.11)

pecanjk commented 1 month ago

Me, too. But the examples/mnist.py can work correctly.

csinva commented 1 month ago

Worked okay when I just lowered the learning rate / switched the optimizer...

Blealtan commented 1 month ago

That one happens somewhat randomly when reconstructing the splines on the new grid (performing update_grid)... Possibly some numerical failure in the Least Square optimization. I also encountered that in some cases. To workaround that, consider pre-normalize input into [-1, 1], and avoid calling update_grid.