KindXiaoming / pykan

Kolmogorov Arnold Networks
MIT License
13.64k stars 1.2k forks source link

Cut off all the branches #149

Open plumv opened 1 month ago

plumv commented 1 month ago

When I first demonstrated using Kan for a demo, the actual execution of the code was not the same as in the sample. I found that when constructing nodes, some nodes can also be trimmed. I don't know if it's possible to optimize the pruning logic so that the minimum model can be achieved every time the pruning is done

Device: Mac Pro M1

Student: https://kindxiaoming.github.io/pykan/intro.html#hello-kan

plumv commented 1 month ago

Actual operating results

image image
AlessandroFlati commented 1 month ago

Could you print symbolic_formula[0][0] and/or share the notebook please?

plumv commented 1 month ago
image
AlessandroFlati commented 1 month ago

Ok, this adds valuable information for me. The point is that the network, altough "pruned", is still a forward connected graph - so even if you don't see the edges, they're actually there nonetheless. Their contribution is minimal, though. You can probably see it if you print out model.symbolic_formula(simplify=True)[0][0].

You're right in saying that if this was a graph network, it should actually prune the node (1, 0) and the edges (0,0,0), (0, 1, 0), (1, 0, 0). But for how is structured right now, it won't.

If you're willing, you can add this consideration inside the #123 or on a new Issue.

KindXiaoming commented 1 month ago

hi, you may want to try a higher threshold in model.prune(threshold=5e-2) (hopefully the first node will be pruned away). The default threshold is 1e-2, but according to the formula, the residual term has magnitude above 1e-2.