search

pprp / Pruner-Zero

Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for LLMs
https://arxiv.org/abs/2406.02924
MIT License
71 stars 6 forks source link

Missing the parameter X for W_metric? #6

Open cquxl opened 3 months ago

cquxl commented 3 months ago

In prune.py, find out if W_metric is missing X,should it be engine.forward(W,G,X)?

1722996516687

pprp commented 2 months ago

In fact, these code is only used for inference, which is correspond to the equation found in our paper.

As you can see, X is just part of the search space, and we actually have tried it before. If you want to search further, you can modify the forward function of engine to incorporate the X.

cquxl commented 2 months ago

But your engine.forward has 3 arguments, W,G,X. Do you mean I need to modify the forward function, i.e. remove X? Because I get an error when running your code directly: TypeError: GPTree.forward() missing 1 required positional argument: 'X'. If engine.forward only has two arguments, would it be best to clarify this on github so we can get fair and correct results?