Update Examples to New UX - Githubissues

neuralmagic / sparseml

Libraries for applying sparsification recipes to neural networks with a few lines of code, enabling faster and smaller models

Apache License 2.0

2.07k stars 148 forks source link

Update Examples to New UX #2301

Closed Satrat closed 5 months ago

Satrat commented 6 months ago

Updating existing examples and adding a new example for quantizing a 2:4 sparse model with a custom dataset.

Evaluation

Ran sparseml.evaluate [MODEL] -d wikitext -i lm-evaluation-harness on the output of each example and the baselines, perplexity results below

base llama2_7b dense ultrachat: 10.10
llama2_7b w8a8 channelwise weights, dynamic per token activations: 10.17
llama2_7b w4a16 channelwise weights: 11.09
baseline llama2_7b cnn_dailymail 2:4 sparsity: 13.35
llama2_7b 2:4 sparsity w4a16 channelwise weights: 17.31

review-notebook-app[bot] commented 6 months ago

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB