What's 'white_box'?

[dbl001]

In `tuned_lens/notebooks/layer_ablation.ipynb:

from white_box.data import (
    chunk_and_tokenize,
    compute_nats_to_bpb_ratio,
    silence_datasets_messages
)

ModuleNotFoundError: No module named 'white_box'

Where do I find

dataset = Dataset.from_json("/mnt/ssd-1/nora/lens-test.jsonl")

[levmckinney]

White box is the old name for this repository. We still need to finish refactoring the notebooks to reflect some recent refactors. I believe lens-test.jsonl is just the test set of the pile see the documentation. Can you confirm @norabelrose?

[norabelrose]

White box is the old name for this repository. We still need to finish refactoring the notebooks to reflect some recent refactors. I believe lens-test.jsonl is just the test set of the pile see the documentation. Can you confirm @norabelrose?

Yes that's correct

[dbl001]

Thanks!
Where's 'tuned-lens'? E.g.

$ tuned-lens train gpt2 val.jsonl
    --dataset the_pile all \
    --split validation \
    --output ./gpt-2-lens

Any thoughts on device='MPS' for the GPU?

[levmckinney]

Once you install the application in should be on your path.