don't fully download the files in the viewers. - Githubissues

hyparam / hyperparam-cli

Hyperparam local dataset viewer

https://hyperparam.app

MIT License

2 stars 0 forks source link

don't fully download the files in the viewers. #10

Open severo opened 3 weeks ago

severo commented 3 weeks ago

For example, this JSON file https://huggingface.co/datasets/BackpropBuff/CCMatrix.en-es/blob/main/CCMatrix.en-es_1.json has a size of 41GB

The viewer will download it all (don't click!): https://huggingface.co/spaces/severo/parquet-viewer?url=https%3A%2F%2Fhuggingface.co%2Fdatasets%2FBackpropBuff%2FCCMatrix.en-es%2Fblob%2Fmain%2FCCMatrix.en-es_1.json

It would be better to stream through it while scrolling.