search

huggingface / dataset-viewer

Lightweight web API for visualizing and exploring any dataset - computer vision, speech, text, and tabular - stored on the Hugging Face Hub
https://huggingface.co/docs/datasets-server
Apache License 2.0
639 stars 65 forks source link

Create pastJobs collection #2931

Closed severo closed 1 week ago

severo commented 1 week ago

It will be used to apply rate limiting on the jobs, based on the total duration in a window (see https://github.com/huggingface/dataset-viewer/issues/2279#issuecomment-2178655627).

severo commented 1 week ago

Just a question, should we delete all PastJobs entries for a dataset when it is being deleted

No, we have to keep it, because the budget for the dataset is still the same, even if the dataset is deleted for some time. It would be complex to remove it on deletion, additionally, because we delete the dataset before re-creating it when we update.