allegroai / clearml

ClearML - Auto-Magical CI/CD to streamline your AI workload. Experiment Management, Data Management, Pipeline, Orchestration, Scheduling & Serving in one MLOps/LLMOps solution
https://clear.ml/docs
Apache License 2.0
5.43k stars 643 forks source link

Make Dataset Preview optional #1227

Open pedroconceicao opened 3 months ago

pedroconceicao commented 3 months ago

In Datasets Details there is a Preview section that shows a small portion of the dataset. In the case is made of images it will show a sample of those, if it is made of a csv it will show a sample of the rows of that file, etc. Example:

image

However, I am currently dealing with sensitive data that cannot and must not be shared and shown.

Inspecting the Dataset class I found that this preview is generated in the method upload() as seen in the snippet below: https://github.com/allegroai/clearml/blob/2ac637bfc6a0d521773b1488a066b5b093ae234c/clearml/datasets/dataset.py#L661

I would like to propose the addition of an argument to the class Dataset or the method upload() in the Dataset class that disables the preview.

For example:

    def upload(
        self,
        ...
        preview=True
    ):

    if preview:
       self._report_dataset_preview()

where the user can set this value to False thus disabling the dataset preview.

Edit: I am currently using WebApp: 1.12.1-397 • Server: 1.12.1-397 • API: 2.26

ainoam commented 3 months ago

@pedroconceicao This makes total sense :) Would you like to issue a PR?

pedroconceicao commented 3 months ago

Thanks @ainoam ! I'll submit a PR soon.

pollfly commented 2 weeks ago

Hey @pedroconceicao! v1.6.2 is now out, supporting making dataset previews optional.