search

rigoudyg / climaf

CliMAF - a Climate Model Analysis Framework - doc at : http://climaf.readthedocs.org/
Other
16 stars 7 forks source link

Is there already a function to get the size of a dataset ? #191

Open jypeter opened 3 years ago

jypeter commented 3 years ago

Maybe there is already a function computing the size of all the files in a dataset, but I have not found it in the documentation. This is not displayed in summary

I have done it the following way in my script, but it could be an interesting information for some users

    if show_ds_details:
        # Get more details about the files
        files_query = ds(model=model_name, **default_query).explore('resolve')
        files_list = files_query.baseFiles().split()
        nb_files = len(files_list)
        files_sizes = list(map(os.path.getsize, files_list))
        files_size_mb = sum(files_sizes) / 1024. / 1024.
        print('%sNb files = %3i - Total size = %7.2f (Mb)' % (lpad, nb_files, files_size_mb))

and I'm getting the last line of

MPI-ESM1-2-LR       [1850-2349]
                    Nb years =   500
                    Nb files =  25 - Total size =  194.92 (Mb)