actris-cloudnet / cloudnetpy

Python package for Cloudnet data processing
MIT License
34 stars 26 forks source link

Allow list input for Level1b conversion #80

Closed martin-rdz closed 9 months ago

martin-rdz commented 1 year ago

It would be convenient, if the functions for Level 1b conversion would accept not only daily files or directories containing only files from one day, but also lists of files. E.g. data that resides in monthly folders can be filtered in the code calling the conversion.

Just as example for mira.mira2nc, this could look like:

  with TemporaryDirectory() as temp_dir:
      if type(raw_mira) is list or os.path.isdir(raw_mira):
          mmclx_filename = f"{temp_dir}/tmp.mmclx"
          if type(raw_mira) is list:
              valid_filenames = sorted(raw_mira)
          else:
              valid_filenames = utils.get_sorted_filenames(raw_mira, ".mmclx")
          valid_filenames = utils.get_files_with_common_range(valid_filenames)
          variables = list(keymap.keys())
          concat_lib.concatenate_files(
              valid_filenames,
              mmclx_filename,
              variables=variables,
              allow_difference=["nave", "ovl"],
          )
      else:
          mmclx_filename = raw_mira

If there are no general objections, I could also file a pull request.

Cheers, Martin

tukiains commented 1 year ago

We have discussed about this and it would be good if all the conversion functions would accept the same input. To keep the existing functionality, maybe it's best to modify the functions so that they would accept either a single file, list of files, or a directory path (although, I am not sure if some instruments always provide single files).

Thus, I have no objections if you start with mira2nc and create a PR. :+1:

tukiains commented 9 months ago

We will harmonize all the high-level API functions at some point.