read swath in loop

[kameshvinjamuri]

hi I am reading slstr l1b product, using satpy by the following

slstr_sawth = 'directory of all folders containing swath folders'
if __name__ == '__main__':
    files = find_files_and_readers(base_dir=slstr_swath, reader='slstr_l1b', start_time=datetime.datetime(2017, 6, 2), end_time=datetime.datetime(2017, 6, 3))
    scn = Scene(filenames=files, reader='slstr_l1b')

when I do the above, I am reading all swath folders on the dates, but I want to iterate through the loop through each folder and extract outpout, and save. is there any easy way to find the swath folders one by one and save the output?

[djhoese]

Could you describe your directory structure more? It sounds like all you want is:

from glob import glob

for swath_dir in glob("dir_of_all_swaths/*"):
    files = find_files_and_readers(base_dir=swath_dir, reader="slstr_l1b")
    scn = Scene(filenames=files, reader="slstr_l1b")

Note I have no experience with this reader and its files so let me know if I'm completely wrong. Most people familiar with this reader might have started their holidays already though so we might have to wait a while for a better answer.

[kameshvinjamuri]

hi, the code you mention is also effective for one swath. Maybe I should explain the structure bit better...

the swath files are as follows ...

/home/usr/slstr/s3a_20170602_unique_id/files.nc
/home/usr/slstr/s3a_20170602_unique_id/files.nc
/home/usr/slstr/s3a_20170602_unique_id/files.nc
/home/usr/slstr/s3a_20170602_unique_id/files.nc

the codes above take base_dir which is till /home/usr/slstr/ but unfortunately not /home/usr/slstr/s3a_20170602_unique_id/

hope this is not reader specific

[djhoese]

If the glob string in my code is /home/usr/slstr/* does that not work? Wouldn't that make each iteration operate on one swath directory at a time?

[kameshvinjamuri]

If the glob string in my code is /home/usr/slstr/* does that not work? Wouldn't that make each iteration operate on one swath directory at a time?

not exactly! the base_dir=swath_dir working with /home/usr/slstr/ but not /home/usr/slstr/s3a_20170602_unique_id/

[simonrp84]

I'm not sure I understand the question fully, but would something like this be what you need?

slstr_swath = 'directory of all folders containing swath folders'
if __name__ == '__main__':
    dir_list = glob(f'{slstr_swath}/*')
    for cur_dir in dir_list:
       if not os.path.isdir(cur_dir):
           continue
       files = find_files_and_readers(base_dir=cur_dir, reader='slstr_l1b', start_time=datetime.datetime(2017, 6, 2), end_time=datetime.datetime(2017, 6, 3))
       scn = Scene(filenames=files, reader='slstr_l1b')

[kameshvinjamuri]

I'm not sure I understand the question fully, but would something like this be what you need?

slstr_swath = 'directory of all folders containing swath folders'
if __name__ == '__main__':
    dir_list = glob(f'{slstr_swath}/*')
    for cur_dir in dir_list:
       if not os.path.isdir(cur_dir):
           continue
       files = find_files_and_readers(base_dir=cur_dir, reader='slstr_l1b', start_time=datetime.datetime(2017, 6, 2), end_time=datetime.datetime(2017, 6, 3))
       scn = Scene(filenames=files, reader='slstr_l1b')

hi, not exactly!! the base_dir in find_files_and_readers is only happy with one folder outside the swath directories like ~/slstr_swath/ but not happy with ~/slstr_swath/uniq_swath_id

[djhoese]

Are your "unique_id" directories the directories ending in ".SEN3"? If so, then I think you can avoid using find_files_and_readers and instead do:

from glob import glob

for swath_dir in glob("dir_of_all_swaths/*.SEN3"):
    scn = Scene(filenames=glob(os.path.join(swath_dir, "*.nc")), reader="slstr_l1b")

That is, I expect swath_dir to be the path that ends in ".SEN3" which should mean the result of the filenames= glob be all of the .nc files in that swath directory.

[kameshvinjamuri]

hi, @djhoese that does work, thanks and cheers both @simonrp84 @djhoese