search

CNES / MAJA

Level-2A processor used for atmospheric correction and cloud-detection. The active repository is the one below, this one is kept to leave access to the older issues.
https://gitlab.orfeo-toolbox.org/maja/maja
Apache License 2.0
141 stars 25 forks source link

Directory of input files as symbolic links #98

Closed iliion closed 11 months ago

iliion commented 2 years ago

Hello, I am working in a cloud and I want to use S2-L1C files that are located in a virtualpath (in this case /eodata ) .

ls -ls /eodata/S2/MSI/LEVEL-1C/S2MSI1C/2021/11/06/

4 dr-xr-xr-x 2 nobody nogroup 4096 Jan  1  2018 S2B_MSIL1C_20211106T231129_N0301_R058_T57FUA_20211107T002549.zip
4 dr-xr-xr-x 2 nobody nogroup 4096 Jan  1  2018 S2B_MSIL1C_20211106T231129_N0301_R058_T57FUV_20211107T002549.zip
4 dr-xr-xr-x 2 nobody nogroup 4096 Jan  1  2018 S2B_MSIL1C_20211106T231129_N0301_R058_T57FVA_20211107T002549.zip
4 dr-xr-xr-x 2 nobody nogroup 4096 Jan  1  2018 S2B_MSIL1C_20211106T231129_N0301_R058_T57FVU_20211107T002549.zip
4 dr-xr-xr-x 2 nobody nogroup 4096 Jan  1  2018 S2B_MSIL1C_20211106T231129_N0301_R058_T57FVV_20211107T002549.zip
4 dr-xr-xr-x 2 nobody nogroup 4096 Jan  1  2018 S2B_MSIL1C_20211106T231129_N0301_R058_T57FWU_20211107T002549.zip
4 dr-xr-xr-x 2 nobody nogroup 4096 Jan  1  2018 S2B_MSIL1C_20211106T231129_N0301_R058_T57FWV_20211107T002549.zip
....
....

I have created a folder with symbolinks that point to the aforementioned S2-L1C files. 

 ls -ls  /workspace/L1C/32TQM/

total 48
4 lrwxrwxrwx 1 dockermanager dockermanager 173 Feb  3 09:50 S2A_MSIL1C_20211106T100221_N0301_R122_T32TQM_20211110T145618.SAFE -> /eodata/S2/MSI/LEVEL-1C/S2MSI1C/2021/11/06/S2A_MSIL1C_20211106T100221_N0301_R122_T32TQM_20211110T145618.zip/S2A_MSIL1C_20211106T100221_N0301_R122_T32TQM_20211110T145618.SAFE
4 lrwxrwxrwx 1 dockermanager dockermanager 173 Feb  3 09:50 S2A_MSIL1C_20211119T101321_N0301_R022_T32TQM_20211119T110331.SAFE -> /eodata/S2/MSI/LEVEL-1C/S2MSI1C/2021/11/19/S2A_MSIL1C_20211119T101321_N0301_R022_T32TQM_20211119T110331.zip/S2A_MSIL1C_20211119T101321_N0301_R022_T32TQM_20211119T110331.SAFE

Then I run maja using a docker image

docker run --rm --name "maja" -v /home/dockermanager/workspace/L1C:/maja/S2-L1C -v /home/dockermanager/workspace/L2A:/maja/S2-L2A -v /home/dockermanager/workspace/L1C/majadata/:/maja/majadata -w /maja maja startmaja -f folder.txt -t T32TQM -d 20211101 -e 20211231 --type_dem srtm -y
2022-02-03 10:54:07,721 [INFO ] =============This is Start_Maja v4.4.0==============
2022-02-03 10:54:07,729 [INFO ] Detecting input products...
Traceback (most recent call last):
  File "/maja/4.5.2/lib/python/StartMaja/Start_maja.py", line 594, in <module>
    type_dem=args.type_dem, skip_errors=args.skip_errors, userconf=args.userconf, max_l2_diff=args.max_l2_diff)
  File "/maja/4.5.2/lib/python/StartMaja/Start_maja.py", line 80, in __init__
    self.avail_input_l1, self.avail_input_l2 = self.get_all_available_products()
  File "/maja/4.5.2/lib/python/StartMaja/Start_maja.py", line 293, in get_all_available_products
    raise IOError("No L1C products detected for %s in %s" % (self.__site_info, self.path_input_l1))
OSError: No L1C products detected for tile 32TQM in /maja/S2-L1C/32TQM

Is it possible to "feed" maja with a list of symbolinc links of S2-L1C files, or am I trying this in vain?

What is the best way to work in a cloud environment?