search

dlr-eoc / prosEO

prosEO – A Processing System for Earth Observation Data
GNU General Public License v3.0
14 stars 1 forks source link

BaseWrapper tries to download "Directory" input entry from Storage Manager #73

Closed tangobravo62 closed 4 years ago

tangobravo62 commented 4 years ago

The BaseWrapper erroneously tries to download a "Directory" input element from the Storage Manager and fails (as is to be expected):

15:06:04.715 [main] INFO de.dlr.proseo.basewrap.rest.RestOps - GET http://storage-mgr-service.default.svc.cluster.local:3000/proseo/storage-mgr/v0.1/productfiles?pathInfo=AUX_DATA%2FDEM%2F
15:06:04.724 [main] DEBUG de.dlr.proseo.basewrap.rest.RestOps - response = org.jboss.resteasy.client.jaxrs.engines.ApacheHttpClient4Engine$1@58594a11
15:06:04.725 [main] ERROR de.dlr.proseo.basewrap.BaseWrapper - Error retrieving input file AUX_DATA/DEM, HTTP status code 500

However the Storage Manager should not abort with status 500, but return 404 (Not Found).

tangobravo62 commented 4 years ago

Corrected. BaseWrapper now skips input entries, which are not flagged as Physical:

{"prosEO" : "A Processing System for Earth Observation Data"}
Starting base-wrapper with JobOrder file /proseo/data/joborders/2020/6/22/16/b2f4e9c6-187c-40e1-a008-fc55a7096a38.xml
16:37:46.829 [main] INFO de.dlr.proseo.basewrap.BaseWrapper - Checking 15 environment variables:
16:37:46.829 [main] INFO de.dlr.proseo.basewrap.BaseWrapper - ... JOBORDER_FS_TYPE = POSIX
[...]
16:37:47.383 [main] INFO de.dlr.proseo.basewrap.BaseWrapper - Skipping input entry of type DEM with filename type Directory
16:37:47.407 [main] INFO de.dlr.proseo.basewrap.BaseWrapper - Fetched 7 input files and prepared directories for 1 outputs -- Ready for processing using Container-JOF /usr/share/cpros-s2-l2a/1592843866.xml
16:37:47.459 [main] INFO de.dlr.proseo.basewrap.BaseWrapper - Starting Processing using command /usr/share/cpros-s2-l2a/1592843866.xml and local JobOrderFile: {}
[...]