search

dlr-eoc / prosEO

prosEO – A Processing System for Earth Observation Data
GNU General Public License v3.0
14 stars 1 forks source link

Ingestor deletes component products without checking for existing product files #85

Closed tangobravo62 closed 3 years ago

tangobravo62 commented 4 years ago

When deleting a composed product, all component products are deleted as well (as intended). However there is apparently no check, whether the component products have product files attached (preventing their deletion):

prosEO (PTM)> product show L1B_______
       153   c6d3731b-7604-45a7-ae71-a1d8ce6dbbf5   2019-11-04T09:00:00.200000  2019-11-04T10:41:10.300000
       175   db04e6ec-9445-4035-97a9-2e43bc8b9ee2   2019-11-04T12:22:20.400000  2019-11-04T14:03:30.500000
       164   6026d05e-07b8-47dd-9a8b-58140f4a75ac   2019-11-04T10:41:10.300000  2019-11-04T12:22:20.400000
prosEO (PTM)> product delete 153
(I2956) Product with database ID 153 deleted
prosEO (PTM)> product delete 175
(I2956) Product with database ID 175 deleted
prosEO (PTM)> product delete 164
(I2956) Product with database ID 164 deleted
prosEO (PTM)> product show L1B_PART1
(E2950) No products found for given search criteria
prosEO (PTM)> product show L1B_PART2
(E2950) No products found for given search criteria

L1B product 175 had two components L1B_PART1 and L1B_PART2 with IDs 176 and 177 respectively. These are still present in the Storage Manager:

localhost:testdata thomas$ find /Volumes/MOBILE_DATA/prosEO_storage/proseodata/17[67] -name 'PTM_TEST_L1B*' -type f
/Volumes/MOBILE_DATA/prosEO_storage/proseodata/176/PTM_TEST_L1B________20191104T122220_20191104T140330_03002_77_0.1.0_20200701T131446/PTM_TEST_L1B_PART2_20191104T122220_20191104T140330_03002_77_0.1.0_20200701T131446.nc/PTM_TEST_L1B_PART2_20191104T122220_20191104T140330_03002_77_0.1.0_20200701T131446.nc
/Volumes/MOBILE_DATA/prosEO_storage/proseodata/177/PTM_TEST_L1B________20191104T122220_20191104T140330_03002_77_0.1.0_20200701T131446/PTM_TEST_L1B_PART1_20191104T122220_20191104T140330_03002_77_0.1.0_20200701T131446.nc/PTM_TEST_L1B_PART1_20191104T122220_20191104T140330_03002_77_0.1.0_20200701T131446.nc
tangobravo62 commented 3 years ago

Fixed:

prosEO (PTM)> product show L1B_______
Database ID UUID                                   Sensing Start              Sensing Stop              
        102 0e750693-47e2-4aa2-a7b5-eef161742acb   2019-11-04T09:00:00.200000 2019-11-04T10:41:10.300000
        115 086b0bf7-d9f6-4de9-836c-88e7a1f9ef55   2019-11-04T10:41:10.300000 2019-11-04T12:22:20.400000
        128 ebba8807-4720-494e-89a2-c9cd107aed2d   2019-11-04T12:22:20.400000 2019-11-04T14:03:30.500000

prosEO (PTM)> product show L1B_PART1
Database ID UUID                                   Sensing Start              Sensing Stop              
        103 b9ff8561-3fca-495e-9d7e-130cd89378c4   2019-11-04T09:00:00.200000 2019-11-04T10:41:10.300000
        116 a2d439d7-d581-4be0-9f14-cb406cef93c9   2019-11-04T10:41:10.300000 2019-11-04T12:22:20.400000
        129 bde5ddcd-eebb-4753-9d8f-396920e93d07   2019-11-04T12:22:20.400000 2019-11-04T14:03:30.500000

prosEO (PTM)> product show -v L1B_PART1 103
---
id: 103
version: 1
uuid: "b9ff8561-3fca-495e-9d7e-130cd89378c4"
missionCode: "PTM"
productClass: "L1B_PART1"
[...]
productFile:
- id: 136
  version: 1
  productId: 103
  processingFacilityName: "localhost"
  productFileName: "PTM_TEST_L1B_PART1_20191104T090000_20191104T104110_03000_2_0.1.0_20210420T131308.nc"
  [...]

prosEO (PTM)> product delete 102
(E2806) Command failed (cause: (E2026) Product with ID 102 (or some component product) has existing files and cannot be deleted)