search

dlr-eoc / prosEO

prosEO – A Processing System for Earth Observation Data
GNU General Public License v3.0
14 stars 1 forks source link

Ingestor: Deleting product files does not delete database entry #80

Closed tangobravo62 closed 4 years ago

tangobravo62 commented 4 years ago

Deleting a product file using the REST method DELETE ingest/{processingFacility}/{productId} deletes the file from the Storage Manager, but does not remove the entry in the metadata database, resulting in a failure of a subsequent product deletion:

2020-06-26 09:05:53.468 TRACE 1 --- [nio-8080-exec-7] d.d.p.i.rest.IngestControllerImpl        : >>> deleteProductFile(254, telekom-otc)
2020-06-26 09:05:53.469 TRACE 1 --- [nio-8080-exec-7] d.d.p.ingestor.rest.ProductIngestor      : >>> deleteProductFile(254, ProcessingFacility [name=telekom-otc, description=Telekom_OTC, processingEngineUrl=https://proseo-k8s-gate.de/, storageManagerUrl=https://proseo-k8s-gate.de/api/v1/namespaces/default/services/storage-mgr-service:service/proxy/proseo/storage-mgr/v1/])
2020-06-26 09:05:53.479 DEBUG 1 --- [nio-8080-exec-7] o.s.web.client.RestTemplate              : HTTP DELETE https://proseo-k8s-gate.de/api/v1/namespaces/default/services/storage-mgr-service:service/proxy/proseo/storage-mgr/v1/products?pathInfo=s3://proseo-data-001/254/S2B_OPER_MSI_L2A_TL_DLR__20200623T163323_A015250_T27QUU_N02.14.tar
[...]
2020-06-26 09:05:53.620 DEBUG 1 --- [nio-8080-exec-7] o.s.web.client.RestTemplate              : Response 200 OK
[...]
2020-06-26 09:05:53.620  INFO 1 --- [nio-8080-exec-7] d.d.p.ingestor.rest.ProductIngestor      : (I2068) Product file S2B_OPER_MSI_L2A_TL_DLR__20200623T163323_A015250_T27QUU_N02.14.tar for product with id 254 deleted

2020-06-26 09:05:53.797 TRACE 1 --- [nio-8080-exec-8] d.d.p.i.rest.ProductControllerImpl       : >>> deleteProductById(254)
2020-06-26 09:05:53.797 TRACE 1 --- [nio-8080-exec-8] d.d.proseo.ingestor.rest.ProductManager  : >>> deleteProductById(254)
2020-06-26 09:05:53.801 ERROR 1 --- [nio-8080-exec-8] d.d.proseo.ingestor.rest.ProductManager  : (E2026) Product with ID 254 has existing files and cannot be deleted

The existence of the product file is confirmed in the CLI:

prosEO (S2)> product file show -v 254
---
- id: 367
  version: 1
  productId: 254
  processingFacilityName: "telekom-otc"
  productFileName: "S2B_OPER_MSI_L2A_TL_DLR__20200623T163323_A015250_T27QUU_N02.14.tar"
  auxFileNames: []
  filePath: "s3://proseo-data-001/254"
  storageType: "S3"
  fileSize: 87316480
  checksum: "12D8234440087B1CF099C4050DC71305"
  checksumTime: "2020-06-23T17:13:14.284026"
  zipFileName: null
  zipFileSize: null
  zipChecksum: null
  zipChecksumTime: null
tangobravo62 commented 4 years ago

Fixed (removal of product file from product was missing):

prosEO (PTM)> product file delete 155 localhost
(I2963) Product file for product database ID 155 and processing facility localhost deleted

prosEO (PTM)> product file show -v 155
(E2964) Product with database ID 155 has no files