[Task]: Cleanup/Removal of unused assets from PCM repo

[collinss-jpl]

Checked for similar tasks

Yes - I've already checked

Describe the task

There are several areas of the OPERA PCM repo containing code or config files that have not been touched since the initial import of the NISAR PCM repository. This implies that said assets could be safely removed to help simplify the repo and better promote maintainability.

This ticket proposes that the following file paths can be removed without impact to the rest of OPERA PCM:

• commons/constants/constants.py : None of the constants are actually used anywhere in the code and are mostly NISAR-related
• commons/constants/short_info_msg.py : Out of the constants in this file, only NO_EXPIRED_STATE_CONFIGS is referenced once in timer/timer_handler.py, so this single constant could be redefined there and short_info_msg.py can be removed entirely
• conf/sds/files/test/pge : All these test files seem specific to the NISAR PGEs. Even if these tests were being run, there's no way they would be passing, so this package should be removed.
• extractor/__init__.py The file iteslf should not be removed, rather, the contents of the file still reference NISAR and should be deleted
• extractor/DayOfYearRostExtractor.py : This extractor class is not used anywhere in OPERA PCM, and inherits from a class we don't actually have defined anywhere (RostFilenameMetExtractor)
• extractor/TemplateMetExtractor.py : This extractor class is not used anywhere in OPERA PCM
• opera-sds-pcm/hello_world : We have never really needed this as an example, so it could be removed. Additionally, the following associated files can also be removed:
  • docker/hysds-io.json.hello_world
  • docker/hysds-io.json.hello_world-gpu
  • docker/job-spec.json.hello_world
  • docker/job-spec.json.hello_world-gpu
  • conf/sds/files/test/submit_hello_world_job.py
  • conf/sds/files/test/submit_hello_world_job-gpu.py
• ingest/INGEST_STAGED : This should be able to be removed since we no longer utilize the Input Staging Location (ISL) buckets. Additionally, the following associated files can also be removed:
  • docker/hysds-io.json.INGEST_STAGED
  • docker/job-spec.json.INGEST_STAGED
• on_demand/ : This job has not been updated (or utilized) since Release 1, since which the notion of how to submit an "on-demand" PGE job has changed significantly, so this should be removed.
• purge_ISL/ : Since we no longer utilize the ISL buckets, this job can be removed. Additionally, the following associated files can also be removed:
  • tests/unit/purge_ISL/test_purge_isl.py
  • timer/ : The timer_handler job seems to be unused and could be removed. Additionally, the following associated files can also be removed:
  • docker/hysds-io.json.timer_handler
  • job-spec.json.timer_handler
  • util/xml2json.py : We have never used this module since its import from NISAR PCM
  • run_pge.sh : This script has never been used since its import from NISAR PCM

Prerequisites or Requirements

1.
2.
3.
...

Expected Deliverables

1.
2.
3.
...

Additional Context

No response

[collinss-jpl]

@hhlee445 @chrisjrd @philipjyoon , if y'all have a chance could you have a look over the list of files/directories I identified as candidates for removal and let me know if you see anything that should not be removed, or if you know of any other files that you know we are not using or have not needed since the initial import from NISAR? Thanks!

[hhlee445]

@collinss-jpl Thanks for checking unnecessary files for OPERA. All files you listed above can be removed for OPERA.

[philipjyoon]

I concur