Call improvements and clarifications

[falkamelung]

• execute_runfiles.py logging: After I modified it, it creates two entries into /log. Please check.

• In job_defaults.cfg lets also specify the walltime and memory for process_rsmas.py (24:00) and execute_runfiles.py (8:00). Lets specify the walltimes for all named scripts. Lets move [Defaults] to the end and use them for job_submission.py, unless they are specified. SM: DONE

• need smallbaseline_wrapper.py that calls smallbaselineApp.py, email_results.py and has the typical submit options (--submit --walltime) FA for SM: DONE

• email_results.py should be included at end of smallbaseline_wrapper.py and ingest_insarmaps.py FA for SM: DONE

• All RSMAS codes (download_rsmas.py, smallbaseline_wrapper.py, ingest_insarmaps.py, hazards*) should use the smart argparse strategy (https://github.com/geodesymiami/rsmas_insar/issues/175). Lets get started with it for the job_submission options (--submit and --walltime). Equivalently the following should be working:

  
  ingest_insarmaps.py /nethome/famelung/insarlab/infiles/famelung/TEMPLATES/EifelSenAT15.template

ingest_insarmaps.py /nethome/famelung/insarlab/infiles/famelung/TEMPLATES/EifelSenAT15.template --submit

job_submission.py /projects/scratch/insarlab/famelung/EifelSenAT15/run_files/run_14_insarmaps

job_submission.py /projects/scratch/insarlab/famelung/EifelSenAT15/run_files/run_14_insarmaps --memory 4000 --walltime 1:00 --queuename general --outdir /projects/scratch/insarlab/famelung/EifelSenAT15/run_files

SM for FA: DONE except for job_submission.py as it has different arguments and needs it's own argparse

- run_files: If there is only one line in the run_file the `_0` should be avoided (it should be `run_13_smallbaseline.job` instead of `run_13_smallbaseline_0.job`) 
SM: DONE

Others:
- `process_rsmas.py --step process`  is a poor choice. Need ability for --start isce (or --start ifgrams) ) and --start smallbaseline
- need some mapping between run numbers what they do so that `execute_runfiles.py --start smallbaseline` , etc, is possible

General comments:
- the mixing of topsStack run_files and rsmas_insar run_files does not improve clarity. Should we consider two separate run_file directories?  
 - I don't see the advantage of using the run_file concept for everything (download, smallbaselineApp and ingest_insarmaps). Outsiders don't understand. Could we call the rsmas_code alternatively the old way from python?). Calling a job for data/dem download is problematic as we need to ssh into the head node.
 - Maybe as follows:  it would create a file `commands.txt` (this is sort-of concatenation of the run_files) which then would be executed by `run_sequential.py commands.txt` :

download_rsmas.py $TE/KrakatoaSenDT47.template --submit dem_rsmas.py $TE/KrakatoaSenDT47.template. --submit create_runfiles.py $TE/KrakatoaSenDT47.template. --submit run_parallel.py /runfiles/run_ --submit smallbaseline_wrapper.py $TE/KrakatoaSenDT47.template. --submit ingest_insarmaps.py $TE/KrakatoaSenDT47.template. --submit generate_hazard_products.py $TE/KrakatoaSenDT47.template. --submit

Alternatively, `create_runfiles.py` and `run_parallel.py ` are included in a function `generate interferograms.py  $TE/KrakatoaSenDT47.template --submit`

ADDED Jul 22:
- There is no out_mintpy.e , out_mintpy.o which allow to see the mintpy progress (bpeek does not work well, which is part of the reason why out_pysar was produced))
SM: DONE

ADDED Aug 4:
- A problem of the run_files is that nodes may not have internet access.   dem_rsmas.py  had this problem, I think.    Sometimes run_files are not created but I did not track this down. Maybe it could not download the orbits? Lets make sure we understand which processing steps need internet access.
- We should have an `out_create_runfiles.o` in the PROJECTDIR for an easy access to the coverage. An enhancement would be to generate a plot of the coverage information together with the boundingBox (`plot_coverage_boundingbox.py`).  Missing coverage of the boundingBox continues to be the main error source.  I am not asking you to do this,  but please consider to add this when you come access the same problem.
SM: DONE
- The topsStack.py (previously sentinelstack,py) command should show up in the /log file (for debugging).
SM: DONE
- In addition an `summary_create_runfiles.txt` showing the main coverage information would be good.
SM:following can be found in out_stackSentinel.o

boundingBox: ..... from *template file ....

---

The overlap region among all dates (based on the preview kml files): South North East West 39.501541 40.511597 75.117966 78.198845

---

All dates (109) dates covering the bbox (109) All SLCs will be coregistered to : 20141014

[mirzaees]

• execute_runfiles.py logging: After I modified it, it creates two entries into /log. Please check.

what do you mean by two entries? I see each command printed out once

[falkamelung]

Sorry, forgot to commit. Done now @mirzaees

I added

 command_line = os.path.basename(sys.argv[0]) + ' ' + ' '.join(sys.argv[1:])
    message_rsmas.log(inps.work_dir, command_line)

There were some other message_rsmas.log calls below. Not sure they are needed. Also not sure I added it to the right place, but I get the --submit and --walltime options which I always forget.

[mirzaees]

• execute_runfiles.py logging: After... SM for FA: DONE
• In job_defaults.cfg lets also specify the walltime... SM for FA: DONE
• need smallbaseline_wrapper.py that calls... SM for FA: DONE
• email_results.py should be included at end ... SM for FA: DONE
• All RSMAS codes ... SM for FA: DONE except for job_submission.py as it has different arguments and needs it's own argparse
• run_files: If there is only one line... SM for FA: DONE
• write something on the wiki ... SM for FA: In Progress
• process_rsmas.py --step process is a poor choice... SM for FA: DONE
• need some mapping between run numbers... SM for FA: Not a good idea
• the mixing of topsStack run_files ... SM for FA: DONE
• I don't see the advantage of using the run_file concept... SM for FA: DONE
• Maybe as follows: it would create a file commands.txt ... SM for FA: DONE . all steps included as --step <> argument in process_rsmas.py
• There is no out_mintpy.e , out_mintpy.o which ... SM for FA: DONE
• We should have an out_create_runfiles.o ... SM for FA: DONE
• The topsStack.py (previously sentinelstack,py) command... SM for FA: DONE
• In addition an summary_create_runfiles.txt ... SM for FA: DONE following can be found in out_stackSentinel.o

  boundingBox:
  ..... from *template file ....
  *****************************************
  The overlap region among all dates (based on the preview kml files):
  South   North   East  West 
  39.501541 40.511597 75.117966 78.198845
  *****************************************
  All dates (109)
  dates covering the bbox (109)
  All SLCs will be coregistered to : 20141014

[falkamelung]

Thank you