Other suggestions for release

[falkamelung]

• Example data: for Pichincha, shall we add what is needed so that mintpy also works?
• Compilation:
  1. Write out the commands (e.g. python setup.py instead of a mysterious bash $MINOPY_HOME/docs/install
  2. When you get errors it is unclear whether the compilation was successful. Whey I run python setup.py I get the cryptic output below. It also should tell me that everything is installed already (if it is)

     python setup.py
     running build_ext
     copying build/lib.linux-x86_64-3.8/utils.cpython-38-x86_64-linux-gnu.so -> 
     copying build/lib.linux-x86_64-3.8/invert.cpython-38-x86_64-linux-gnu.so -> 

  3. To ensure that the conda-installed gcc is used we need something like

     make CC=$PYTHON3DIR/bin/x86_64-conda_cos6-linux-gnu-gcc
     alias gcc=$PYTHON3DIR/bin/x86_64-conda_cos6-linux-gnu-gcc

[yunjunz]

I would recommend separating jupyter notebooks from the code repo, as the former can be large in size, and not very track-change friendly.

It would be even better if you could use git rebase to remove the jupyter notebook from the minopy repo commit history, before more people are using it. ISCE-2 had this issue before.

[falkamelung]

Suggestions to improve "Brief description of processing steps and/or Jupyter Notebook:

• explain files produced by each step. What is in the "inverted/PATCHES' directory, e.g. what is in the files flag.npy mask_ps.npy phase_ref.npy quality.npy shp.npy and what does the extension npy stands for? Can they be examined to understand details of the processing? How are the PATCHES created? How would locate the patch covering your particular area of interest? ToDo
• also explain how u have implemented the writing of results and why you can restart the program at any time
• Explain briefly the files that are produced: phase_series.h5. Should/can the quality files get a more meaningful name, such as pta_tempCoherence? They are produced at the end. Briefly explain why.

• Quality files are now renamed to tempCoh_average and tempCoh_full.
• They are created for each patch but concatenated at the end

• the generation of ifgramStack.h5 should also be explained (if it is not already)
• After the phase_inversion step is completed an empty directory interferograms_delaunay shows up. Should this be created in a later step? EXPLAIN

OTHERS:

• Discuss with Yunjun: velocity versus phase
• can we make the section numbers in the notebook resemble the run_0* file numbers?
• info.py --slice gives the slc number. It would make more sense to show the date?

• this issue is because mintpy readfile does not support slc as a dataset,
• it has to be added to the function where it reads slices

 info.py inputs/slcStack.h5 --slice
slc-1
slc-2
slc-3
slc-4
slc-5
slc-6
slc-7
slc-8
slc-9
slc-10

• clarify verbose output of phase_inversion.py. What is a task? What is a job? what is index? Maybe with a graphic showing how the processed area is subdivided?

• HPC of phase_inversion.py: Why not many small jobs with short walltime instead of a too short walltime and timing out?

• The current design is such that it uses the full potential of the node.

• each patch is processed with one processor, so it is a waste of resources to

• submit each patch in a separate node when only one processor is used

• change numBursts to a different name (numUnit??)

• this issue is for job submission

//login3/scratch/05861/tg851601/HaitiSenDT142/minopy/run_files[1069] wc -l run_02_minopy_phase_inversion
646 run_02_minopy_phase_inversion

wc -l run_02_minopy_phase_inversion_{0,1,2,3,4,5,6,7,8,9,10,11,12,13}
    47 run_02_minopy_phase_inversion_0
    47 run_02_minopy_phase_inversion_1
    47 run_02_minopy_phase_inversion_2
    47 run_02_minopy_phase_inversion_3
    47 run_02_minopy_phase_inversion_4
    47 run_02_minopy_phase_inversion_5
    47 run_02_minopy_phase_inversion_6
    47 run_02_minopy_phase_inversion_7
    47 run_02_minopy_phase_inversion_8
    47 run_02_minopy_phase_inversion_9
    47 run_02_minopy_phase_inversion_10
    47 run_02_minopy_phase_inversion_11
    47 run_02_minopy_phase_inversion_12
    35 run_02_minopy_phase_inversion_13
   646 total

In other cases I get exactly one jobfile for each line in run_02_minopy_phase_inversion. So maybe there is a bug?

+ not a bug, it is related to the number of processors in each node, 
+ and how many you ask for in your template

Miami2SenAT48/minopy_vlarge/run_files[1081]  wc -l run_02_minopy_phase_inversion
18 run_02_minopy_phase_inversion
Miami2SenAT48/minopy_vlarge/run_files[1083] ls run_02*job | wc -l
18

• We may want to use PATCH_0009 and PATCH_0895 format instead of PATCH_9 for easier sorting and better ls returns.

• Done

• There was a question about the location of the logfile. It should be MiamiHighResSenAT48/log and not MiamiHighResSenAT48/minopy/run_files/log

• Yes but it should be in minopy directory to be independant from project dir. Done

• rename repo MiNoPy --> MinoPy ?? (ore rename package to MinloPy, MinliPy, MifucPy, MifucoPy = Miami InSAR Full Coherence software) MilipPy = Miami Insar software for Linking of the Interferometric Phase in Python. MipalPy Miami Insar PhAse Linking in Python. (Lets not do Fuc.... Pal is better.....) (MiphalPy) MilpPy MilPy Miami Insar Linking Phase in pYthon

• it will be nice to have minopy_run_files_list similar to minsar

• Done but named run_files_list

  run_files_list /scratch/05861/tg851601/qqunittestGalapagosSenDT128/run_files/run_01_unpack_topo_reference /scratch/05861/tg851601/qqunittestGalapagosSenDT128/run_files/run_02_unpack_secondary_slc /scratch/05861/tg851601/qqunittestGalapagosSenDT128/run_files/run_03_average_baseline /scratch/05861/tg851601/qqunittestGalapagosSenDT128/run_files/run_04_fullBurst_geo2rdr

• Read waterMask in the same way as MintPy. It need to understand

  minopy.load.waterMaskFile  = /work2/05861/tg851601/stampede2/insarlab/DEM/MiamiSenAT48/waterMask_large.h5  #[path2water_mask_file], optional

• Can we remove the discrepancy between the numbers in the minopyApp.cfg and the run_files?

  ########## 6. Unwrap parameters
  minopy.unwrap.removeFilter               = auto     # [yes, no], auto for yes, remove filter after unwrap
  minopy.unwrap.maxDiscontinuity           = auto     # (snaphu parameter) max phase discontinuity in cycle, auto for 1.2

  But run_06 is actually unwrap error correction....Clarity which options affect which run_file would be good. For the steps that are done by MintPy we can some comments, e.g.

  ########## run_06:  unwrap error correction: see smallbaselineApp.cfg step 4 ########
  ########## run_07:  network inversion: see smallbaselineApp.cfg step 5  invert_network   ########
  ########## run_08...: 

• We need two post-prrocessing option. Once is to save the S1_*_PS.he5 and the other to downsample. if people don't want high-res result. Lets talk about how to do this and whether it should be part of MinoPy.

[falkamelung]

And I wonder whether we should change the directory structure to facilitate the testing of different networks. Two options (there are many, I like the first) are:

minopy_inverted/
minopy/

minopy_sequential/inputs/ifgramStack.h5
minopy_delaunay/inputs/ifgramStack.h5

• on a related issues: what is an efficient approach to run using different options? Running minopyApp.py throws an error if minopy/inputs exists (although I am not sure what exactly causes the error).
• short_baseline_ifgs.txt is network dependent, right? Should it have the network type in the name?

[falkamelung]

Thank you for implementing most (or all?) suggestions