search

awi-response / macs_processing

all elements (scripts, workflows, descriptions, ...) needed for processing macs images for publication
0 stars 0 forks source link

pipelining postprocessing #21

Closed initze closed 3 months ago

initze commented 1 year ago

setup variable set SITE=<SITE_NAME>

run full postprocessing python 02_Postprocessing.py -s %SITE%.py -pc both && python 03_MoveProducts.py -s %SITE%.py && python 04_ArchiveData.py -s %SITE%.py -d

Archive to F: python 02_Postprocessing.py -s %SITE%.py -pc both && python 03_MoveProducts.py -s %SITE%.py && python 04_ArchiveData.py -s %SITE%.py -a F:\MACS_Data_Storage -d

Post Processing clip Point Cloud Tiles python 07_Clip_PointClouds_from_Archive.py -i G:\MACS_Data_Storage\NA_AnaktuvukRiverFire_20190722_7cm_03.tar

Potential issues

  1. stripy DSM output
    • could happen with NIR and RGB pc mismatch
    • use only one of them
  2. Crashing IDW interpolation
    • might use only NIR point clod to remove file size
initze commented 1 year ago

Access to sftp

ssh @hsm.dmawi.de /hs/Proj-f/p_macsprocessing/backup_processing_files

initze commented 1 year ago

Workflow for reanalysis

  1. pull from Archive to disk (F:\MACS_Data_Storage) via sftp (filezilla)
  2. extract files to working dir (faster than directly) python 05_Pull_Backup.py -a F:\MACS_Data_Storage -t E:\MACS_Batch_Processing -d NA_KetikFire_04_20190727_7cm -r
  3. rename
  4. run pix4d
  5. postprocessing