search

isce-framework / isce2

InSAR Scientific Computing Environment version 2
Other
516 stars 253 forks source link

bug in topsStack `logging/logging.conf parameters: too small filesize #143

Open falkamelung opened 4 years ago

falkamelung commented 4 years ago

For large data sets and running ~100 jobs in parallel, occasionally the Logging error below appears (it may also say isce.log.3, etc). Allowing for larger file size in isce/defaults/logging/logging.conf fixes the problem ( I did args=('isce.log','a',1000048576,5) ),

I don't understand the purpose of the size restriction, that is why I don't do a PR, hoping you guys can fix it. If you are not sure please just put a big number.

I see a lot of Depreciation Warning (see below). That could contribute to the file size issue. Thank you

log erros:

cat run_7_pairs_misreg_0_36.e
--- Logging error ---
Traceback (most recent call last):
  File "/work/05861/tg851601/stampede2/test/dev2/rsmas_insar/3rdparty/miniconda3/lib/python3.7/logging/handlers.py", line 70, in emit
    self.doRollover()
  File "/work/05861/tg851601/stampede2/test/dev2/rsmas_insar/3rdparty/miniconda3/lib/python3.7/logging/handlers.py", line 166, in doRollover
    os.remove(dfn)
FileNotFoundError: [Errno 2] No such file or directory: '/scratch/05861/tg851601/HanumangarhSenDT34/run_files/isce.log.5'

Depraciation warning message:

cat run_7_pairs_misreg_0_34.e
/work/05861/tg851601/stampede2/test/dev2/rsmas_insar/sources/isceStack/isce2/contrib/stack/topsStack/estimateAzimuthMisreg.py:144: VisibleDeprecationWarning: Passing `normed=True` on non-uniform bins has always been broken, and computes neither the probability density function nor the probability mass function. The result is only correct if the bins are uniform, when density=True will produce the same result anyway. The argument will be removed in a future version of numpy.
  hist, bins = np.histogram(val, 50, normed=1)
/work/05861/tg851601/stampede2/test/dev2/rsmas_insar/sources/isceStack/isce2/contrib/stack/topsStack/estimateRangeMisreg.py:208: VisibleDeprecationWarning: Passing `normed=True` on non-uniform bins has always been broken, and computes neither the probability density function nor the probability mass function. The result is only correct if the bins are uniform, when density=True will produce the same result anyway. The argument will be removed in a future version of numpy.
  hist, bins = np.histogram(val, 50, normed=1)
hfattahi commented 4 years ago

Thanks for reporting the issue. I suggest issue a PR.