msimet / Stile

Stile: the Systematics Tests In Lensing pipeline
BSD 3-Clause "New" or "Revised" License
9 stars 6 forks source link

#60 Multiple tract/visit runs in HSC module #71

Closed msimet closed 8 years ago

msimet commented 9 years ago

This pull request implements two new HSC scripts, StileMultiVisit.py and StileMultiTract.py, that allow us to analyze multiple visits or tracts together as one instead of running through each tract or visit individually. These work exactly as StileVisit.py or StileTract.py but produce one output file per SysTest no matter how many input visits or tracts are given.

Changes involve:

HironaoMiyatake commented 9 years ago

I do not think #60 work. Instead, we switched to #60a, so should we make a pull request for #60a?

msimet commented 9 years ago

Oh, right! I forgot about that.

I did some work on this branch to make it functional because I forgot about #60a, and looking at a diff I think this is more correct than #60a was (though #60a worked). Does this run for you? If so, I think I'd prefer to stay with this branch.

HironaoMiyatake commented 9 years ago

Sorry that I did not carefully check "#60a"! It works fine on may side, so it's almost reedy to merge.

I still have another concern. I found that the filename is too long to be saved when running all tracts in some field. What do you think the best way to treat this?

msimet commented 9 years ago

I added a quick fix for this (25ac3ad) and it seems to still work--does it fix your too-long-filename problem?

HironaoMiyatake commented 8 years ago

I got the the following error. Is it okay to replace os.pathconf('sys_test_adapters.py', 'PC_NAME_MAX') with os.pathconf('.', 'PC_NAME_MAX')?

[miyatake@master reduction_for_galaxy_shape_tests_20150904]$ StileMultiTract.py /lustre/Subaru/SSP --rerun=miyatake/SSP3.8.5_20150725_bfcorr:miyatake/stile_test --id tract=8280^8281^8282^8283^8284^8285^8521^8522^8523^8524^8525^8526^8764^8765^8766^8767^8768 filter=HSC-I  -c "sys_tests.names=['Rho1']"   --clobber-config 
Traceback (most recent call last):
  File "/lustre/work/miyatake/Stile/bin/StileMultiTract.py", line 3, in <module>
    from stile.hsc.base_tasks import MultiTractSingleEpochStileTask
  File "/lustre/work/miyatake/Stile/stile/hsc/base_tasks.py", line 28, in <module>
    max_path_length = os.pathconf('sys_test_adapters.py', 'PC_NAME_MAX')
OSError: [Errno 2] No such file or directory: 'sys_test_adapters.py'
msimet commented 8 years ago

I don't think that should cause a problem!

HironaoMiyatake commented 8 years ago

Okay, now it works. Please merge this to master.