MPI_Select_Mock_Target - Githubissues

zhengyun0215 commented 3 years ago

Hi all , I try to use MPI_Select_Mock_Target to make mtl file. But I got this error:

ERROR:mpi_select_mock_targets:182:<module>: Pixels [28120] failed after 1.7 minutes

ERROR:mpi_select_mock_targets:185:<module>: Traceback (most recent call last):
  File "/global/common/software/desi/cori/desiconda/20200801-1.4.0-spec/code/desispec/master/py/desispec/parallel.py", line 453, in stdouterr_redirected
    yield # allow code to be run with the redirected output
  File "/global/common/software/desi/cori/desiconda/20200801-1.4.0-spec/code/desitarget/master/bin/mpi_select_mock_targets", line 173, in <module>
    targets_truth(params, output_dir=args.output_dir, seed=rankseeds[i],
  File "/global/common/software/desi/cori/desiconda/20200801-1.4.0-spec/code/desitarget/master/py/desitarget/mock/build.py", line 988, in targets_truth
    write_skies(output_dir, skytargets.as_array(), nside=nside, nsidefile=nside,
  File "/global/common/software/desi/cori/desiconda/20200801-1.4.0-spec/code/desitarget/master/py/desitarget/io.py", line 1210, in write_skies
    subpseed += int(hpxlist[0])
IndexError: invalid index to scalar variable.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/global/common/software/desi/cori/desiconda/20200801-1.4.0-spec/code/desitarget/master/bin/mpi_select_mock_targets", line 173, in <module>
    targets_truth(params, output_dir=args.output_dir, seed=rankseeds[i],
  File "/global/common/software/desi/cori/desiconda/20200801-1.4.0-spec/conda/lib/python3.8/contextlib.py", line 131, in __exit__
    self.gen.throw(type, value, traceback)
  File "/global/common/software/desi/cori/desiconda/20200801-1.4.0-spec/code/desispec/master/py/desispec/parallel.py", line 477, in stdouterr_redirected
    with open(fname) as infile:
FileNotFoundError: [Errno 2] No such file or directory: '/global/cscratch1/sd/yunzheng/BGSmock/data/runmock1/281/28120/build-64-28120.log_0'

Which means the sky.fits file couldn't be made and I couldn't make build-64-28120.log_0 in my catalog. It seems that I could run MPI_Select code last week but after updating, it failed. Hope you can give some help, thank you so much 😊

yaojian95 commented 3 years ago

I also have the same error during this step and by looking at the details I find a conflict between the input from targets_truthand the definition for the function write_skies.

In the definiton of targets_truth function (https://github.com/desihub/desitarget/blob/master/py/desitarget/mock/build.py#L796), in line 988, it uses the function write_skies and passes the value healpix to the argument hpxlist. The value healpix is a scalar produced by for healpix, healseed in zip(healpixels, healpixseeds): in the function targets_truth (line 890), while in the definition of write_skies(https://github.com/desihub/desitarget/blob/master/py/desitarget/io.py#L1090), the hpxlist should be a list. So I think it is this mismatch that causes the problem...

moustakas commented 3 years ago

@zhengyun0215 thanks for the crash report. I'll tackle this and related tickets (https://github.com/desihub/desitarget/issues/745#issuecomment-850769119) soon.

The most significant updates to the targeting code in support of the main survey have been implemented and then I plan on updating the targeting simulations / mocks.

moustakas commented 2 years ago

Fixed in https://github.com/desihub/desitarget/pull/788/commits/ae46447987fd10a2690ce9d2eee2825f372d5e88 in https://github.com/desihub/desitarget/pull/788.

@geordie666 this is the only place I touch non-mock code, but the change should be innocuous to production runs of desitarget.

desihub / desitarget

MPI_Select_Mock_Target #747