search

bilby-dev / bilby

A unified framework for stochastic sampling packages and gravitational-wave inference in Python. Note that we are currently transitioning from git.ligo.org/lscsoft/bilby, please bear with us!
https://bilby-dev.github.io/bilby/
MIT License
57 stars 63 forks source link

bilby saves the absolute outdir to the result object when a relative path is passed to run_sampler #815

Open bilby-bot opened 1 month ago

bilby-bot commented 1 month ago

In GitLab by @git.ligo:sylvia.biscoveanu on Aug 26, 2024, 16:58

Regardless of whether an absolute or relative path is passed to run_sampler, bilby saves the absolute path to result.outdir. This causes issues when running on condor with OSG + file transfer, as the job running on a remote node is trying to write to a specific location on CIT that doesn't exist from its perspective.

bilby-bot commented 1 month ago

In GitLab by @git.ligo:sylvia.biscoveanu on Aug 26, 2024, 17:04

It looks like this is intentional: https://git.ligo.org/lscsoft/bilby/-/blob/master/bilby/core/result.py?ref_type=heads#L469 I guess one reason for this is to give the option to read in the result file and then save plots without needing to be in the same directory as the original run. I'm not sure how to resolve this favorably for both that use case and the OSG issue described above.

bilby-bot commented 1 month ago

In GitLab by @git.ligo:sylvia.biscoveanu on Aug 28, 2024, 20:02

Traceback (most recent call last):
  File "/srv/mixed_background_3g_relbin.py", line 173, in <module>
    main(**vars(args))
  File "/srv/mixed_background_3g_relbin.py", line 123, in main 
    noise_result.plot_corner()
  File "/cvmfs/software.igwn.org/conda/envs/igwn-py310/lib/python3.10/site-packages/bilby/core/utils/plotting.py", line 65, in wrapper_decorator
    return func(*args, **kwargs)
  File "/cvmfs/software.igwn.org/conda/envs/igwn-py310/lib/python3.10/site-packages/bilby/core/result.py", line 1315, in plot_corner
    outdir = self._safe_outdir_creation(kwargs.get('outdir'), self.plot_corner)
  File "/cvmfs/software.igwn.org/conda/envs/igwn-py310/lib/python3.10/site-packages/bilby/core/result.py", line 1644, in _safe_outdir_creation
    raise FileMovedError("Can not write in the out directory.\n"
bilby.core.result.FileMovedError: Can not write in the out directory.
Did you move the here file from another system?
Try calling plot_corner with the 'outdir' keyword argument, e.g. plot_corner(outdir='.')
bilby-bot commented 1 month ago

In GitLab by @git.ligo:michael.williams on Sep 5, 2024, 14:04

I'm slightly confused by this, if a relative path is given and this code is executed on the remote note, won't the absolute path point to somewhere on that node?

bilby_pipe passes run_sampler a relative path and that seems to work on the OSG, though I might be missing something.

bilby-bot commented 1 month ago

In GitLab by @git.ligo:sylvia.biscoveanu on Sep 19, 2024, 01:59

Yes, that is exactly the issue I am running into. I'm running jobs on the OSG, which means that the outdir saved in the result object is some remote node directory to which I cannot write. It's possible that bilby_pipe has a hard-coded work-around to this.