Usage of full paths in association files has been discussed multiple times in the past (e.g., JP-2038), and the pipeline currently prints a warning message that these associations should contain only filenames, not paths.
There are historical reasons for this, as the pipeline was not designed to handle path usage like this, and there have been cases where this caused problems for some users, in addition to making association files non-portable between users.
However, since this time the benefits of such usage have also become clear, allowing users to organize their data products more clearly (especially when combining across multiple observations) and easily produce and compare multiple versions of data products with different kinds of processing. As such, the default pipeline notebooks that STScI is developing use full paths for user at-home processing.
We should thus update the note to simply indicate that paths can sometimes cause issues for some users.
Old text: " 'expname' contains path but should only be a filename. All input files should be in a single directory, so no path is needed."
New text: "Input association file contains path information; note that this can complicate usage and/or sharing of such files."
stscijgbot-jp
commented
1 week ago
Issue JP-3731 was created on JIRA by David Law:
Usage of full paths in association files has been discussed multiple times in the past (e.g., JP-2038), and the pipeline currently prints a warning message that these associations should contain only filenames, not paths.
There are historical reasons for this, as the pipeline was not designed to handle path usage like this, and there have been cases where this caused problems for some users, in addition to making association files non-portable between users.
However, since this time the benefits of such usage have also become clear, allowing users to organize their data products more clearly (especially when combining across multiple observations) and easily produce and compare multiple versions of data products with different kinds of processing. As such, the default pipeline notebooks that STScI is developing use full paths for user at-home processing.
We should thus update the note to simply indicate that paths can sometimes cause issues for some users.
Old text: " 'expname' contains path but should only be a filename. All input files should be in a single directory, so no path is needed."
New text: "Input association file contains path information; note that this can complicate usage and/or sharing of such files."