file_path vs extra_file_path

[martenson]

Touched at https://github.com/galaxyproject/galaxy/pull/3918

It is my impression these are the same thing and Galaxy supports both. What is the recommended standard?

ping @galaxy-iuc/iuc

[peterjc]

Quoting @natefoo on galaxy-dev http://dev.list.galaxyproject.org/Running-jobs-as-real-user-and-extra-file-path-td4586608.html#a4598557

On Mon, Apr 30, 2012 at 4:47 PM, Nate Coraor nate@bx.psu.edu wrote:

On Apr 25, 2012, at 9:39 AM, Louise-Amélie Schmitt wrote:

Hi everyone,

I just wanted to ask how the extra_file_path is handled in case of job running as the real user since the file_path is only writable by the galaxy user. Any clue?

Hi L-A,

There are actually two dataset attributes for accessing the extra files directory, 'files_path' and 'extra_files_path'.

'extra_files_path' always points to the real path under the directory specified in the config file's 'file_path' option and should be used when providing a dataset's extra files directory as the input to a tool.

'files_path' points at the location that should be written to when used in a job so that the job's finish method can find the output files. This will be under the job_working_directory and thus writable by the actual user. So any tool outputs that use the extra files directory should use the output's 'files_path' attribute.

Sorry for this confusion. There was talk some time ago about merging both attributes to work correctly in all cases, but this was never done.

--nate

I don't know how things stand now, but agree file_path vs extra_file_path would be good to address in the IUC guidelines.

[gregvonkuster]

So any tool outputs that use the extra files directory should use the output's 'files_path' attribute.

From Nate's statement above, I understand that tools should be using $output.files_path. Is that correct? ping @natefoo

[jmchilton]

This thread is newer http://dev.list.galaxyproject.org/snpeff-tool-for-Galaxy-extra-files-path-td4665799.html and includes a writeup of my attempt to sort this all out in https://bitbucket.org/galaxy/galaxy-central/pull-requests/532/fix-for-datatypes-consuming-output-extra/diff.

The tl;dr is that the @jmchilton endorsed approach is:

$input.extra_files_path for inputs and $output.files_path for outputs

[peterjc]

Thanks @jmchilton - I was puzzled not to be able to find anything more recent on this in my email archive.

[gregvonkuster]

I believe this has been clarified, so can it be closed?

[bgruening]

Is this, or can we add this as text to the standards?

[peterjc]

Let's add something based on John's recommendation to the docs under http://galaxy-iuc-standards.readthedocs.io/en/latest/best_practices/tool_xml.html