s3df setup

[vespos]

S3DF setup:

• Add code to ask the elog about file location. The env var SIT_PSDM_DATA is set accordingly
• Revamp producer to handle the ffb and s3df situation

On the S3DF, we will rely on the SIT_PSDM_DATA to choose where to get data. By default it will look in /sdf/data/lcls/ds/<hutch>/<exp>/xtc

[vespos]

Some more changes:

• The log file name is now submitted using the #SBATCH syntax. This way the elog can pick the right log filename.
• To use this syntax, the EXPERIMENT and RUN_NUM must be exported as env var by the script, so that it also works in the CLI mode
• This in turns break the logic that was testing whether a job is submitted from the elog. A better variable (ARP_JOB_ID) is now used instead.
• A --s3df argument has been added to the script. This will force the script to look for data inthe s3df, regardless of the elog setup. This is intended mostly for testing purposes, but can perhaps also be used for when an experiment is moved offline or if the elog info a not working as intended.

[silkenelson]

I failed at seeing where the #BATCH syntax is used - which file is that? I also don't get the reason for the --s3df argument even for a test case - can you give an example where this is needed and what if toggles between?