looping multiple files: inputRedirect error

[AGrusz]

Greetings,

I would like to run "PairwiseRelativeRate.bf" for several hundred files. I've created a file, "inputRedirect.bf" for this task and placed it and all relevant files in the directory containing my HyPhy GUI executable. I receive the following error when attempting to run this file through the GUI (File -> Open Batch File -> "/Applications/HyPhy/inputRedirect.bf" -> Provide a list of files to process: -> "/Applications/HyPhy/test_list.txt"):

"Error: Could not read command file in ExecuteAFile. Original path: 'Macintosh HD:Applications:TemplateBatchFiles:PairwiseRelativeRate.bf'. Expanded path: 'Macintosh HD:Applications:TemplateBatchFiles:PairwiseRelativeRate.bf'

My list of files to process, "test_list.txt", contained the following two lines:

/Applications/HyPhy/1.aa_to_nuc.fasta /Applications/HyPhy/100.aa_to_nuc.fasta

My "inputRedirect.bf" file contains the following:

fileToExe = HYPHY_BASE_DIRECTORY + "TemplateBatchFiles" + DIRECTORY_SEPARATOR + "PairwiseRelativeRate.bf";

/* a list of file paths */ SetDialogPrompt ( "Provide a list of files to process:" ); fscanf (PROMPT_FOR_FILE, "Lines", _inDirectoryPaths );

fprintf (stdout, "[READ ", Columns (_inDirectoryPaths), " file path lines]\n");

/* the options passed to the GUI are encoded here */ inputRedirect = {}; inputRedirect["01"]="Nucleotide/Protein"; inputRedirect["03"]="WQML"; inputRedirect["04"]="Global"; inputRedirect["05"]="012345"; inputRedirect["06"]="y";

for ( _fileLine = 0; _fileLine < Columns ( _inDirectoryPaths ); _fileLine = _fileLine + 1 ) {

inputRedirect [ "02" ]  = _inDirectoryPaths[ _fileLine ];
ExecuteAFile ( fileToExe, inputRedirect );

}

I've tried many variations of submitting these files and can't seem to get it running. Any suggestions would be greatly appreciated.

Thanks, Amanda

[spond]

Hi Amanda,

I think this issue is due to the presence of a space in "Macintosh HD", and some legacy MacOS file path parsing. You should be able to run the same analysis using the same redirect file if you download and compile a command line version of HyPhy. Alternatively, you could try to rename "Macintosh HD" to something like "Macintosh-HD" and see if the problem goes away.

Sergei

[AGrusz]

Thanks, Sergei—I'll give it a try. Cheers, Amanda

On Wed, Jan 14, 2015 at 12:50 PM, Sergei Pond notifications@github.com wrote:

Hi Amanda,

I think this issue is due to the presence of a space in "Macintosh HD", and some legacy MacOS file path parsing. You should be able to run the same analysis using the same redirect file if you download and compile a command line version of HyPhy. Alternatively, you could try to rename "Macintosh HD" to something like "Macintosh-HD" and see if the problem goes away.

Sergei

— Reply to this email directly or view it on GitHub https://github.com/veg/hyphy/issues/249#issuecomment-69959022.

Amanda Grusz Email: amanda.grusz@gmail.com

[AGrusz]

Hi again,

I was able to execute "inputRedirect.bf" using the command line, but this returned the following syntax error: "Error: Expression appears to be incomplete/syntax error. Scope: 0, paretheses depth: 0, matrix scope: 0.0.In a literal. "

Do you know what this may be referring to?

Thanks, Amanda

P.S. I've copied the entire command line session below:

/HYPHY 2.220150114beta(MP) for Darwin on x86_64\
***** TYPES OF STANDARD ANALYSES *****

(1) Basic Analyses
(2) Codon Selection Analyses
(3) Compartmentalization
(4) Data File Tools
(5) Miscellaneous
(6) Model Comparison
(7) Kernel Analysis Tools
(8) Molecular Clock
(9) Phylogeny Reconstruction
(10) Positive Selection
(11) Recombination
(12) Selection/Recombination
(13) Relative Rate
(14) Relative Ratio
(15) Substitution Rates

Please select type of analyses you want to list (or press ENTER to process custom batch file):

/usr/local/lib/hyphy/Batch file to run::/Users/Grusz/Documents/A_Vittariods/Pteridaceae_transcriptomes/Rates_Analyses/backtranslated/test/inputRedirect.bf Error: Expression appears to be incomplete/syntax error. Scope: 0, paretheses depth: 0, matrix scope: 0.0.In a literal. fileToExe = "/Users/Grusz/bin/hyphy-2.2.3/res/TemplateBatchFiles/"PairwiseRelativeRate.bf";

/* a list of file paths */ SetDialogPrompt ( "Provide a list of files to process:" ); fscanf (PROMPT_FOR_FILE, "Lines", _inDirectoryPaths );

fprintf (stdout, "[READ ", Columns (_inDirectoryPaths), " file path lines]\n");

/* the options passed to the GUI are encoded here */ inputRedirect = {}; inputRedirect["01"]="Nucleotide/Protein"; inputRedirect["03"]="WQML"; inputRedirect["04"]="Global"; inputRedirect["05"]="012345"; inputRedirect["06"]="y";

for ( _fileLine = 0; _fileLine < Columns ( _inDirectoryPaths ); _fileLine = _fileLine + 1 ) {

inputRedirect [ "02" ]  = _inDirectoryPaths[ _fileLine ];
ExecuteAFile ( fileToExe, inputRedirect );

}

[AGrusz]

Forgive me—there was a quote out of place. Thanks again!

[AGrusz]

I have (hopefully) one last question regarding the "inputRedirect.bf" loop batch file. I would like to either:

(1) print the output files to unique filenames (e.g., if line one in my list of files is "1.nuc.fasta" I would like the corresponding output to be "1.hyphy.out") OR (2) print using identical filenames but each in written to a unique existing directory (e.g., "1/hyphy.out").

I'm having some trouble with the HBL syntax and any suggestions would be helpful. I've attached my file below.

Thanks!

Amanda

fileToExe = "/Users/Grusz/bin/hyphy-2.2.3/res/TemplateBatchFiles/PairwiseRelativeRate.bf";

/* a list of file paths */ SetDialogPrompt ( "Provide a list of files to process:" ); fscanf (PROMPT_FOR_FILE, "Lines", _inDirectoryPaths );

fprintf (stdout, "[READ ", Columns (_inDirectoryPaths), " file path lines]\n");

/* the options passed to the GUI are encoded here */ inputRedirect = {}; inputRedirect["01"]="Nucleotide/Protein"; inputRedirect["03"]="WQML"; inputRedirect["04"]="CUSTOM"; inputRedirect["05"]="012345"; inputRedirect["06"]="Global"; inputRedirect["07"]="Estimated"; inputRedirect["08"]="Observed"; inputRedirect["09"]= "PairRelRate.out";

for ( _fileLine = 0; _fileLine < Columns ( _inDirectoryPaths ); _fileLine = _fileLine + 1 ) {

inputRedirect [ "02" ]  = _inDirectoryPaths[ _fileLine ];
ExecuteAFile ( fileToExe, inputRedirect );

}

[spond]

Hi Amanda,

Try this

LoadFunctionLibrary  ("GrabBag");
...
pathInfo = splitFilePath (PATH_TO_CURRENT_BF + _inDirectoryPaths[ _fileLine ]); 
// if you have relative path names OR
pathInfo = splitFilePath (_inDirectoryPaths[ _fileLine ]); 
// if you have absolute path names
outputFile = pathInfo["DIRECTORY"] + DIRECTORY_SEPARATOR + pathInfo["FILENAME"] + ".hyphy.out";
...

Sergei

[AGrusz]

This worked! In the variable "pathInfo" I deleted the PATH_TO_CURRENT_BF because that information was contained within _inDirectoryPaths[_fileLine]), but once I did that it ran like a charm.

Thanks for your help!

Amanda

On Thu, Jan 15, 2015 at 2:34 PM, Sergei Pond notifications@github.com wrote:

Hi Amanda,

Try this

LoadFunctionLibrary ("GrabBag"); ... pathInfo = splitFilePath (PATH_TO_CURRENT_BF + _inDirectoryPaths[ _fileLine ]); // if you have relative path names OR pathInfo = splitFilePath (_inDirectoryPaths[ _fileLine ]); // if you have absolute path names outputFile = pathInfo["DIRECTORY"] + DIRECTORY_SEPARATOR + pathInfo["FILENAME"] + ".hyphy.out"; ...

Sergei

— Reply to this email directly or view it on GitHub https://github.com/veg/hyphy/issues/249#issuecomment-70146876.

Amanda Grusz Email: amanda.grusz@gmail.com