No such file or directory: 'dstats/sim1.tests'

[angelesdecara]

Dear Evan,

I have managed to run all the graphs, and now I was going to calculate the Bayes factors. But it fails:

% python ~/Downloads/qpbrute/qpbayes.py --geno 6Pops.eigenstratgeno --ind 6PopsGood.ind --snp 6Pops.snp --prefix sim1 --pops CNWB2 P08 P09 RUWB3 P10 --out SCEL --threads 1

Traceback (most recent call last): File "/Users/angeles/Downloads/qpbrute/qpbayes.py", line 223, in calculate_bayes_factors(argv.geno, argv.snp, argv.ind, argv.prefix, argv.pops, argv.out, nthreads=argv.threads) File "/Users/angeles/Downloads/qpbrute/qpbayes.py", line 198, in calculate_bayes_factors qpb.calculate_dstats() File "/Users/angeles/Downloads/qpbrute/qpbayes.py", line 102, in calculate_dstats with open(self.dstat_tests, 'w') as fout: IOError: [Errno 2] No such file or directory: 'dstats/sim1.tests'

It seems like a very stupid problem. There's indeed no such file, and I don't know whether I need to generate it by hand.

By the way, none of the pdf files generated have the circle with the population name, other than the outgroup. I can generate them with dot from the graph files, but they're cleaner without all the intermediate populations. Here's one sim1-n6-o0-a3-0efd784.pdf

Sorry to bother you again and many thanks!

[ekirving]

IOError: [Errno 2] No such file or directory: 'dstats/sim1.tests'

This is an very odd error, as that file should not exist prior to running qpbayes.py. The error is occuring at the point where the code is attempting to make the file. The most likely cause is that the folder dstats is missing, or your python process doesn't have write permission to that folder.

By the way, none of the pdf files generated have the circle with the population name, other than the outgroup. I can generate them with dot from the graph files, but they're cleaner without all the intermediate populations. Here's one

Sorry about this one. The rendering error was caused by unexpected numbers in your population names. I have corrected the problem by updating the code to allow numbers in population names.

[angelesdecara]

I've created the dstats and the bayes folders by hand, and then

% python ~/Downloads/qpbrute/qpbayes.py --geno 6Pops.eigenstratgeno --ind 6PopsGood.ind --snp 6Pops.snp --prefix sim1 --pops CNWB2 P08 P09 RUWB3 P10 --out SCEL --threads 1

INFO: There are 60 D-stat tests to compute for 5 populations. INFO: There are 783 graphs to compute Bayes factors for. Traceback (most recent call last): File "/Users/angeles/Downloads/qpbrute/qpbayes.py", line 223, in calculate_bayes_factors(argv.geno, argv.snp, argv.ind, argv.prefix, argv.pops, argv.out, nthreads=argv.threads) File "/Users/angeles/Downloads/qpbrute/qpbayes.py", line 201, in calculate_bayes_factors qpb.calculate_bayes_factors() File "/Users/angeles/Downloads/qpbrute/qpbayes.py", line 154, in calculate_bayes_factors self.model_likelihood(graph) File "/Users/angeles/Downloads/qpbrute/qpbayes.py", line 172, in model_likelihood MCMC_NUM_BURN], env={'OMP_NUM_THREADS': '1'}, stdout=open(log_file, 'w')) File "/Users/angeles/Downloads/qpbrute/utils.py", line 50, in run_cmd raise RuntimeError(err) RuntimeError

Many thanks for all the time you're taking!

[ekirving]

On closer inspection, I can now see the cause of the first error. The code for qpbayes.py and qpbrute.py uses relative paths for each of the output folders, however, you are not running qpbayes.py from the root folder of the project, so it it looking for a dstats folder in whichever directory you are calling qpbayes.py from. Try rerunning from inside ~/Downloads/qpbrute/

[ekirving]

File "/Users/angeles/Downloads/qpbrute/utils.py", line 50, in run_cmd raise RuntimeError(err) RuntimeError

This error is likely cause by the same problem as before, as qpbayes.py cannot find the R script rscript/model_likelihood.R. Try rerunning from the root folder of the project.

[angelesdecara]

Running from the Downloads folder pointing to my data

% python qpbayes.py --geno ~/CiBioPorks/QPbrute/6Pops.eigenstratgeno -- ind ~/CiBioPorks/QPbrute/6PopsGood.ind --snp ~/CiBioPorks/QPbrute/6Pops.snp --prefix wildboar --pops CNWB2 P08 P09 RUWB3 P10 --out SCEL --threads 1

I now obtain

INFO: There are 60 D-stat tests to compute for 5 populations. INFO: There are 0 graphs to compute Bayes factors for. Traceback (most recent call last): File "qpbayes.py", line 223, in calculate_bayes_factors(argv.geno, argv.snp, argv.ind, argv.prefix, argv.pops, argv.out, nthreads=argv.threads) File "qpbayes.py", line 204, in calculate_bayes_factors qpb.find_best_model() File "qpbayes.py", line 185, in find_best_model MCMC_NUM_BURN], stdout=open(log_file, 'w')) File "/Users/angeles/Downloads/qpbrute/utils.py", line 50, in run_cmd raise RuntimeError(err) RuntimeError: Registered S3 methods overwritten by 'ggplot2': method from [.quosures rlang c.quosures rlang print.quosures rlang

Attaching package: 'scales' The following object is masked from 'package:viridis':

viridis_pal

Attaching package: 'data.table'

The following object is masked from 'package:raster':

shift

The following objects are masked from 'package:reshape2':

dcast, melt

Error in -ll$mean : invalid argument to unary operator Calls: [ -> [.data.frame -> order Execution halted

[angelesdecara]

It's working by moving all the files in graphs/ and pdf/ to the ~/Downloads/qpbrute/ respective folders.

[ekirving]

OK, great. The code was never intended to be run outside of the project folder, so doing so will cause all sorts of weird errors.

[ekirving]

I have updated the README.md file to make this clear to other users.

Thanks for persevering with all the setup issues!