error when running test files

[angelesdecara]

Hi,

I'm on a Mac 10.12.6. I've tried

python qpbrute.py --par test/sim1.par --prefix sim1 --pops A B C X --out Out

but I get the following: INFO: There are 24 possible starting orders for the given nodes. INFO: Performing an exhaustive search. INFO: Starting list ['A', 'B', 'C', 'X'] Traceback (most recent call last): File "qpbrute.py", line 571, in permute_qpgraph(argv.par, argv.prefix, argv.pops, argv.out) File "qpbrute.py", line 534, in permute_qpgraph pq.find_graph() File "qpbrute.py", line 494, in find_graph self.recurse_tree(root_tree, self.nodes[1], self.nodes[2:]) File "qpbrute.py", line 101, in recurse_tree results = self.test_trees(new_trees, depth) File "qpbrute.py", line 169, in test_trees results = pool.map(self.run_qpgraph, itertools.izip(new_trees, itertools.repeat(depth))) File "/usr/local/lib/python2.7/site-packages/pathos/multiprocessing.py", line 137, in map return _pool.map(star(f), zip(*args)) # chunksize File "/usr/local/lib/python2.7/site-packages/multiprocess/pool.py", line 253, in map return self.map_async(func, iterable, chunksize).get() File "/usr/local/lib/python2.7/site-packages/multiprocess/pool.py", line 572, in get raise self._value OSError: [Errno 2] No such file or directory

All help will be greatly appreciated. Thanks!

[ekirving]

The most likely cause of this error that the qpGraph binary (from AdmixTools) is not in your path.

Assuming you have installed AdmixTools, you can run the following command to add the binaries to your path (you will need to update this with your actual install path): echo 'export PATH="/path/to/AdmixTools/bin:$PATH"' >> ~/.bash_profile

Then reload your bash profile: source ~/.bash_profile

[angelesdecara]

Sorry, but it hasn't worked... I still get the same message with AdmixTools in my path.

[ekirving]

What happens if you run qpGraph without any parameters in the qpbrute folder?

[angelesdecara]

I get this

no parameters

qpGraph version: 6450

[ekirving]

That is the expected output, so something else must be wrong.

It's hard to debug python errors that occur inside a multi-processing pool, as the stack trace does not show the actual location of the error.

I have updated the code to include a new command line parameter --threads which will allow you to turn off multithreading, so we can see a more informative error message.

Please run the following, and then post the error message.

git pull python qpbrute.py --par test/sim1.par --prefix sim1 --pops A B C X --out Out --threads 1

[angelesdecara]

Many thanks for all your help! Now I get this error:

Out --threads 1 INFO: There are 24 possible starting orders for the given nodes. INFO: Performing an exhaustive search. INFO: Starting list ['A', 'B', 'C', 'X'] Traceback (most recent call last): File "qpbrute.py", line 573, in permute_qpgraph(argv.par, argv.prefix, argv.pops, argv.out, nthreads=argv.threads) File "qpbrute.py", line 534, in permute_qpgraph pq.find_graph() File "qpbrute.py", line 494, in find_graph self.recurse_tree(root_tree, self.nodes[1], self.nodes[2:]) File "qpbrute.py", line 101, in recurse_tree results = self.test_trees(new_trees, depth) File "qpbrute.py", line 168, in test_trees pool = mp.ProcessingPool(self.nthreads) File "/Users/angeles/anaconda2/lib/python2.7/site-packages/pathos/multiprocessing.py", line 109, in init self._serve() File "/Users/angeles/anaconda2/lib/python2.7/site-packages/pathos/multiprocessing.py", line 121, in _serve _pool = Pool(nodes) File "/Users/angeles/anaconda2/lib/python2.7/site-packages/multiprocess/pool.py", line 161, in init self._repopulate_pool() File "/Users/angeles/anaconda2/lib/python2.7/site-packages/multiprocess/pool.py", line 216, in _repopulate_pool for i in range(self._processes - len(self._pool)): TypeError: unsupported operand type(s) for -: 'str' and 'int'

[ekirving]

Sorry, I didn't properly test that change and forgot to cast a string to integer.

Please run git pull and the other command again.

[angelesdecara]

Sorry, still errors. I've run python qpbrute.py --par test/sim1.par --prefix sim1 --pops A B C X --out Out --threads 1 and I get this

INFO: There are 24 possible starting orders for the given nodes. INFO: Performing an exhaustive search. INFO: Starting list ['A', 'B', 'C', 'X'] Traceback (most recent call last): File "qpbrute.py", line 573, in permute_qpgraph(argv.par, argv.prefix, argv.pops, argv.out, nthreads=argv.threads) File "qpbrute.py", line 534, in permute_qpgraph pq.find_graph() File "qpbrute.py", line 494, in find_graph self.recurse_tree(root_tree, self.nodes[1], self.nodes[2:]) File "qpbrute.py", line 101, in recurse_tree results = self.test_trees(new_trees, depth) File "qpbrute.py", line 174, in test_trees result = self.run_qpgraph((new_tree, depth)) File "qpbrute.py", line 285, in run_qpgraph log = run_cmd(["qpGraph", "-p", self.par_file, "-g", grp_file, "-d", dot_file]) File "/Users/angeles/Downloads/qpbrute/utils.py", line 38, in run_cmd proc = subprocess.Popen(cmd, shell=shell, stdout=stdout, stderr=stderr, env=local_env) File "/Users/angeles/anaconda2/lib/python2.7/subprocess.py", line 679, in init errread, errwrite) File "/Users/angeles/anaconda2/lib/python2.7/subprocess.py", line 1249, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory

[ekirving]

OK, so we're back to the original error now, but with a better stack trace.

The problem appears to be that python's subprocess module is not finding qpGraph in the PATH. I don't know why that is the case (it works fine for me), but a workaround would be to use an absolute path.

If you edit qpbrute.py line 285, and replace "qpGraph" with the absolute path to your qpGraph binary:

log = run_cmd(["/path/to/AdmixTools/bin/qpGraph", "-p", self.par_file, "-g", grp_file, "-d", dot_file])

[angelesdecara]

I've changed my PATH so that instead of ~/Downloads/AdmixTools/bin it has the whole (absolute) path /Users/angeles/Downloads/AdmixTools/bin, it works! Many many thanks!

[ekirving]

OK, great. Glad you got it working!