invalid syntax

[tbazilegith]

I got this error when running pygap.py Does anyone know what it suggest? *pgap.py", line 164 self.cwlfile = f"pgap/{pipeline}.cwl" ^ SyntaxError: invalid syntax

Thanks!

[azat-badretdin]

Thank you for your report, Tassy, what is your command line?

[tbazilegith]

Below is my command python /color/bphl-state/myusername/repos/ncbi/pgap/pgap.py -r -o sample547_results input.yaml

my pgap directory looks like this: input-2022-04-14.build6021 mg37_results0728 pgap_2022-04-14.build6021.sif pgap.py test_genomes test_genomes-2022-04-14.build6021 VERSION The test has worked, but when I provide a local fasta, I got the above error. Also, the output directory isn't created. Thanks!

[azat-badretdin]

Thank you.

What is the output of python --version

[tbazilegith]

Python 2.7.5 as default

[azat-badretdin]

That is the version we no longer support, sorry. I suggest to run pgap.py as is, i.e. the same way you run it for the test case, sans python call.

[tbazilegith]

Let me try with load python module, given the version is Python 3.8.5 Also, when run ./pgap from a directory other than the one where it's installed, providing that full path /color/bphl-state/myusername/repos/ncbi/pgap/./pgap.py -r -o ... it redownloads the installation files on my home directory: input-2022-04-14.build6021, pgap_2022-04-14.build6021.sif, pgap.py, test_genomes, test_genomes-2022-04-14.build6021 And the run ends up failing with the message "No more space left on disk". It's weird because I installed pgap on a HPC directory /color/bphl-state/myusername/repos/ncbi/pgap/pgap.py

[azat-badretdin]

Also, when run ./pgap from a directory other than the one where it's installed, providing that full path /color/bphl-state/myusername/repos/ncbi/pgap/./pgap.py -r -o ... it redownloads the installation files on my home directory

This looks like a bug. Let me try this at home

[azat-badretdin]

As a workaround, what happens if you cd to the directory with pgap.py file and run it there with full path pointing to a data dir?

[tbazilegith]

Is it something like this? cd /color/bphl-state/myusername/repos/ncbi/pgap/./pgap.py -r -o ...

[azat-badretdin]

Yes

[tbazilegith]

Here is what I got cd /blue/bphl-state/myusername/repos/ncbi/pgap/./pgap.py -r -o mg37_results test_genomes/MG37/input.yaml bash: cd: /blue/bphl-state/myusername/repos/ncbi/pgap/./pgap.py: Not a directory

• Then I created path env variable for pgap.py PGAP_INPUT_DIR=/color/bphl-state/myusername/repos/ncbi/pgap/
• Ran pgap.py $PGAP_INPUT_DIR/./pgap.py -r -o mg37_results $PGAP_INPUT_DIR/test_genomes/MG37/input.yaml PGAP version 2022-04-14.build6021 is up to date. Output will be placed in: /color/bphl-state/myusername/aphl_bi_fellow/data/cult_for_id/SR_LR_hybrid/annotations_dir/mg37_results.2 WARN[0000] "/run/user/6210" directory set by $XDG_RUNTIME_DIR does not exist. Either create the directory or unset $XDG_RUNTIME_DIR.: stat /run/user/6210: no such file or directory: Trying to pull image in the event that it is a public image. PGAP failed, docker exited with rc = 1 Unable to find error in log file.

[azat-badretdin]

cd: /blue/bphl-state/myusername/repos/ncbi/pgap/./pgap.py: Not a directory

You forgot a semicolon :-) between directory name and ./pgap.py

cd /blue/bphl-state/myusername/repos/ncbi/pgap/
./pgap.py -r -o mg37_results test_genomes/MG37/input.yaml

[tbazilegith]

I did that but it seems like the program sought the directory "/input-2022-04-14.build6021/16S_rRNA" in my home while it is in /blue/bphl-state/myusername/repos/ncbi/pgap/

OSError: [Errno 28] No space left on device: '/home/myusername/.pgap/input-2022-04-14.build6021/16S_rRNA'

The full error message Downloaded 512 of 16487221477 bytes (0.00%)^MDownloaded 10752 of 16487221477 bytes (0.00%) ERROR: Failed to extract tarball; to install manually, try something like: curl -OLC - https://s3.amazonaws.com/pgap/input-2022-04-14.build6021.tgz tar xvf input-2022-04-14.build6021.tgz Process {package} installation: Traceback (most recent call last): File "/usr/lib64/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/usr/lib64/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) File "./pgap.py", line 116, in install_url tar.extractall(path=path) File "/usr/lib64/python3.6/tarfile.py", line 2010, in extractall numeric_owner=numeric_owner) File "/usr/lib64/python3.6/tarfile.py", line 2052, in extract numeric_owner=numeric_owner) File "/usr/lib64/python3.6/tarfile.py", line 2124, in _extract_member self.makedir(tarinfo, targetpath) File "/usr/lib64/python3.6/tarfile.py", line 2153, in makedir os.mkdir(targetpath, 0o700) OSError: [Errno 28] No space left on device: '/home/myusername/.pgap/input-2022-04-14.build6021/16S_rRNA'

As mentioned before, it redownloads the installation files Thanks!

[azat-badretdin]

it seems like the program sought the directory "/input-2022-04-14.build6021/16S_rRNA" in my home while it is in /blue/bphl-state/myusername/repos/ncbi/pgap/

Did you set envar PGAP_INPUT_DIR poitning to the installation before the run?

That is the prime suspect.

I would recommend adding this envar into your startup file ~/.bash_profile (or, if you are not using BASH as your primary shell, it could be some other setup script file, please consult your local system person) like this:

...
export PGAP_INPUT_DIR=<directory_name
....

[tbazilegith]

Yes! I set envar PGAP_INPUT_DIR while installing pgap, and it's installed in /blue/bphl-state/myusername/repos/ncbi/pgap/ Each time I run, I provide that path... (either on command line or in bash scripts) Another weird event, although in the installation directory, one day the run finishes and next day it fails, looking files elsewhere. Had to reinstall last time Let me check out the startup file Thanks, TJ

[tbazilegith]

Got this when running from another directory $PGAP_INPUT_DIR/./pgap.py --cpus 8 -r -o M37_results0731 $PGAP_INPUT_DIR/test_genomes/MG37/input.yaml

time="2022-07-31T23:38:49-04:00" level=warning msg="\"/run/user/6210\" directory set by $XDG_RUNTIME_DIR does not exist. Either create the directory or unset $XDG_RUNTIME_DIR.: stat /run/user/6210: no such file or directory: Trying to pull image in the event that it is a public image." Thanks

[tbazilegith]

Issued solved by adding the line export PGAP_INPUT_DIR= in .bashrc file, making the env var permanent. Then by running this command ${PGAP_INPUT_DIR}./pgap.py -n --no-internet --ignore-all-errors --container-path ${PGAP_INPUT_DIR}pgap_2022-04-14.build6021.sif -o M37_results${today} ${PGAP_INPUT_DIR}test_genomes/MG37/input.yaml --docker singularity

[tbazilegith]

*Issue solved