[BUG] <title>PGAP failed

[StickHu]

Hi, recently when I was using the PGAP, a problem occurred.

Describe the bug

/data/workdir/huwa/annotation/PGAP/pgap/scripts/./pgap.py:133: SyntaxWarning: invalid escape sequence '[' r = "^[(?P[^]]+)] (?P[^ ]+) [(?P[^ ]) (?P[^]])] (?P.*)" /data/workdir/huwa/annotation/PGAP/pgap/scripts/./pgap.py:978: SyntaxWarning: invalid escape sequence '-' if not re.match("^[a-zA-Z0-9_-]+$", prefix): /data/workdir/huwa/annotation/PGAP/pgap/scripts/./pgap.py:1109: SyntaxWarning: invalid escape sequence '-' parser.error("Invalid Command Line Argument Error: Both arguments -s--organism and -g--genome must be provided if no YAML file is provided.") Output will be placed in: /data/workdir/huwa/annotation/PGAP/pgap/scripts/output The latest version of PGAP is 2024-07-18.build7555, you have nothing installed locally. Not trying to update self, because the --no-self-update flag is enabled. installation directory: /home/huwa/.pgap Filesystem 1K-blocks Used Available Use% Mounted on /dev/md126p2 403223544 210127984 172539428 55% / Not trying to update self, because the --no-self-update flag is enabled. Skipping already installed tarball: https://s3.amazonaws.com/pgap/input-2024-07-18.build7555.tgz Downloading (as needed) Docker image docker.io/ncbi/pgap:2024-07-18.build7555 Trying to pull docker.io/ncbi/pgap:2024-07-18.build7555... WARN[0033] Failed, retrying in 1s ... (1/3). Error: parsing image configuration: Get "https://production.cloudflare.docker.com/registry-v2/docker/registry/v2/blobs/sha256/25/25d52721af3b9f188b0ae4e640c9ce03ffae89076278cac8e2ac78f0cdcc436e/data?verify=1724380823-2s8kK%2FMDGVZ3DonS8S1hyEzuBn0%3D": dial tcp 199.59.148.201:443: i/o timeout WARN[0067] Failed, retrying in 1s ... (2/3). Error: parsing image configuration: Get "https://production.cloudflare.docker.com/registry-v2/docker/registry/v2/blobs/sha256/25/25d52721af3b9f188b0ae4e640c9ce03ffae89076278cac8e2ac78f0cdcc436e/data?verify=1724380857-RqPtPjAtvienzWWPa4BE6OhsiCI%3D": dial tcp 104.244.43.52:443: i/o timeout WARN[0101] Failed, retrying in 1s ... (3/3). Error: parsing image configuration: Get "https://production.cloudflare.docker.com/registry-v2/docker/registry/v2/blobs/sha256/25/25d52721af3b9f188b0ae4e640c9ce03ffae89076278cac8e2ac78f0cdcc436e/data?verify=1724380891-SOds%2BM%2FG49VBUHf0te7swGX9Sso%3D": dial tcp 108.160.170.33:443: i/o timeout Error: parsing image configuration: Get "https://production.cloudflare.docker.com/registry-v2/docker/registry/v2/blobs/sha256/25/25d52721af3b9f188b0ae4e640c9ce03ffae89076278cac8e2ac78f0cdcc436e/data?verify=1724380926-ynvXfe%2FwBzhGiAaPZjNzJHm%2FlTI%3D": dial tcp 108.160.170.33:443: i/o timeout None Traceback (most recent call last): File "/data/workdir/huwa/annotation/PGAP/pgap/scripts/./pgap.py", line 1123, in main params = Setup(args) ^^^^^^^^^^^ File "/data/workdir/huwa/annotation/PGAP/pgap/scripts/./pgap.py", line 630, in init self.update() File "/data/workdir/huwa/annotation/PGAP/pgap/scripts/./pgap.py", line 782, in update raise Exception(f'installation of some or all of components failed. Please remove {self.data_path}, {self.install_dir}/test_genomes, {self.test_genomes_path} directories and try again.') Exception: installation of some or all of components failed. Please remove /home/huwa/.pgap/input-2024-07-18.build7555, /home/huwa/.pgap/test_genomes, /home/huwa/.pgap/test_genomes-2024-07-18.build7555 directories and try again

My command line is "./pgap.py -g /data/workdir/huwa/oral_DB/allspeciesFastq/assembly_dealed/Porphyromonas_gingivalis_fastq/raw_spades/all_checkm_bins/ERR5385081_bin_52.fa -s 'Porphyromonas gingivalis' -r --no-self-update"

Could you please help me solve that? Thanks for your reply.

[azat-badretdin]

Thank you for your report, user @StickHu !

The error you posted might be a transient Internet issue. Could you please retry?

It looks like you have a rigorous Python execution setting that elucidated number of warnings in our Python code that are worth fixing. Thank you for that too!

[StickHu]

Thanks for your reply. However, I have retried several times. Could you please help me fix that?

[azat-badretdin]

Thanks. That must be some systemic Internet problem in a way you set up things. Could you please post your command line?

[StickHu]

Here is my command line "./pgap.py -g /data/workdir/huwa/oral_DB/allspeciesFastq/assembly_dealed/Porphyromonas_gingivalis_fastq/raw_spades/all_checkm_bins/ERR5385081_bin_52.fa -s 'Porphyromonas gingivalis' -r --no-self-update". Thanks for your patient reply

[azat-badretdin]

It looks like you are not requesting airgapped execution, so the nature of your, as determined now, persistent internet problem baffles me.

One of the IPs mentioned, 199.59.148.201, resolves to a Twitter host: r-199-59-148-201.twttr.com. The originating request also comes from a third party: docker.

So two third party host are failing while trying to talk to each other. This must be something that could be observed from our end, but we haven't seen it on 23rd of August, a day after you posted this problem, when we did our massive weekly test that involves installation of PGAP on multiple cloud computers.

I would recommend to consult local specialists that are familiar with your local internet setup.

Bottom line: we can't reproduce this Internet connection problem.

PS. I presume that when you reported that you retried, you also observed the same sort of error diagnostics, correct?