search

ucscGenomeBrowser / kent

UCSC Genome Browser source tree. Stable branch: "beta".
http://genome.ucsc.edu/
Other
219 stars 89 forks source link

profile db not found in sqlProfileToMyCnf() #65

Closed Ziwei-Liu closed 1 month ago

Ziwei-Liu commented 2 years ago

Thank you for making all these bioinfo scripts, they are very useful! I'm constructing some karyotype chromosomes now using some de-novo genome assembly, and I'm trying to adopt the doBlastzChainNet.pl into my pipeline, following a manuscript from http://genomewiki.ucsc.edu/index.php/DoBlastzChainNet.pl#The_new_streamline_pairLastz_script.

I've already started a localhost parasol-hub according to a manuscript from http://genomewiki.ucsc.edu/index.php/Parasol_job_control_system. Then I started the doBlastzChainNet.pl, and it ran for about 30 minutes before crushes. It produced a log file named do.log.

I check the log file, found two parts may be indicating an error, and they are as bellow:

  1. do.log line8~line13 profile db not found in sqlProfileToMyCnf() -- failed for file /GPUFS/sysu_mhwang_1/.hgsql.cnf-m2VqGT failed with errno 2 profile db not found in sqlProfileToMyCnf() -- failed for file /GPUFS/sysu_mhwang_1/.hgsql.cnf-C7Hy3D failed with errno 2 HgStepManager: executing from step 'partition' through step 'syntenicNet'. HgStepManager: executing step 'partition' Wed Dec 29 09:02:38 2021. sort: write failed: 'standard output': Broken pipe sort: write error

  2. do.log line536~ gensub2 ptri.lst pcla.lst gsub jobList para make jobList Checking input files 14535 jobs written to /GPUFS/sysu_mhwang_1/sysu_mhwang_1/zwei_liuziwei/02_parasol/data/genomes/run.blastz/batch 14535 jobs in batch 0 jobs (including everybody's) in Parasol queue or running. Checking finished jobs updated job database on disk Pushed Jobs: 14535 total sick machines: 1 failures: 5

    Checking job status 0 minutes after launch 14535 jobs in batch 0 jobs (including everybody's) in Parasol queue or running. Sick Batch: consecutive crashes (36) >= sick batch threshold (25) Checking finished jobs updated job database on disk total sick machines: 1 failures: 36 Sick batch! will sleep 10 minutes, clear sick nodes and retry Told hub to clear sick nodes ... ... ...

    Checking job status 33 minutes after launch 14535 jobs in batch 0 jobs (including everybody's) in Parasol queue or running. Sick Batch: consecutive crashes (36) >= sick batch threshold (25) Checking finished jobs updated job database on disk Batch failed after 4 tries on /GPUFS/sysu_mhwang_1/sysu_mhwang_1/zwei_liuziwei/02_parasol/data/scripts/blastz-run-ucsc -outFormat psl /GPUFS/sysu_mhwang_1/sysu_mhwang_1/zwei_liuziwei/02_parasol/data/genomes/ptri.2bit:NC_059295.1:0-32110000 /GPUFS/sysu_mhwang_1/sysu_mhwang_1/zwei_liuziwei/02_parasol/data/genomes/pcla.2bit:LG03:0-10000000 ../DEF ../psl/ptri.2bit:NC_059295.1:0-32110000/ptri.2bit:NC_059295.1:0-32110000_pcla.2bit:LG03:0-10000000.psl Command failed: ssh -x -o 'StrictHostKeyChecking = no' -o 'BatchMode = yes' localhost nice /GPUFS/sysu_mhwang_1/sysu_mhwang_1/zwei_liuziwei/02_parasol/data/genomes/run.blastz/doClusterRun.csh

real 33m9.299s user 0m0.082s sys 0m0.069s

2021/12/29 07:56:39: info: starting paraNode on zwei-mysql5-29153733-5c7c7d66c4-gdqnz 2021/12/29 08:26:10: warn: duplicate packet filtered out: 192.168.206.146-9694-104-1 2021/12/29 08:26:10: warn: duplicate packet filtered out: 192.168.206.146-9694-105-1 2021/12/29 08:26:10: warn: duplicate packet filtered out: 192.168.206.146-9694-103-1 2021/12/29 08:26:10: warn: duplicate packet filtered out: 192.168.206.146-9694-102-1 2021/12/29 08:26:10: warn: duplicate packet filtered out: 192.168.206.146-9694-101-1 2021/12/29 08:26:10: warn: duplicate packet filtered out: 192.168.206.146-9694-107-1 2021/12/29 08:26:10: warn: duplicate packet filtered out: 192.168.206.146-9694-108-1 2021/12/29 09:03:10: warn: duplicate packet filtered out: 192.168.206.146-9694-136-1 2021/12/29 09:03:10: warn: duplicate packet filtered out: 192.168.206.146-9694-138-1 2021/12/29 09:03:10: warn: duplicate packet filtered out: 192.168.206.146-9694-139-1 2021/12/29 09:03:10: warn: duplicate packet filtered out: 192.168.206.146-9694-137-1 2021/12/29 09:03:10: warn: duplicate packet filtered out: 192.168.206.146-9694-140-1 2021/12/29 09:03:10: warn: duplicate packet filtered out: 192.168.206.146-9694-143-1 2021/12/29 09:03:10: warn: duplicate packet filtered out: 192.168.206.146-9694-142-1 2021/12/29 09:03:10: warn: duplicate packet filtered out: 192.168.206.146-9694-141-1 2021/12/29 09:03:10: warn: duplicate packet filtered out: 192.168.206.146-9694-144-1 2021/12/29 09:03:10: warn: duplicate packet filtered out: 192.168.206.146-9694-145-1 2021/12/29 09:13:40: warn: duplicate packet filtered out: 192.168.206.146-9694-175-1 2021/12/29 09:13:40: warn: duplicate packet filtered out: 192.168.206.146-9694-176-1 2021/12/29 09:13:40: warn: duplicate packet filtered out: 192.168.206.146-9694-178-1 2021/12/29 09:13:40: warn: duplicate packet filtered out: 192.168.206.146-9694-177-1 2021/12/29 09:13:40: warn: duplicate packet filtered out: 192.168.206.146-9694-179-1 2021/12/29 09:13:40: warn: duplicate packet filtered out: 192.168.206.146-9694-180-1 2021/12/29 09:13:40: warn: duplicate packet filtered out: 192.168.206.146-9694-181-1 2021/12/29 09:24:10: warn: duplicate packet filtered out: 192.168.206.146-9694-211-1 2021/12/29 09:24:10: warn: duplicate packet filtered out: 192.168.206.146-9694-205-1 2021/12/29 09:24:10: warn: duplicate packet filtered out: 192.168.206.146-9694-207-1 2021/12/29 09:24:10: warn: duplicate packet filtered out: 192.168.206.146-9694-206-1 2021/12/29 09:24:10: warn: duplicate packet filtered out: 192.168.206.146-9694-212-1 2021/12/29 09:24:10: warn: duplicate packet filtered out: 192.168.206.146-9694-209-1 2021/12/29 09:24:10: warn: duplicate packet filtered out: 192.168.206.146-9694-208-1 2021/12/29 09:24:10: warn: duplicate packet filtered out: 192.168.206.146-9694-210-1 2021/12/29 09:24:10: warn: duplicate packet filtered out: 192.168.206.146-9694-213-1 2021/12/29 09:24:10: warn: duplicate packet filtered out: 192.168.206.146-9694-214-1 2021/12/29 09:24:10: warn: duplicate packet filtered out: 192.168.206.146-9694-215-1 2021/12/29 09:24:10: warn: duplicate packet filtered out: 192.168.206.146-9694-216-1 2021/12/29 09:24:10: warn: duplicate packet filtered out: 192.168.206.146-9694-217-1 2021/12/29 09:34:55: warn: duplicate packet filtered out: 192.168.206.146-9694-246-1 2021/12/29 09:34:55: warn: duplicate packet filtered out: 192.168.206.146-9694-248-1 2021/12/29 09:34:55: warn: duplicate packet filtered out: 192.168.206.146-9694-249-1 2021/12/29 09:34:55: warn: duplicate packet filtered out: 192.168.206.146-9694-250-1 2021/12/29 09:34:55: warn: duplicate packet filtered out: 192.168.206.146-9694-251-1 2021/12/29 09:34:55: warn: duplicate packet filtered out: 192.168.206.146-9694-252-1 2021/12/29 09:34:55: warn: duplicate packet filtered out: 192.168.206.146-9694-253-1

maximilianh commented 1 month ago

This question was sent and answered on our mailing list two years ago.

Also, this happened because the db.* statements are missing in ~/hg.conf. We've improved the error message and the new message is a lot clearer. Closing the ticket.