dereneaton / ipyrad

Interactive assembly and analysis of RAD-seq data sets
http://ipyrad.readthedocs.io
GNU General Public License v3.0
70 stars 39 forks source link

Step 1 has been running and does not end, and there is no error message #487

Closed Zeroo11 closed 1 year ago

Zeroo11 commented 1 year ago

Only 4 demultiplexed/sorted RAD-seq data are used. After running the first step, it has been running and never ends, no any file is generated, but there is no error message.

The parameter file is as follows:

image

They are 4 demultiplexed/sorted RAD-seq:

image

It's a state that never stops (no error message):

image

This is the cpu status:

image

I have tried reinstalling ipyrad, installing ipyrad with a lower version, and even reinstalling the centos system, but still can't solve it. I urge friends and developers to help, thank you very much!

isaacovercast commented 1 year ago

Hm, are you running this on an HPC system or a local workstation?

You might try launching ipcluster by hand, it seems there is some coordination issue with the python parallelization backend.

## Replace 64 here with however many cores your workstation has
ipcluster start --n 64 --daemonize
## Wait for this to return and make look for the message saying the cluster is running

## Now run ipyrad with the --ipcluster flag
ipyrad -p params-iplex.txt -s 1 --ipcluster

Try that and let me know how it goes.

Zeroo11 commented 1 year ago

Hm, are you running this on an HPC system or a local workstation?

You might try launching ipcluster by hand, it seems there is some coordination issue with the python parallelization backend.

## Replace 64 here with however many cores your workstation has
ipcluster start --n 64 --daemonize
## Wait for this to return and make look for the message saying the cluster is running

## Now run ipyrad with the --ipcluster flag
ipyrad -p params-iplex.txt -s 1 --ipcluster

Try that and let me know how it goes.

Thank you very much ! According to your method, there are some error messages, as follows `(ipyrad) [pse_01@localhost params_file]$ ipyrad -p params-ipilex.txt -s 1 --ipcluster


ipyrad [v.0.9.84] Interactive assembly and analysis of RAD-seq data


Traceback (most recent call last): File "/home/pse_01/miniconda3/envs/ipyrad/bin/ipyrad", line 10, in sys.exit(main()) File "/home/pse_01/miniconda3/envs/ipyrad/lib/python3.9/site-packages/ipyrad/main.py", line 605, in main CLI() File "/home/pse_01/miniconda3/envs/ipyrad/lib/python3.9/site-packages/ipyrad/main.py", line 75, in init self.run() File "/home/pse_01/miniconda3/envs/ipyrad/lib/python3.9/site-packages/ipyrad/main.py", line 496, in run ipyclient = ipp.Client(profile=self.args.ipcluster) File "/home/pse_01/miniconda3/envs/ipyrad/lib/python3.9/site-packages/ipyparallel/client/client.py", line 607, in init self._connect(sshserver, ssh_kwargs, timeout) File "/home/pse_01/miniconda3/envs/ipyrad/lib/python3.9/site-packages/ipyparallel/client/client.py", line 758, in _connect raise TimeoutError("Hub connection request timed out") TimeoutError: Hub connection request timed out `

Do you still know how to solve this error? My research team and I are very grateful for your help!

Zeroo11 commented 1 year ago

Hm, are you running this on an HPC system or a local workstation?

You might try launching ipcluster by hand, it seems there is some coordination issue with the python parallelization backend.

## Replace 64 here with however many cores your workstation has
ipcluster start --n 64 --daemonize
## Wait for this to return and make look for the message saying the cluster is running

## Now run ipyrad with the --ipcluster flag
ipyrad -p params-iplex.txt -s 1 --ipcluster

Try that and let me know how it goes.

We are using a server, which should be a local workstation, with 28 cores.

isaacovercast commented 1 year ago

Yes, it looks like you need to wait longer for the ipcluster instance to start up before launching ipyrad. You can see some documentation about running ipcluster by hand here.

The call to sleep gives the ipcluster engines time to spin up. You might need to experiment with longer sleep times depending on your particular clusters behavior.

ipcluster start --n 20 --daemonize
sleep 60
 ipyrad -p params-test.txt -s 2 --ipcluster
isaacovercast commented 1 year ago

Fixed?