Eaton&Ree CLI Tutorial Issue

[konradtaube]

I am trying to practice the CLI tutorial using the 2013 paper, but am having an issue pretty early on. Using the ipyrad command I am able to get steps 1 and 2 done just fine, but on step 3 the program does not work. Typing in ipyrad -p params-pedicularis.txt -s 3 -r results in the following error at the aligning clusters step:

ipyrad -p params-pedicularis.txt -s 3 -r loading Assembly: pedicularis from saved path: ~/analysis-ipyrad/pedicularis.json

---

ipyrad [v.0.9.64] Interactive assembly and analysis of RAD-seq data

Parallel connection | DESKTOP-OJ5M50N: 12 cores

Step 3: Clustering/Mapping reads within samples [####################] 100% 0:00:09 | dereplicating [####################] 100% 0:04:03 | clustering/mapping [####################] 100% 0:00:01 | building clusters [####################] 100% 0:00:00 | chunking clusters [####################] 100% 0:24:54 | aligning clusters

Encountered an Error. Message: IndexError: string index out of range Parallel connection closed.

Summary stats of Assembly pedicularis

                               state  reads_raw  reads_passed_filter

29154_superba_SRR1754715 2 696994 689996 30556_thamno_SRR1754720 2 1452316 1440314 30686_cyathophylla_SRR1754730 2 1253109 1206947 32082_przewalskii_SRR1754729 2 964244 955480 33413_thamno_SRR1754728 2 636625 626084 33588_przewalskii_SRR1754727 2 1002923 993873 35236_rex_SRR1754731 2 1803858 1787366 35855_rex_SRR1754726 2 1409843 1397068 38362_rex_SRR1754725 2 1391175 1379626 39618_rex_SRR1754723 2 822263 813990 40578_rex_SRR1754724 2 1707942 1695523 41478_cyathophylloides_SRR1754722 2 2199740 2185364 41954_cyathophylloides_SRR1754721 2 2199613 2176210

Full stats files

step 1: ./analysis-ipyrad/pedicularis_s1_demultiplex_stats.txt step 2: ./analysis-ipyrad/pedicularis_edits/s2_rawedit_stats.txt step 3: None step 4: None step 5: None step 6: None step 7: None

Any idea of what I can do to get this program running through step3?

[isaacovercast]

What version of python are you running?

[konradtaube]

Right now I have just been using my Windows terminal/Unix stuff. Is that a problem? I've gone through the beginner tutorial and advanced tutorial just fine

[isaacovercast]

What version of python? python -V If the intro and advanced tutorials worked it's weird the pedicularis doesn't work. Can you do this:

ipyrad -p params-pedicularis.txt -s 3 -d

[isaacovercast]

Are you sure you have plenty of free disk space? df -h

If you run out of disk space wacky things can happen.

[isaacovercast]

It works for me with both python 2.7 and 3.6

[konradtaube]

Python is 3.8.5

I am currently running the ipyrad -p params-pedicularis.txt -s 3 -d command, I will let you know what happens once it finishes! Do you think the version of python could be the issue?

[isaacovercast]

Did you check your disk space? df -h

[konradtaube]

I just did, I get the following:

df -h Filesystem Size Used Avail Use% Mounted on rootfs 466G 452G 14G 98% / none 466G 452G 14G 98% /dev none 466G 452G 14G 98% /run none 466G 452G 14G 98% /run/lock none 466G 452G 14G 98% /run/shm none 466G 452G 14G 98% /run/user tmpfs 466G 452G 14G 98% /sys/fs/cgroup C:\ 466G 452G 14G 98% /mnt/c

Also, I ran the ipyrad -p params-pedicularis.txt -s 3 -d command and it also got stuck at 98/99% again, the time was just over an hour and a half on that particular step.

[isaacovercast]

Well, it looks like your disk is 98% full, so I guarantee you are running out of disk space during step 3 and this is causing the crash. ipyrad assemblies can consume a TON of disk, so we recommend having so much free disk you don't even have to worry about it. You need to have at least 10 times the size of the raw data in free disk space.

On Fri, Jan 29, 2021 at 6:27 PM Konrad Taube notifications@github.com wrote:

I just did, I get the following:

df -h Filesystem Size Used Avail Use% Mounted on rootfs 466G 452G 14G 98% / none 466G 452G 14G 98% /dev none 466G 452G 14G 98% /run none 466G 452G 14G 98% /run/lock none 466G 452G 14G 98% /run/shm none 466G 452G 14G 98% /run/user tmpfs 466G 452G 14G 98% /sys/fs/cgroup C:\ 466G 452G 14G 98% /mnt/c

Also, I ran the ipyrad -p params-pedicularis.txt -s 3 -d command and it also got stuck at 98/99% again, the time was just over an hour and a half on that particular step.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/dereneaton/ipyrad/issues/432#issuecomment-769940812, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNSXP4TF7JOM7OV53OZJL3S4LVYLANCNFSM4WY7KG3A .

[isaacovercast]

This also explains why you got through the simulated analyses. The simulated datasets are quite small.

On Fri, Jan 29, 2021 at 6:32 PM isaac overcast isaac.overcast@gmail.com wrote:

Well, it looks like your disk is 98% full, so I guarantee you are running out of disk space during step 3 and this is causing the crash. ipyrad assemblies can consume a TON of disk, so we recommend having so much free disk you don't even have to worry about it. You need to have at least 10 times the size of the raw data in free disk space.

On Fri, Jan 29, 2021 at 6:27 PM Konrad Taube notifications@github.com wrote:

I just did, I get the following:

df -h Filesystem Size Used Avail Use% Mounted on rootfs 466G 452G 14G 98% / none 466G 452G 14G 98% /dev none 466G 452G 14G 98% /run none 466G 452G 14G 98% /run/lock none 466G 452G 14G 98% /run/shm none 466G 452G 14G 98% /run/user tmpfs 466G 452G 14G 98% /sys/fs/cgroup C:\ 466G 452G 14G 98% /mnt/c

Also, I ran the ipyrad -p params-pedicularis.txt -s 3 -d command and it also got stuck at 98/99% again, the time was just over an hour and a half on that particular step.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/dereneaton/ipyrad/issues/432#issuecomment-769940812, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNSXP4TF7JOM7OV53OZJL3S4LVYLANCNFSM4WY7KG3A .

[isaacovercast]

Free up some disk and try again. Or try on another computer. Good luck!