Known issues

[crimBubble]

Running ECCsplorer pipeline on devices with high CPU-core counts

Running the ECCsplorer pipeline using many CPU-cores might result in crashes in the mapping module, showing errors comparable to:

Calculating mean coverage of TR_map-all.MAPPING (ProcessID <x>; MultiID <y>) Killed

Current solution: reducing the CPU-core count (-cpu/--max_threads option) can avoid crashes.

[hqyone]

Is this possible to make a docker image to make the installation easier. Thank you.

[crimBubble]

Installing ECCsplorer pipeline

Is this possible to make a docker image to make the installation easier. Thank you.

Currently it is not possible to provide a docker image and the easiest way of installation is using a conda environment with mamba for a faster solving of the environment.

Try the following steps:

1. install anaconda3 or miniconda3
2. install mamba using conda install -c conda-forge mamba
3. download ECCsplorer and create the environment using: git clone https://github.com/crimBubble/ECCsplorer cd ECCsplorer mamba env create -f environment.yml this environment will already contain all third party tools except RepeatExplorer2
4. optional: link the ECCsplorer.py to the environment bin /.../anaconda3/envs/eccsplorer/bin/...
5. install RepeatExplorer2 unsing git clone https://bitbucket.org/petrnovak/repex_tarean.git cd repex_tarean conda activate eccsplorer make sudo dpkg --add-architecture i386 sudo apt-get update sudo apt-get install libc6:i386 libncurses5:i386 libstdc++6:i386
6. edit the "Locations of 3rd party tools (PATH)" for Trimmomatic and RepeatExploer2 in ECCsplorer/lib/config.py to match your installation path
7. test your installation

[hanssli]

Running ECCsplorer pipeline on devices with high CPU-core counts

Running the ECCsplorer pipeline using many CPU-cores might result in crashes in the mapping module, showing errors comparable to:

Calculating mean coverage of TR_map-all.MAPPING (ProcessID <x>; MultiID <y>) Killed

Current solution: reducing the CPU-core count (-cpu/--max_threads option) can avoid crashes.

Hello crimBubble, Is this problem solved? I have run the test data sucessfully. But when I run my own data, the terminal crased in the mapping step, whatever CPU core counts I set, I always got this problem. I have tried CPU 1, 4, 6 and all of cores. What should I do now?

Thanks and best wishes, Han

[crimBubble]

Dear @hanssli , thanks for using our pipeline. There is no update on this specific porblem. However, do you get an error message that you can send and provide the scale of the data your running? The amount of input data in GB and the genome size would be helpful for now to narrow down your issue. Best, L

[hanssli]

Dear @crimBubble , I have run many times, but no success except for the test run. I found it's stopped at the step "[mapper_coordinator] INFO: Creating index file for segmehl mapping" and then crased along with terminal closed itself. There is no other error message. I have attached the screenshot here.

The amount of input data is four 19+G fasta format files; the data is a little big, I use comparative moudle. However, it seems the prepared steps were completed sucessfully. The genome size I set is '-cnt auto --genome_size 750000000'. Are there any parameters I need to change?

Thanks and best wishes, Han

[hanssli]

Dear @crimBubble , The problem seems solved by close SWAP partition just now. I searched internet and found someone said the SWAP may affect memory, I tried to closed it, the eccsplorer is running now. Hope it can be success this time.

[1zhikun]

Dear @crimBubble , May I ask how long it will take to create this environment? I have been creating it all night without passing the "Solving Environment".

[crimBubble]

Dear @1zhikun, which method are you using for installation? If you are using conda, please try again using mamba (see the tutorials for detailed instructions). If this doesn't help, please try a different channel priority as mentioned here #7