search

bioinfomaticsCSU / deepsignal

Detecting methylation using signal-level features from Nanopore sequencing reads
GNU General Public License v3.0
108 stars 21 forks source link

multiprocessing #3

Closed PengNi closed 5 years ago

PengNi commented 5 years ago

  1. Testing on CPUs, when using multiprocessing.process(), even use only one process, multi cores (in _extractfeatures module) and all cores (in _callmodifications module) are used. Better figure out why and fix it, but not urgent.

  2. complete the _callmodifications module, take the fast5s as input, test it on CPU.

  3. need to test on GPU, see if the _callmodifications module can run as expect.

  4. need to figure out how to check if tensorflow runs on GPU or CPU inside the python code, for setting the nproc param properly under 2 (GPU/CPU) cases. Cause on GPU the nproc may not be set to >1.

after commit 3fa49c4.

PengNi commented 5 years ago

done task 2, see commit ebbbec1.

PengNi commented 5 years ago

about task 1, tried on another machine without using conda, the situation didn't occur. multi cores used by one process may be caused by the conda virtual environment.

test on two machines, the multi-core problem is likely to be associated with conda virtual environment.

However, in all three machines (one uses conda virtual env, the other two don't), the process with tensorflow in it always uses all cores whether in or not in conda virtual environment.

PengNi commented 5 years ago

about task 4, add a param "is_gpu" in _callmodification module, make users to decide whether to use gpu or not manually.

PengNi commented 5 years ago

done task 3, see commit ec55a3b.