Error while calling mod and assigning threads

[vahidAK]

Hi @PengNi I am calling modification from the extracted feature file. After running for a while the software gives me this error: return array(a, dtype, copy=False, order=order) ValueError: setting an array element with a sequence. I would appreciate if you could help me with this matter. BTW: another problem I face is that no matter how many threads you define for the deepsignal in nproc when extracting features or calling mods on CPU, it will take all the available threads. Thanks, Vahid.

[PengNi]

@vahidAK , could you give a more complete log, which can help to locate the specific line in the script that raised the error? This is a error I didn't met before.

The threads problem may caused by the tensorflow. Sorry we haven't figure out how to solve it (refer to #3 task1).

[vahidAK]

Sure @PengNi ! here is the more complete log:

2019-10-16 15:46:36.698999: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1053] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:7 with 29925 MB memory) -> physical GPU (device: 7, name: Tesla V100-SXM2-32GB, pci bus id: 0000:8a:00.0, compute capability: 7.0) Process Process-2: Traceback (most recent call last): File "/projects/vakbari_prj/anaconda3/envs/deepsignal_gpu/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/projects/vakbari_prj/anaconda3/envs/deepsignal_gpu/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) File "/projects/vakbari_prj/anaconda3/envs/deepsignal_gpu/lib/python3.6/site-packages/deepsignal-0.1.6-py3.6.egg/deepsignal/call_modifications.py", line 186, in _call_mods_q pred_str, accuracy = _call_mods(features_batch, sess, model, init_learning_rate) File "/projects/vakbari_prj/anaconda3/envs/deepsignal_gpu/lib/python3.6/site-packages/deepsignal-0.1.6-py3.6.egg/deepsignal/call_modifications.py", line 140, in _call_mods [model.activation_logits, model.prediction], feed_dict=feed_dict) File "/projects/vakbari_prj/anaconda3/envs/deepsignal_gpu/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 900, in run run_metadata_ptr) File "/projects/vakbari_prj/anaconda3/envs/deepsignal_gpu/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1104, in _run np_val = np.asarray(subfeed_val, dtype=subfeed_dtype) File "/projects/vakbari_prj/anaconda3/envs/deepsignal_gpu/lib/python3.6/site-packages/numpy/core/_asarray.py", line 85, in asarray return array(a, dtype, copy=False, order=order) ValueError: setting an array element with a sequence.

[PengNi]

@vahidAK , can you give me a sample of your input file? I'm not sure but I think the format of the input may be wrong. Not all lines in the input file are wrong, maybe one line somewhere.

[vahidAK]

Yes! I think the problem is with the input. My input is a big file how ever here is some line in between of the file: X 65114766 - 105916532 5cd65c07-f25d-46e2-b444-1c4898c03c30 t TCTTCATACGTGAGGTC 1.744726,-0.408391,0.682519,1.142503,0.268993,-0.344657,1.103272,0.260598,0.325909,-0.936457,0.632334,-0.677931,-0.722668,-1.564732,-1.292772,1.371061,0.562074 0.340877,0.185864,0.13954,0.150632,0.179413,0.197377,0.56048,0.192661,0.202407,0.076116,0.362686,0.130315,0.039337,0.149759,0.761081,0.188771,0.147623 7,43,3,7,6,13,5,11,17,16,8,7,3,23,3,12,3 1.132179,2.24027,2.023469,1.445335,1.734402,1.734402,1.903025,0.120445,-0.409512,-0.481778,-0.457689,-0.746757,-0.457689,-0.481778,-0.626312,-0.602223,-0.2168,-0.481778,-0.602223,-0.722668,-0.409512,-0.529956,-0.409512,-0.505867,-0.337245,-0.409512,-0.409512,-0.433601,-0.578134,-0.289067,-0.433601,-0.409512,-0.409512,-0.361334,-0.529956,-0.361334,-0.2168,-0.337245,-0.409512,-0.337245,-0.433601,-0.361334,-0.481778,-0.289067,-0.192711,-0.457689,-0.289067,-0.794934,0.2168,-0.192711,0.867201,0.650401,0.529956,1.035824,1.276713,1.228535,1.180357,1.204446,1.252624,0.819023,0.337245,0.626312,0.144534,0.144534,0.264978,0.096356,0.144534,-0.409512,-0.457689,-0.313156,-0.048178,-0.289067,-0.337245,-0.578134,-0.602223,-0.289067,-0.433601,-0.361334,-0.505867,1.132179,1.710313,1.734402,0.481778,0.457689,0.505867,0.289067,0.457689,0.554045,0.289067,-0.024089,0.144534,0.409512,0.144534,0.024089,0.072267,0.361334,0.481778,0.409512,0.144534,0.505867,0.361334,0.337245,0.602223,0.240889,0.457689,0.240889,0.481778,0.337245,0.481778,0.024089,-0.264978,0.337245,-0.987646,-0.939468,-0.819023,-1.084001,-0.987646,-0.843112,-0.963557,-0.939468,-0.819023,-0.963557,-0.915379,-0.867201,-0.939468,-0.987646,-0.867201,-1.059912,1.035824,0.770845,0.361334,0.843112,0.602223,0.843112,0.794934,-0.192711,-0.602223,-0.505867,-0.602223,-0.650401,-0.939468,-0.770845,-0.67449,-0.67449,-0.722668,-0.770845,-1.613958,-1.758491,-1.324891,-1.589869,-1.397157,-1.56578,-1.589869,-1.662135,-1.445335,-1.421246,-1.493513,-1.469424,-1.469424,-1.493513,-1.493513,-1.56578,-1.373068,-1.469424,-1.56578,-1.903025,-1.662135,-1.830758,-1.830758,-1.854847,-1.806669,-0.2168,1.638047,1.638047,1.445335,1.421246,1.541691,1.300802,1.34898,1.34898,1.132179,1.156268,1.469424,1.011735,0.457689,0.770845,0.457689,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 1
X 65114458 - 105916840 5cd65c07-f25d-46e2-b444-1c4898c03c30 t CTCTTCCTCGTTCTCCT -0.120444,1.312846,-0.371658,0.537986,0.941762,0.208771,-0.065697,1.585051,0.257752,-1.597095,0.313156,0.554045,0.473749,1.691042,0.586164,-0.004015,1.686224 0.048178,0.301593,0.115064,0.081887,0.108471,0.097022,0.109714,0.233252,0.143344,0.175387,0.104076,0.437332,0.640763,0.166475,0.18328,0.170003,0.136268 4,4,14,3,21,3,11,10,10,20,3,5,3,5,6,6,6 -0.072267,-0.072267,-0.168622,-0.168622,1.204446,1.589869,1.589869,0.867201,-0.313156,-0.264978,-0.409512,-0.433601,-0.361334,-0.313156,-0.554045,-0.433601,-0.337245,-0.626312,-0.361334,-0.289067,-0.361334,-0.144534,0.650401,0.457689,0.505867,0.939468,0.819023,0.794934,1.10809,0.939468,0.987646,1.011735,0.963557,0.89129,0.770845,0.915379,0.915379,0.963557,1.084001,1.011735,0.722668,1.10809,0.987646,0.867201,1.10809,0.867201,0.289067,0.264978,0.072267,0.072267,-0.048178,0.0,-0.240889,0.072267,-0.289067,-0.072267,-0.120445,-0.024089,0.0,-0.072267,1.710313,1.734402,1.56578,1.830758,1.56578,1.806669,1.541691,1.686224,1.421246,0.987646,0.433601,0.385423,0.192711,0.168622,0.264978,0.433601,0.144534,0.0,0.144534,0.409512,-1.10809,-1.493513,-1.613958,-1.445335,-1.589869,-1.734402,-1.830758,-1.734402,-1.734402,-1.734402,-1.830758,-1.469424,-1.541691,-1.589869,-1.397157,-1.56578,-1.758491,-1.710313,-1.686224,-1.373068,0.2168,0.457689,0.264978,0.096356,0.385423,1.156268,0.987646,0.144534,0.120445,-0.072267,1.373068,1.758491,1.686224,1.445335,1.951202,1.613958,0.89129,0.626312,0.554045,0.409512,0.698579,0.337245,0.240889,0.168622,-0.240889,-0.168622,0.0,-0.024089,1.710313,1.710313,1.734402,1.878936,1.662135,1.421246,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0 1

[PengNi]

@vahidAK , these two lines look fine to me. Did you split the extracted_feature file for parallel run? If you did, how did you split it?

[vahidAK]

No, I separated my fast5 files and separation was at fast5 level

[PengNi]

@vahidAK , I'd really appreciate it if you send me an sample input file for debug (onedrive or google drive or some other way).

Also, I suggest you to call mods from fast5 files directly, skip the extract_features step, see if this error is still raised.

[vahidAK]

Thanks, @PengNi , Sure! but first, let me try once more maybe there was sth wrong during extracting features. Regarding callin mods from fast5s, I actually did that but it just starts running without doing anything when I look at the log file.

[vahidAK]

Hi @PengNi , I extracted the features again and called modification again. This time it worked just fine, so maybe I did something wrong in feature extraction.

[PengNi]

@vahidAK ，OK, now I'm closing this issue. If you have any other problem, please tell me.