Segmentation fault in running m6anet-run_inference

[kwonej0617]

Hi Developer!

I was running m6anet with demo data.

m6anet-dataprep appeared to run successfully though it had a warning message. m6anet-dataprep --eventalign demo/eventalign.txt --out_dir dataprep_out --n_processes 4

/home/ek81w/.conda/envs/m6anet/lib/python3.8/site-packages/m6anet/scripts/dataprep.py:143: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy chunk_split['line_length'] = np.array(lines) /home/ek81w/.conda/envs/m6anet/lib/python3.8/site-packages/m6anet/scripts/dataprep.py:101: PerformanceWarning: indexing past lexsort depth may impact performance. pos_end += eventalign_result.loc[index]['line_length'].sum()

However, when I run m6anet-run_inference, it showed Segmentation fault message. m6anet-run_inference --input_dir dataprep_out --out_dir inference_out --infer_mod-rate --n_processes 4 I tried increasing the memory request, but it didn't work. By chance, if you know an idea to fix this problem, please let me know.

Thank you.

EuiJin

[chrishendra93]

Hi @kwonej0617, this is the first time I encounter this segmentation fault issue. Can you show me the first couple of lines of data.json, data.readcount, and data.index files?

[kwonej0617]

Hi @chrishendra93! Thank you for your quick reply.

{"ENST00000361055.8":{"549":{"TGGACTG":[[0.00996,2.666,120.7,0.00498,6.486,125.0,0.00465,1.6779999999999997,93.4],[0.00332,3.6,119.7,0.00498,8.115,124.2,0.00266,6.294,89.4],[0.00299,4.273,116.1,0.00564,5.211,127.7,0.00299,6.636,91.9],[0.01195,2.415,122.0,0.01295,11.208,124.9,0.00398,1.534,91.8],[0.00498,2.524,120.6,0.014279999999999998,7.385,127.6,0.00664,2.133,86.8],[0.00498,2.377,128.2,0.00332,16.305,111.4,0.00232,2.091,90.6],[0.00266,3.741,120.4,0.00797,3.504,127.7,0.00299,5.565,93.2],[0.00398,6.171,119.1,0.00398,8.95,126.9,0.00896,2.58,91.2],[0.00398,2.217,119.8,0.0073,5.797,125.8,0.00332,4.15,90.6],[0.00266,3.091,119.2,0.0093,6.720999999999999,127.5,0.00631,7.571,94.1],[0.00697,4.316,119.2,0.00631,7.902999999999999,123.8,0.00332,1.9490000000000003,94.6]]}}} {"ENST00000361055.8":{"554":{"TGGACTC":[[0.01096,2.923,123.7,0.00531,1.579,129.0,0.00332,2.647,91.1],[0.00598,3.351,120.7,0.00896,11.744999999999997,122.3,0.00332,2.316,87.6],[0.00299,2.364,125.2,0.007640000000000001,4.523,122.5,0.00498,2.74,92.6],[0.00797,3.113,120.8,0.00365,9.832,123.6,0.00232,1.702,89.7],[0.00863,2.746,111.1,0.007640000000000001,2.765,126.0,0.00398,6.800999999999999,93.8],[0.00232,9.756,107.8,0.01826,7.024,125.0,0.01295,2.41,91.2],[0.00365,1.96,122.5,0.007640000000000001,9.402,122.5,0.00564,3.9180000000000006,92.4],[0.00531,3.01,122.8,0.00332,0.67,131.8,0.00398,3.389,96.1],[0.01328,8.426,120.4,0.00299,1.564,129.6,0.00664,2.967,94.2],[0.01096,2.604,122.5,0.0073,4.883,123.0,0.00266,1.752,91.8],[0.01494,3.111,124.7,0.00432,14.92,110.5,0.00797,3.285,91.0],[0.0093,2.255,111.0,0.01461,3.6570000000000005,123.8,0.00598,3.7789999999999995,90.5]]}}} transcript_id,transcript_position,n_reads ENST00000361055.8,549,11 ENST00000361055.8,554,12 ENST00000475035.5,133,3 ENST00000222329.8,309,11 ENST00000222329.8,2496,15 ENST00000222329.8,2631,23 ENST00000523944.5,72,1 ENST00000523944.5,2196,14 ENST00000523944.5,2462,17 ENST00000523944.5,3348,30 ENST00000262105.6,34,14 transcript_id,transcript_position,start,end ENST00000361055.8,549,0,773 ENST00000361055.8,554,773,1645 ENST00000475035.5,133,1645,1910 ENST00000222329.8,309,1910,2724 ENST00000222329.8,2496,2724,3797 ENST00000222329.8,2631,3797,5388 ENST00000523944.5,72,5388,5492 ENST00000523944.5,2196,5492,6410 ENST00000523944.5,2462,6410,7656 ENST00000523944.5,3348,7656,9682 ENST00000262105.6,34,9682,10662 ENST00000262105.6,2274,10662,11679 Thank you for your help in advance!

[chrishendra93]

They look fine to me. Can you run it without the --infer_mod-rate option?

[kwonej0617]

I got the same Segmentation fault even in running without --infer_mod-rate option. Actually, the Segmentation fault appears when I run the command, m6anet-run_inference without any requirement and option.

Do you know how much memory it takes when running m6anet-run_inference?

[chrishendra93]

I am not too sure, should not be that much. May I know your RAM size? Can you try reducing the batchsize? This can be done with --batch_size option, maybe try --batch_size 1 or some other smaller number?

[kwonej0617]

Hi,

Is this a proper way to check RAM size? [ek81w@ghpcc06 m6anet]$ grep MemTotal /proc/meminfo MemTotal: 264,498,876 kB

Also, I run the m6anet-run_inference --input_dir dataprep_out --out_dir inference_out --infer_mod-rate --n_processes 4 --batch_size 1 and also got Segmentation fault error. :(

[chrishendra93]

This feels like a cpp related error instead of python especially since batch size 1 does not work. Can you show me the full error, also do you happen to have another machine that you can try for this?

[kwonej0617]

Sure, Here is the error log file.

Conda is an open source package management system and environment management system.

To use conda packages, first run source /share/pkg/condas/2018-05-11/bin/activate

Then, to list available environments conda env list

To activate a conda enviroment source activate [environment_name] e.g.: source activate biopython_1.71_py35h637b7d7_0

Your current environment will be displayed on your shell prompt.

To configure conda so you can install your own environments, you can run:

cp /share/pkg/condas/2018-05-11/condarc_example ~/.condarc

This config tells conda to install environment and package files under a .conda directory in your home, and allow you to install your own environments. Modify the .condarc file to change where your conda environment files are stored.

/home/ek81w/.lsbatch/1658517905.8853878.shell: line 15: 383887 Segmentation fault m6anet-run_inference --input_dir dataprep_out --out_dir inference_out --infer_mod-rate --n_processes 4 --batch_size

Also, my colleague run the command line for me and she got the same error message.

[chrishendra93]

Okay let me check this on my machine. Are you running the demo data? Which os are you using?

[kwonej0617]

Yes, I was running the demo data provided. I used linux.

Thank you so much!

[chrishendra93]

Sorry can you also tell me which python version are you using?

Thank you!

[kwonej0617]

No problem. I used python=3.8.

Thank you so much for your help!

[kwonej0617]

Hello, Christopher!

I just wanted to check if you have solved this problem. I asked IT administration in my school about Segmentation fault issue. He said it's caused by a glibc mis-match and recommend that I should use a singularity container with the appropriate package versions. Do you think it would resolve the problem? Also, do you have a plan to make a singularity container?

I am looking forward to hearing from you! Thank you.

[chrishendra93]

Hi @kwonej0617, sorry but currently I am overseas so it is kind of hard to troubleshoot this. I am going to be back next week and I can take a look after. There was a plan to make a docker container for this too

[kwonej0617]

No problem! I am looking forward to hearing from you! Have a safe trip.

[chrishendra93]

hi @kwonej0617, sorry for the delay

I could not replicate the error message with linux, but just in case, I have updated the package because I realized there was a missing dependency in setup.py file. Can you try installing the latest version through pip / from running python setup.py install on a new conda environment to see if it's working?

Thank you

Regards

Christopher Hendra

[chrishendra93]

hi @kwonej0617 , from your recent issues, I am assuming that you have managed to run m6anet on your machine? If so I am going to close this issue

Thanks!