search

Xinglab / rmats-turbo

Other
209 stars 48 forks source link

Error running test_rmats script, some use cases failed #215

Open 13679253292 opened 1 year ago

13679253292 commented 1 year ago

Hi, when I run ./test_rmats, there are 6 test cases that are not successfully executed, I've pasted the complete output from running ./test_rmats below. Can you help me to troubleshoot?

Thank you very much for your help,

  **[root@55-12-19-171 rmats-4.1.2]# ./test_rmats

test (tests.allow_clipping.test.ClippingAllowedTest) ... ok test (tests.allow_clipping.test.ClippingNotAllowedTest) ... ok test (tests.alternative_3_splice_site_novel.test.NovelJunction) ... ok test (tests.alternative_3_splice_site_novel.test.NovelSpliceSite) ... ok test (tests.alternative_5_splice_site_novel.test.NovelJunction) ... ok test (tests.alternative_5_splice_site_novel.test.NovelSpliceSite) ... ok test (tests.fixed_event_set.test.Test) ... ok test (tests.mutually_exclusive_exons_novel.test.NovelJunction) ... ok test (tests.mutually_exclusive_exons_novel.test.NovelSpliceSite) ... ok test (tests.only_one_sample.test.StatOffTest) ... ok test (tests.only_one_sample.test.StatOnTest) ... ok test (tests.overlapped_gene.test.Test) ... ok test (tests.paired_stats.test.FilteredEventTest) ... FAIL test (tests.paired_stats.test.OneEventTest) ... FAIL test (tests.paired_stats.test.TwoEventTest) ... FAIL test (tests.prep_post.test.Test) ... ok test (tests.retained_intron_novel.test.NovelJunction) ... ok test (tests.retained_intron_novel.test.NovelSpliceSite) ... ok test (tests.skipped_exon_basic.test.Test) ... ok test (tests.skipped_exon_novel.test.NovelJunction) ... ok test (tests.skipped_exon_novel.test.NovelSpliceSite) ... ok test (tests.skipped_exon_novel.test.NovelSpliceSiteNotNovelJunction) ... ok test (tests.stat_large_file.test.Test) ... ok test (tests.stranded.test.PairedFirstStrandTest) ... ok test (tests.stranded.test.PairedSecondStrandTest) ... ok test (tests.stranded.test.PairedUnstrandedTest) ... ok test (tests.stranded.test.SingleEndFirstStrandTest) ... ok test (tests.stranded.test.SingleEndSecondStrandTest) ... ok test (tests.stranded.test.SingleEndUnstrandedTest) ... ok test (tests.task_stat.test.Test) ... FAIL test (tests.variable_read_length.test.Length1Test) ... ok test (tests.variable_read_length.test.Length1VariableTest) ... FAIL test (tests.variable_read_length.test.Length2Test) ... ok test (tests.variable_read_length.test.Length2VariableTest) ... FAIL test (tests.variable_read_length.test.NoLengthTest) ... ok

==================================================================== == FAIL: test (tests.paired_stats.test.FilteredEventTest)

Traceback (most recent call last): File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/tests/paired_stats/test.py", line 314, in test self._run_test() File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/tests/base_test.py", line 17, in _run_test self._check_results() File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/tests/paired_stats/test.py", line 373, in _check_results self._check_no_error_results() File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/tests/base_test.py", line 61, in _check_no_error_results self.assertEqual(self._rmats_return_code, 0) AssertionError: 1 != 0

====================================================================== FAIL: test (tests.paired_stats.test.OneEventTest)

Traceback (most recent call last): File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/tests/paired_stats/test.py", line 156, in test self._run_test() File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/tests/base_test.py", line 17, in _run_test self._check_results() File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/tests/paired_stats/test.py", line 199, in _check_results self._check_no_error_results() File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/tests/base_test.py", line 61, in _check_no_error_results self.assertEqual(self._rmats_return_code, 0) AssertionError: 1 != 0

====================================================================== FAIL: test (tests.paired_stats.test.TwoEventTest)

Traceback (most recent call last): File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/tests/paired_stats/test.py", line 229, in test self._run_test() File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/tests/base_test.py", line 17, in _run_test self._check_results() File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/tests/paired_stats/test.py", line 280, in _check_results self._check_no_error_results() File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/tests/base_test.py", line 61, in _check_no_error_results self.assertEqual(self._rmats_return_code, 0) AssertionError: 1 != 0

====================================================================== FAIL: test (tests.task_stat.test.Test)

Traceback (most recent call last): File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/tests/task_stat/test.py", line 68, in test self._run_test() File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/tests/base_test.py", line 17, in _run_test self._check_results() File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/tests/task_stat/test.py", line 246, in _check_results self._check_results_selected_stat() File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/tests/task_stat/test.py", line 391, in _check_results_selected_stat self._check_no_error_results() File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/tests/base_test.py", line 67, in _check_no_error_results self.assertEqual(err_lines, list()) AssertionError: Lists differ: ['Traceback (most recent call last):\n', '[1023 chars]y\n'] != []

First list contains 10 additional elements. First extra element 0: 'Traceback (most recent call last):\n'

Diff is 1165 characters long. Set self.maxDiff to None to see it.

====================================================================== FAIL: test (tests.variable_read_length.test.Length1VariableTest)

Traceback (most recent call last): File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/tests/variable_read_length/test.py", line 251, in test self._run_test() File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/tests/base_test.py", line 17, in _run_test self._check_results() File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/tests/variable_read_length/test.py", line 263, in _check_results self._check_no_error_results() File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/tests/base_test.py", line 67, in _check_no_error_results self.assertEqual(err_lines, list()) AssertionError: Lists differ: ['Traceback (most recent call last):\n', '[534 chars]y\n'] != []

First list contains 5 additional elements. First extra element 0: 'Traceback (most recent call last):\n'

====================================================================== FAIL: test (tests.variable_read_length.test.Length2VariableTest)

Traceback (most recent call last): File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/tests/variable_read_length/test.py", line 374, in test self._run_test() File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/tests/base_test.py", line 17, in _run_test self._check_results() File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/tests/variable_read_length/test.py", line 386, in _check_results self._check_no_error_results() File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/tests/base_test.py", line 67, in _check_no_error_results self.assertEqual(err_lines, list()) AssertionError: Lists differ: ['Traceback (most recent call last):\n', '[534 chars]y\n'] != []

First list contains 5 additional elements. First extra element 0: 'Traceback (most recent call last):\n'

Ran 35 tests in 48.343s

FAILED (failures=6)**

I looked up the error log（command_output），There are three main types of errors： (1)./tests/paired_stats/filtered_event_test/command_output: Traceback (most recent call last): File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/rmats.py", line 595, in main() File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/rmats.py", line 583, in main args.paired_stats, python_executable, root_dir) File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/rmats.py", line 422, in process_counts cwd=sec_tmp) File "/usr/lib64/python2.7/subprocess.py", line 172, in call return Popen(*popenargs, **kwargs).wait() File "/usr/lib64/python2.7/subprocess.py", line 394, in init errread, errwrite) File "/usr/lib64/python2.7/subprocess.py", line 1047, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory (2)./tests/paired_stats//task_stat/command_output: Traceback (most recent call last): File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/rMATS_P/FDR.py", line 53, in ifile=open(sys.argv[1]);title=ifile.readline(); IOError: [Errno 2] No such file or directory: '/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats- 4.1.2/tests/task_stat/out_select/tmp/JC_SE/rMATS_result_P-V.txt' paste: /wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/tests/task_stat/out_select/tmp/JC_SE/rMATS_result_FDR.txt: No such file or directory Traceback (most recent call last): File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/rMATS_P/FDR.py", line 53, in ifile=open(sys.argv[1]);title=ifile.readline(); IOError: [Errno 2] No such file or directory: '/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats- 4.1.2/tests/task_stat/out_select/tmp/JCEC_SE/rMATS_result_P-V.txt' paste: /wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/tests/task_stat/out_select/tmp/JCEC_SE/rMATS_result_FDR.txt: No such file or directory (3）./tests/variable_read_length/length_1_variable/command_output/ Traceback (most recent call last): File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/rMATS_P/FDR.py", line 53, in ifile=open(sys.argv[1]);title=ifile.readline(); IOError: [Errno 2] No such file or directory: '/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats- 4.1.2/tests/variable_read_length/length_1_variable/out/tmp/JCEC_SE/rMATS_result_P-V.txt' paste: /wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats- 4.1.2/tests/variable_read_length/length_1_variable/out/tmp/JCEC_SE/rMATS_result_FDR.txt: No such file or directory

EricKutschera commented 1 year ago

These errors are for the paired model:

test (tests.paired_stats.test.FilteredEventTest) ... FAIL
test (tests.paired_stats.test.OneEventTest) ... FAIL
test (tests.paired_stats.test.TwoEventTest) ... FAIL

You may not have PAIRADISE installed: https://github.com/Xinglab/rmats-turbo/blob/v4.1.2/build_rmats#L75

These errors are for the regular statistical model:

test (tests.task_stat.test.Test) ... FAIL
test (tests.variable_read_length.test.Length1VariableTest) ... FAIL
test (tests.variable_read_length.test.Length2VariableTest) ... FAIL

The message in the log just shows that the output file from the statistical model is not found: IOError: [Errno 2] No such file or directory: '/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/tests/task_stat/out_select/tmp/JCEC_SE/rMATS_result_P-V.txt' This is the code that calls the statistical model to create that file: https://github.com/Xinglab/rmats-turbo/blob/v4.1.2/rmats.py#L428 The output is redirected to os.devnull because when that command runs correctly it produces a lot of output showing progress. In your case there is an error, but that is being sent to devnull instead of being shown. You could edit rmats.py to remove , stdout=FNULL and then re-run the tests to hopefully see a more helpful error message. Or you can manually run the command for a failed test like: ./rMATS_C/rMATSexe -i ./tests/task_stat/out_select/JC.raw.input.SE.txt -t 1 -o ./rMATS_result_P-V.txt -c 0.0001 to see the output

13679253292 commented 1 year ago

Hi @EricKutschera,

Thank you very much for your quick reply. Installed PAIRADISE according to your instructions, the paired_stats related use case passed, but the task_stat use case has been failing, and the error message is as follows:

FAIL: test (tests.task_stat.test.Test)

Traceback (most recent call last): File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/tests/task_stat/test.py", line 68, in test self._run_test() File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/tests/base_test.py", line 17, in _run_test self._check_results() File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/tests/task_stat/test.py", line 246, in _check_results self._check_results_selected_stat() File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/tests/task_stat/test.py", line 391, in _check_results_selected_stat self._check_no_error_results() File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/tests/base_test.py", line 67, in _check_no_error_results self.assertEqual(err_lines, list()) AssertionError: Lists differ: ['Traceback (most recent call last):\n', '[486 chars]y\n'] != []

First list contains 5 additional elements. First extra element 0: 'Traceback (most recent call last):\n'

I also found that the use cases of Length1VariableTest, Length2VariableTest, and stat_large_file may fail when ./test_rmats are repeatedly executed. I think this should be a similar problem, so I posted the error message of Length1VariableTest, as follows: FAIL: test (tests.variable_read_length.test.Length1VariableTest)

Traceback (most recent call last): File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/tests/variable_read_length/test.py", line 251, in test self._run_test() File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/tests/base_test.py", line 17, in _run_test self._check_results() File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/tests/variable_read_length/test.py", line 263, in _check_results self._check_no_error_results() File "/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/tests/base_test.py", line 67, in _check_no_error_results self.assertEqual(err_lines, list()) AssertionError: Lists differ: ['Traceback (most recent call last):\n', '[534 chars]y\n'] != []

First list contains 5 additional elements. First extra element 0: 'Traceback (most recent call last):\n'

Finally, thank you for your patient guidance.

EricKutschera commented 1 year ago

The main takeaway from those error messages is: IOError: [Errno 2] No such file or directory: '/wangming/soft/rMATs-4.1.2/pkg/rmats-4.1.2-hpc/rmats-4.1.2/tests/task_stat/out_select/tmp/JC_SE/rMATS_result_P-V.txt' That message says that the output file from the statistical model is not found. This is the code that calls the statistical model to create that file: https://github.com/Xinglab/rmats-turbo/blob/v4.1.2/rmats.py#L428

The stdout from the statistical model is redirected to os.devnull to avoid printing a lot of progress update messages. In your case there is an error, but the code is sending the message to devnull. You could edit rmats.py to remove , stdout=FNULL and then re-run the tests to hopefully see a more helpful error message. Or you can manually run the command for a failed test like: ./rMATS_C/rMATSexe -i ./tests/task_stat/out_select/JC.raw.input.SE.txt -t 1 -o ./rMATS_result_P-V.txt -c 0.0001 When running manually there is no redirect to devnull so you should be able to see an error message

I was able to run ./test_rmats repeatedly with all tests passing. I'm not sure what could cause a test to fail only when re-running ./test_rmats

13679253292 commented 1 year ago

EricKutschera，I don't know if I understand what you said. According to my understanding, after deleting stdout=FNULL in rmats.py, run the use case again. The above result is the result of running after I delete stdout=FNULL. From the result I can't locate the cause of the problem. In addition, add the results of ./rMATS_C/rMATSexe -i ./tests/task_stat/out_select/JC.raw.input.SE.txt -t 1 -o ./rMATS_result_P-V.txt -c 0.0001, as follows:

number of thread=1; input file=./tests/task_stat/out_select/JC.raw.input.SE.txt; output folder=./rMATS_result_P-V.txt; cutoff=0.0001; Testing 0 Testing 1 3 return from: 0 3 return from: 1 Total Wallclock time taken 1 seconds 320 milliseconds Wallclock time per thread taken 1 seconds 320 milliseconds Time for func(single thread): 0 seconds 0 milliseconds

13679253292 commented 1 year ago

Is it related to samtools？

EricKutschera commented 1 year ago

Based on the output you posted, it seems like the test should have passed. If any test is still failing can you post what ./test_rmats outputs and also the files in ./tests/{failed_test_name}/command_output/rmats_stderr and rmats_stdout

Now that you deleted the stdout=FNULL portion of the statistical model command there should hopefully either be an error message or the test will pass

13679253292 commented 1 year ago

image This is the result of running the same command twice, the first time an error occurs, the second time it succeeds, what is the reason?

EricKutschera commented 1 year ago

I'm not sure what could cause that segfault. You could try running under gdb similar to this post: https://github.com/Xinglab/rmats-turbo/issues/101#issuecomment-804881108

If you compiled rmats then you can try re-compiling with -O0 -ggdb as described in that post to get a better error message from gdb