Using DE results from DESeq2

[gilbertlsk]

Hi, I am trying to use DE results from DESeq for Progeny. But I don't know how to feed in the data. In Step 3, the t values were used from the limma results. What would that be in DESeq results?

[adugourd]

hi,

Do you have t-values in the DESeq output ? If you can't access those, you can use either fold-changes or signed p-values instead.

Progeny's algorithm is essentially an enrichment analysis, similar to PAGE (https://bmcbioinformatics.biomedcentral.com/articles/10.1186/1471-2105-6-144). You may want to look up this tools if you are looking to get a better understanding of the input and output of such algorithm :)

Cheers,

Aurelien

[gilbertlsk]

Thanks for the reply. Have tried with fold-change. Got results from progeny and dorothea but no network results from carnival. The solver used was Coinor-CBC. Tried with 2 RNA seq datasets, both returned no network found. Perhaps I am missing something? Any pointers that can help me troubleshoot will be great.

Also found that the threads parameter for doesn't seem to work, all the while it is just running single thread. Could be my setup, am using WSL linux under Windows Environment. Will try on a purely linux setup.

[adugourd]

Hi, thread parameter might be tied to CPLEX solver indeed. may not work with CBC.

Can you please copy the message that carnival output when it finished ?

[gilbertlsk]

Below is the output for runCanival()

Writing constraints... Solving LP problem... Welcome to the CBC MILP Solver Version: 2.10.3 Build Date: Mar 24 2020

command line - /usr/bin/cbc testFile_1_1.lp -seconds 7200 -ratio 0 solve printi csv solu results_cbc_1_1.txt (default strategy 1)

CoinLpIO::is_invalid_name(): Name contains illegal character ' '

CoinLpIO::are_invalid_names(): Invalid name: vnames[53677]:

CoinLpIO::is_invalid_name(): Name c107927: contains illegal character ':'

CoinLpIO::are_invalid_names(): Invalid name: vnames[53678]: c107927:

CoinLpIO::readLp(): Invalid column names

Now using default column names. seconds was changed from 1e+100 to 7200 ratioGap was changed from 0 to 0 Continuous objective value is 10.7383 - 0.40 seconds Cgl0002I 2962 variables fixed Cgl0003I 0 fixed, 0 tightened bounds, 17453 strengthened rows, 635900 substitutions Cgl0003I 0 fixed, 0 tightened bounds, 16177 strengthened rows, 208370 substitutions Cgl0003I 0 fixed, 0 tightened bounds, 26054 strengthened rows, 6698 substitutions Cgl0003I 0 fixed, 0 tightened bounds, 9178 strengthened rows, 1392 substitutions Cgl0003I 0 fixed, 0 tightened bounds, 2813 strengthened rows, 8 substitutions Cgl0003I 0 fixed, 0 tightened bounds, 122 strengthened rows, 28 substitutions Cgl0003I 0 fixed, 0 tightened bounds, 42 strengthened rows, 0 substitutions Cgl0003I 0 fixed, 0 tightened bounds, 2 strengthened rows, 0 substitutions Cgl0004I processed model has 48758 rows, 13879 columns (11896 integer (11086 of which binary)) and 208301 elements Cbc0038I Initial state - 143 integers unsatisfied sum - 66.1782 Cbc0038I Pass 1: (11.67 seconds) suminf. 22.15842 (52) obj. 21.1308 iterations 1207 Cbc0038I Pass 2: (11.78 seconds) suminf. 14.80524 (57) obj. 26.8175 iterations 247 Cbc0038I Pass 3: (12.10 seconds) suminf. 8.33943 (39) obj. 29.6254 iterations 369 Cbc0038I Pass 4: (12.28 seconds) suminf. 6.29208 (40) obj. 29.6113 iterations 254 Cbc0038I Pass 5: (12.63 seconds) suminf. 4.30693 (37) obj. 29.6127 iterations 494 Cbc0038I Pass 6: (12.66 seconds) suminf. 2.80693 (34) obj. 29.6127 iterations 10 Cbc0038I Pass 7: (13.45 seconds) suminf. 5.00000 (10) obj. 41.6907 iterations 1184 Cbc0038I Pass 8: (13.48 seconds) suminf. 2.00000 (4) obj. 43.1983 iterations 12 Cbc0038I Pass 9: (13.72 seconds) suminf. 1.50000 (3) obj. 43.2983 iterations 258 Cbc0038I Pass 10: (13.75 seconds) suminf. 0.00000 (0) obj. 43.2983 iterations 26 Cbc0038I Solution found of 43.2983 Cbc0038I Relaxing continuous gives 43.2983 Cbc0038I Cleaned solution of 43.2983 Cbc0038I Before mini branch and bound, 11687 integers at bound fixed and 1760 continuous of which 740 were internal integer and 0 internal continuous Cbc0038I Full problem 48758 rows 13879 columns, reduced to 79 rows 42 columns Cbc0038I Mini branch and bound improved solution from 43.2983 to 39.6407 (13.99 seconds) Cbc0038I Round again with cutoff of 36.7529 Cbc0038I Pass 11: (14.06 seconds) suminf. 22.15842 (52) obj. 21.1308 iterations 0 Cbc0038I Pass 12: (15.41 seconds) suminf. 14.80524 (57) obj. 26.8175 iterations 1884 Cbc0038I Pass 13: (16.36 seconds) suminf. 8.33943 (39) obj. 29.6254 iterations 1030 Cbc0038I Pass 14: (16.78 seconds) suminf. 6.29208 (40) obj. 29.6113 iterations 488 Cbc0038I Pass 15: (18.72 seconds) suminf. 5.65842 (25) obj. 33.1823 iterations 1923 Cbc0038I Pass 16: (19.20 seconds) suminf. 2.65842 (19) obj. 33.1823 iterations 457 Cbc0038I Pass 17: (28.16 seconds) suminf. 2.65842 (19) obj. 33.1823 iterations 8695 Cbc0038I Pass 18: (29.27 seconds) suminf. 3.58845 (8) obj. 36.7529 iterations 1210 Cbc0038I Pass 19: (29.52 seconds) suminf. 3.22263 (8) obj. 36.7529 iterations 212 Cbc0038I Pass 20: (39.02 seconds) suminf. 2.42923 (8) obj. 36.7529 iterations 8749 Cbc0038I Pass 21: (39.05 seconds) suminf. 2.45515 (8) obj. 36.7529 iterations 1 Cbc0038I Pass 22: (40.25 seconds) suminf. 2.44136 (6) obj. 36.7529 iterations 1007 Cbc0038I Pass 23: (40.32 seconds) suminf. 1.44136 (4) obj. 36.7529 iterations 38 Cbc0038I Pass 24: (41.24 seconds) suminf. 0.37507 (5) obj. 36.7529 iterations 715 Cbc0038I Pass 25: (48.50 seconds) suminf. 20.92861 (51) obj. 36.7529 iterations 5619 Cbc0038I Pass 26: (54.21 seconds) suminf. 0.16832 (9) obj. 36.3219 iterations 5157 Cbc0038I Pass 27: (70.81 seconds) suminf. 0.17822 (6) obj. 36.3601 iterations 11965 Cbc0038I Pass 28: (72.48 seconds) suminf. 0.37141 (4) obj. 36.7529 iterations 1401 Cbc0038I Pass 29: (73.31 seconds) suminf. 0.15842 (4) obj. 36.5581 iterations 661 Cbc0038I Pass 30: (78.33 seconds) suminf. 36.03502 (94) obj. 36.7529 iterations 3914 Cbc0038I Pass 31: (81.17 seconds) suminf. 2.89645 (28) obj. 36.7529 iterations 2216 Cbc0038I Pass 32: (82.52 seconds) suminf. 0.20211 (13) obj. 36.7529 iterations 1002 Cbc0038I Pass 33: (83.32 seconds) suminf. 0.20718 (11) obj. 36.7529 iterations 451 Cbc0038I Pass 34: (84.45 seconds) suminf. 0.59014 (4) obj. 36.7529 iterations 874 Cbc0038I Pass 35: (85.29 seconds) suminf. 0.15842 (4) obj. 36.3581 iterations 707 Cbc0038I Pass 36: (90.39 seconds) suminf. 17.00321 (54) obj. 36.7529 iterations 3780 Cbc0038I Pass 37: (102.12 seconds) suminf. 0.15539 (6) obj. 36.7529 iterations 7975 Cbc0038I Pass 38: (103.44 seconds) suminf. 0.38588 (8) obj. 36.7529 iterations 1044 Cbc0038I Pass 39: (119.31 seconds) suminf. 0.38588 (8) obj. 36.7529 iterations 10534 Cbc0038I Pass 40: (120.16 seconds) suminf. 1.43164 (8) obj. 36.7529 iterations 634 Cbc0038I Rounding solution of 38.0709 is better than previous of 39.6407

Cbc0038I Before mini branch and bound, 11348 integers at bound fixed and 1517 continuous of which 726 were internal integer and 0 internal continuous Cbc0038I Full problem 48758 rows 13879 columns, reduced to 111 rows 56 columns Cbc0038I Mini branch and bound did not improve solution (120.26 seconds) Cbc0038I Round again with cutoff of 31.5548 Cbc0038I Pass 40: (120.30 seconds) suminf. 22.15842 (52) obj. 21.1308 iterations 0 Cbc0038I Pass 41: (123.48 seconds) suminf. 14.80524 (57) obj. 26.8175 iterations 4171 Cbc0038I Pass 42: (124.61 seconds) suminf. 8.33943 (39) obj. 29.6254 iterations 1016 Cbc0038I Pass 43: (126.00 seconds) suminf. 6.29208 (40) obj. 29.6113 iterations 1435 Cbc0038I Pass 44: (127.33 seconds) suminf. 4.33663 (28) obj. 29.6333 iterations 1381 Cbc0038I Pass 45: (143.51 seconds) suminf. 2.80693 (34) obj. 29.6127 iterations 12224 Cbc0038I Pass 46: (146.69 seconds) suminf. 5.62520 (20) obj. 31.5548 iterations 2640 Cbc0038I Pass 47: (147.67 seconds) suminf. 5.39012 (20) obj. 31.5548 iterations 1053 Cbc0038I Pass 48: (151.69 seconds) suminf. 6.66314 (20) obj. 31.5548 iterations 3187 Cbc0038I Pass 49: (156.66 seconds) suminf. 4.72125 (16) obj. 31.5548 iterations 4254 Cbc0038I Pass 50: (157.05 seconds) suminf. 4.15842 (18) obj. 31.5548 iterations 337 Cbc0038I Pass 51: (161.37 seconds) suminf. 4.72125 (16) obj. 31.5548 iterations 3540 Cbc0038I Pass 52: (166.22 seconds) suminf. 37.68901 (85) obj. 31.5548 iterations 3600 Cbc0038I Pass 53: (173.92 seconds) suminf. 4.65842 (19) obj. 31.5548 iterations 5496 Cbc0038I Pass 54: (175.09 seconds) suminf. 4.40522 (16) obj. 31.5548 iterations 777 Cbc0038I Pass 55: (187.42 seconds) suminf. 4.15842 (18) obj. 31.5548 iterations 8958 Cbc0038I Pass 56: (187.58 seconds) suminf. 4.26079 (18) obj. 31.5548 iterations 248 Cbc0038I Pass 57: (194.03 seconds) suminf. 34.93939 (78) obj. 31.5548 iterations 4336 Cbc0038I Pass 58: (198.29 seconds) suminf. 4.15842 (18) obj. 31.5548 iterations 3154 Cbc0038I Pass 59: (199.07 seconds) suminf. 4.40522 (16) obj. 31.5548 iterations 646 Cbc0038I Pass 60: (203.17 seconds) suminf. 47.81550 (105) obj. 31.5548 iterations 3120 Cbc0038I Pass 61: (210.72 seconds) suminf. 4.20832 (20) obj. 31.5548 iterations 5794 Cbc0038I Pass 62: (210.75 seconds) suminf. 4.20832 (20) obj. 31.5548 iterations 3 Cbc0038I Pass 63: (214.43 seconds) suminf. 54.77561 (128) obj. 31.5548 iterations 2887 Cbc0038I Pass 64: (217.87 seconds) suminf. 5.50420 (25) obj. 31.5548 iterations 2637 Cbc0038I Pass 65: (218.54 seconds) suminf. 4.26671 (20) obj. 31.5548 iterations 348 Cbc0038I Pass 66: (220.74 seconds) suminf. 4.26671 (20) obj. 31.5548 iterations 1758 Cbc0038I Pass 67: (222.24 seconds) suminf. 4.57619 (20) obj. 31.5548 iterations 1168 Cbc0038I Pass 68: (224.43 seconds) suminf. 4.57619 (20) obj. 31.5548 iterations 1712 Cbc0038I Pass 69: (228.03 seconds) suminf. 4.26671 (20) obj. 31.5548 iterations 2799 Cbc0038I No solution found this major pass Cbc0038I Before mini branch and bound, 11201 integers at bound fixed and 1465 continuous of which 731 were internal integer and 0 internal continuous Cbc0038I Mini branch and bound did not improve solution (228.07 seconds) Cbc0038I After 228.07 seconds - Feasibility pump exiting with objective of 38.0709 - took 216.93 seconds Cbc0012I Integer solution of 38.070864 found by feasibility pump after 0 iterations and 0 nodes (228.09 seconds) Cbc0038I Full problem 48758 rows 13879 columns, reduced to 28 rows 18 columns Cbc0031I 470 added rows had average density of 18.431915 Cbc0013I At root node, 470 cuts changed objective from 10.762611 to 19.568592 in 11 passes Cbc0014I Cut generator 0 (Probing) - 2371 row cuts average 2.1 elements, 1 column cuts (1 active) in 1.266 seconds - new frequency is 1 Cbc0014I Cut generator 1 (Gomory) - 278 row cuts average 239.7 elements, 0 column cuts (0 active) in 0.365 seconds - new frequency is 1 Cbc0014I Cut generator 2 (Knapsack) - 2 row cuts average 4.5 elements, 0 column cuts (0 active) in 0.107 seconds - new frequency is -100 Cbc0014I Cut generator 3 (Clique) - 0 row cuts average 0.0 elements, 0 column cuts (0 active) in 0.003 seconds - new frequency is -100 Cbc0014I Cut generator 4 (MixedIntegerRounding2) - 3007 row cuts average 3.0 elements, 0 column cuts (0 active) in 0.269 seconds - new frequency is 1 Cbc0014I Cut generator 5 (FlowCover) - 1 row cuts average 4.0 elements, 0 column cuts (0 active) in 0.147 seconds - new frequency is -100 Cbc0014I Cut generator 6 (TwoMirCuts) - 3276 row cuts average 6.0 elements, 0 column cuts (0 active) in 0.600 seconds - new frequency is 1 Cbc0014I Cut generator 7 (ZeroHalf) - 30 row cuts average 2.0 elements, 0 column cuts (0 active) in 1.280 seconds - new frequency is -100 Cbc0010I After 0 nodes, 1 on tree, 38.070864 best solution, best possible 19.568592 (264.11 seconds) Cbc0012I Integer solution of 21.883803 found by DiveCoefficient after 9154 iterations and 5 nodes (299.17 seconds) Cbc0038I Full problem 48758 rows 13879 columns, reduced to 20 rows 14 columns Cbc0012I Integer solution of 19.568594 found by RINS after 33391 iterations and 43 nodes (396.40 seconds) Cbc0001I Search completed - best objective 19.56859438352776, took 33391 iterations and 43 nodes (396.40 seconds) Cbc0032I Strong branching done 602 times (40503 iterations), fathomed 7 nodes and fixed 1 variables Cbc0035I Maximum depth 13, 45 variables fixed on reduced cost Cuts at root node changed objective from 10.7626 to 19.5686 Probing was tried 51 times and created 5387 cuts of which 0 were active after adding rounds of cuts (1.561 seconds) Gomory was tried 50 times and created 366 cuts of which 0 were active after adding rounds of cuts (0.711 seconds) Knapsack was tried 11 times and created 2 cuts of which 0 were active after adding rounds of cuts (0.107 seconds) Clique was tried 11 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.003 seconds) MixedIntegerRounding2 was tried 50 times and created 5751 cuts of which 0 were active after adding rounds of cuts (1.160 seconds) FlowCover was tried 11 times and created 1 cuts of which 0 were active after adding rounds of cuts (0.147 seconds) TwoMirCuts was tried 50 times and created 3305 cuts of which 0 were active after adding rounds of cuts (0.915 seconds) ZeroHalf was tried 11 times and created 30 cuts of which 0 were active after adding rounds of cuts (1.280 seconds) ImplicationCuts was tried 23 times and created 88 cuts of which 0 were active after adding rounds of cuts (0.012 seconds)

Result - Optimal solution found

Objective value: 19.56859438 Enumerated nodes: 43 Total iterations: 33391 Time (CPU seconds): 397.23 Time (Wallclock seconds): 399.13

Option for printingOptions changed from normal to csv Total time (CPU seconds): 397.69 (Wallclock seconds): 400.01

── Column specification ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── cols( name = col_character(), solution = col_double() )

No network was generated for this setting.. No result to be written Warning messages: 1: In controlNodeIdentifiers(netObj = netObj) : Your network contains identifiers with '-' symbol and they will be replaced with '' 2: In controlNodeIdentifiers(netObj = netObj) : Your network contains identifiers with '/' symbol and they will be replaced with ''

[adugourd]

This seems to be a problem with the optimisation. 2 solutions are found but 1 of them seems to be an empty network, with a slightly lower score. Thus it is the solution that is returned in the end. I will check with the optimisation expert what to do.

In the meantime, you can try different parameters or different optimiser.

[gilbertlsk]

I have tried with CPLEX, but the community edition. It didn't finished running as problem size limits exceeded. I do not have access to the academic edition.

[adugourd]

Does the problem size limit arise from the memory limitation of the machine you are using ?

[enio23]

Hi. I am a bit busy at this moment but I will try to check this as soon as I can. Meanwhile @gilbertlsk can you try to give a higher time-limit? Maybe a network solution with zero nodes and edges might have been retrieved for the current settings and a larger amount of time might be needed to get a meaningful result.

I would indeed suggest using CPLEX if possible since CARNIVAL was mainly maintained for this kind of solver. The Academic edition of CPLEX is supposed to be free to use for academic purposes.

[gilbertlsk]

@adugourd I am runningon a laptop with 16GB, when running, I only saw utilisation of about 82%, don't think it is memory limitiation. I think it is more likely that the community edition limit compared to other edition as stated here.

@enio23 Will try with higher time-limit and see if I can get the academic edition.

[adugourd]

Hi @gilbertlsk , did you solve your issue ?

[gilbertlsk]

The IBM portal to register for academic account has been down since I last commented. I couldn't try with the CPLEX academic edition.

[adugourd]

I've just checked the IBM portal and succesfully logged in to the cplex download section. You may want to try again or check if something else is wrong.

[gilbertlsk]

Yes, the portal is working now. It wasn't working last night. I have tested with IBM Cplex. Finally getting results from runCarnival. Thanks.