firasmidani / amiga

Analysis of Microbial Growth Assays
https://firasmidani.github.io/amiga/
GNU General Public License v3.0
12 stars 3 forks source link

IndexError: list index out of range #17

Open ofleitas opened 1 year ago

ofleitas commented 1 year ago

Hello I am trying to normalize my pooled data by division, but I got an error message.

C:\Users\ofm83\Documents\Projects_python\amiga-master>python amiga.py normalize --input C:\Users\ofm83\Documents\A.baumannii_amiga\summary\Fit_pool_summary.txt --group-by "Isolate" --normalize-method "division" --normalize-by "Control" --ve rbose

-----------------------------------------------------

User provided the following command-line arguments:

-----------------------------------------------------

input...............C:\Users\ofm83\Documents\A.baumannii_amiga\summary\Fit_pool_summary.txt over_write..........False verbose.............True group_by............Isolate normalize_by........Control normalize_method....division

-------------------------------

AMiGA is parsing your file(s)

-------------------------------

Traceback (most recent call last): File "amiga.py", line 400, in AMiGA() File "amiga.py", line 90, in init getattr(self, args.command)() File "amiga.py", line 304, in normalize normalize(args) File "C:\Users\ofm83\Documents\Projects_python\amiga-master\libs\normalize.py", line 56, in main df = normalizeParameters(args,df) File "C:\Users\ofm83\Documents\Projects_python\amiga-master\libs\normalize.py", line 115, in normalizeParameters controlby = checkParameterCommand(args.normalize_by) File "C:\Users\ofm83\Documents\Projects_python\amiga-master\libs\interface.py", line 132, in checkParameterCommand lines_values = [[jj.strip() for jj in re.split(sep,ii.split(':')[1])] for ii in lines] File "C:\Users\ofm83\Documents\Projects_python\amiga-master\libs\interface.py", line 132, in lines_values = [[jj.strip() for jj in re.split(sep,ii.split(':')[1])] for ii in lines] IndexError: list index out of range

C:\Users\ofm83\Documents\Projects_python\amiga-master>

firasmidani commented 1 year ago

Thank you for sharing the command and error trace, it's quite helpful.

Based on your command, it seems that Fit_pool_summary.tx should have a column called Control and you are trying to normalize your data based on this column.

If you use the normalize function and do not call the --normalize-by argument, then, AMiGA will use the Control column by default. If you would still like to be explicit and call the --normalize-by argument, you will need to tell AMiGA which column in your summary file distinguish treatments and control (e.g. Control column) but you also need to tell AMiGA which value in the Control column corresponds to your control wells/conditions (e.g. Control:1). So, you can do either of the following commands:

option 1

Here, I specify which value in the Control column corresponds to control wells by including ":1" in the normalize-by value

C:\Users\ofm83\Documents\Projects_python\amiga-master>python amiga.py normalize --input C:\Users\ofm83\Documents\A.baumannii_amiga\summary\Fit_pool_summary.txt --group-by "Isolate" --normalize-method "division" --normalize-by "Control:1" --verbose

option 2

Here, we assume that your control rows/wells have values of 1 in the Control column (this is the default behavior by AMiGA), so we do not use the --normalize-by argument at all.

C:\Users\ofm83\Documents\Projects_python\amiga-master>python amiga.py normalize --input C:\Users\ofm83\Documents\A.baumannii_amiga\summary\Fit_pool_summary.txt --group-by "Isolate" --normalize-method "division"  --verbose

Try these out and let me know if you are still having this issue.

ofleitas commented 1 year ago

Hello

The two options apparently worked. However, when I checked the output file, there were no normalized values. Only there were the values (1) of the control. Something similar happened in the summary report, that there were no values of fold change, only the values of the control (1). What can I do? Thank you very much for the help. Best regards, Osmel

On Wed, May 31, 2023 at 1:44 PM Firas Midani @.***> wrote:

Thank you for sharing the command and error trace, it's quite helpful.

Based on your command, it seems that Fit_pool_summary.tx should have a column called Control and you are trying to normalize your data based on this column.

If you use the normalize function and do not call the --normalize-by argument, then, AMiGA will use the Control column by default. If you would still like to be explicit and call the --normalize-by argument, you will need to tell AMiGA which column in your summary file distinguish treatments and control (e.g. Control column) but you also need to tell AMiGA which value in the Control column corresponds to your control wells/conditions (e.g. Control:1). So, you can do either of the following commands: option 1

Here, I specify which value in the Control column corresponds to control wells by including ":1" in the normalize-by value

C:\Users\ofm83\Documents\Projects_python\amiga-master>python amiga.py normalize --input C:\Users\ofm83\Documents\A.baumannii_amiga\summary\Fit_pool_summary.txt --group-by "Isolate" --normalize-method "division" --normalize-by "Control:1" --verbose

option 2

Here, we assume that your control rows/wells have values of 1 in the Control column (this is the default behavior by AMiGA), so we do not use the --normalize-by argument at all.

C:\Users\ofm83\Documents\Projects_python\amiga-master>python amiga.py normalize --input C:\Users\ofm83\Documents\A.baumannii_amiga\summary\Fit_pool_summary.txt --group-by "Isolate" --normalize-method "division" --verbose

Try these out and let me know if they solved your problem.

— Reply to this email directly, view it on GitHub https://github.com/firasmidani/amiga/issues/17#issuecomment-1570837748, or unsubscribe https://github.com/notifications/unsubscribe-auth/AM624NDN2HDXBTLBZQQF663XI6NTPANCNFSM6AAAAAAYV3ZZBU . You are receiving this because you authored the thread.Message ID: @.***>

firasmidani commented 1 year ago

It is possible there is an issue in your mapping file. Are you able to share your mapping file and/or summary data (output of amiga fit? I might find a clue by glancing at the files.

ofleitas commented 1 year ago

Follow the files attached.

On Wed, May 31, 2023 at 4:14 PM Firas Midani @.***> wrote:

It is possible there is an issue in your mapping file. Are you able to share your mapping file and/or summary data (output of amiga fit? I might find a clue by glancing at the files.

— Reply to this email directly, view it on GitHub https://github.com/firasmidani/amiga/issues/17#issuecomment-1571035421, or unsubscribe https://github.com/notifications/unsubscribe-auth/AM624NGJ7Y7BXHH2C2IB3ILXI67FRANCNFSM6AAAAAAYV3ZZBU . You are receiving this because you authored the thread.Message ID: @.***>

Sample_ID Isolate Group Control mean(auc_log) mean(k_log) mean(death_log) mean(gr) mean(dr) mean(td) mean(lagC) mean(lagP) mean(t_k) mean(t_gr) mean(t_dr) std(auc_log) std(k_log) std(death_log) std(gr) std(dr) std(td) std(lagC) std(lagP) std(t_k) std(t_gr) std(t_dr) diauxie MSE 0 A. baumannii Ab6 1 0 85.98840943933985 4.362074184945453 0.3269830372848596 0.6439205277871125 -0.14520766148413145 1.0854854205370552 0.6148154768113446 0.0 16.26 2.195 23.465 1.1625358529200425 0.06844039755779131 0.19773602706217144 0.06027613777160853 0.08689925987597152 0.09824797377178598 0.21005187746326306 0.0 4.362072531226041 0.9946701397290896 2.68859826044365 0.0 1.1301529788787634 1 A. baumannii Ab7 2 0 95.8164998014529 4.852753233145662 0.20893239946225395 0.7757253590419698 -0.1218919723251473 0.8964821174045938 0.9935476733965054 0.0 20.025 3.36 23.17 0.946266968088969 0.07018792447256901 0.15760788729742123 0.04476746173662795 0.09269936199270262 0.05151612947610217 0.19493547066846464 0.0 2.980894550840791 0.38323450636481965 2.0141168453361056 0.0 0.5697259388665561 2 A. baumannii Ab8 3 0 94.13485176090357 4.913269023545606 0.24324735902834518 0.5810358954884709 -0.06016358070657599 1.203288890408893 0.12072212567413806 0.0 18.125 0.54 22.59 1.0420238971737736 0.06900575609941258 0.17418957537887753 0.054454592418774254 0.05182765105517279 0.11241835138471701 0.24794763503848044 0.0 2.771039807754774 0.7709367252527505 2.5270056540810457 0.0 1.0016332775783239 3 A. baumannii Abc 4 0 91.59160182465061 4.5824255955962965 0.3255313891182939 0.6633037516962338 -0.11279283177715037 1.054917384323647 0.26876832501408 0.0 15.185 1.66 22.455 1.1280584705718393 0.08213524055239274 0.20701514602392482 0.06570638403984684 0.08774114521753364 0.10209467053075111 0.22593583934333566 0.0 4.536415507486683 0.9869860253019866 3.6880197801787746 0.0 1.0862933703273445 4 A. baumannii Ab0 5 1 100.7119569608968 4.877339032782131 0.36243535536285615 0.8203634514032027 -0.1351630549361422 0.8581138918322685 -0.10753723770101024 0.0 15.975 0.34 22.96 1.3432469127096645 0.08327820121994098 0.23155871433505654 0.10395125994621535 0.10218395378121002 0.10633283372990514 0.27756398089004797 0.0 4.817044129070663 0.5216523862891959 3.4080119504698687 0.0 1.604857610598914

Well Plate_ID Isolate Comments Group Control Flag Subset OD_Baseline OD_Min OD_Max OD_Emp_AUC Fold_Change B2 Plate_1 A. baumannii Ab6 PolymyxinB_resistant 1 0 0 1 0.0 0.0 0.6247 10.7993 B3 Plate_1 A. baumannii Ab7 PolymyxinB_resistant 2 0 0 1 -0.002 -0.002 0.8193 14.1873 B4 Plate_1 A. baumannii Ab8 PolymyxinB_resistant 3 0 0 1 0.003 0.002333333 0.55 8.669833333 B5 Plate_1 A. baumannii Abc PolymyxinB_sensitive 4 0 0 1 0.001 0.001 0.762 12.9753 B6 Plate_1 A. baumannii Ab0 Parental 5 1 0 1 0.002 0.002 0.8123 12.3758 1.0 F2 Plate_10 A. baumannii Ab8 PolymyxinB_resistant 3 0 0 1 0.003 0.003 0.85 8.067599999999999
F3 Plate_10 A. baumannii Abc PolymyxinB_sensitive 4 0 0 1 0.004 0.004 1.0123 14.650599999999999
F4 Plate_10 A. baumannii Ab0 Parental 5 1 0 1 0.003 0.003 1.0743 14.761100000000003 1.0 G3 Plate_10 A. baumannii Ab7 PolymyxinB_resistant 2 0 0 1 0.004 0.004 0.9573 12.299100000000001
G4 Plate_10 A. baumannii Ab6 PolymyxinB_resistant 1 0 0 1 0.004 0.004 0.773 10.476100000000002
C3 Plate_2 A. baumannii Ab6 PolymyxinB_resistant 1 0 0 1 0.001 0.001 0.5957 10.737649999999999
C4 Plate_2 A. baumannii Ab7 PolymyxinB_resistant 2 0 0 1 0.006 0.002 0.6147 11.239650000000001
C5 Plate_2 A. baumannii Ab8 PolymyxinB_resistant 3 0 0 1 0.007 0.001 0.9407 15.27215
C6 Plate_2 A. baumannii Abc PolymyxinB_sensitive 4 0 0 1 0.0 0.0 0.6587 11.47215
C7 Plate_2 A. baumannii Ab0 Parental 5 1 0 1 0.001 0.001 0.5327 9.27065 1.0 D4 Plate_3 A. baumannii Ab6 PolymyxinB_resistant 1 0 0 1 0.001 0.0 0.5257 8.61925 D5 Plate_3 A. baumannii Ab7 PolymyxinB_resistant 2 0 0 1 0.0 0.0 1.0577 17.04525
D6 Plate_3 A. baumannii Ab8 PolymyxinB_resistant 3 0 0 1 0.0 0.0 0.514 8.42325 D7 Plate_3 A. baumannii Abc PolymyxinB_sensitive 4 0 0 1 0.0 0.0 0.6527 11.003400000000001
D8 Plate_3 A. baumannii Ab0 Parental 5 1 0 1 0.0 0.0 0.4883 8.21775 1.0 D5 Plate_4 A. baumannii Ab6 PolymyxinB_resistant 1 0 0 1 0.0 0.0 0.5023 8.205100000000002
D6 Plate_4 A. baumannii Ab7 PolymyxinB_resistant 2 0 0 1 0.0027 0.0003 0.84 11.7302 D7 Plate_4 A. baumannii Ab8 PolymyxinB_resistant 3 0 0 1 0.0047 0.0047 0.8763 13.4162 D8 Plate_4 A. baumannii Abc PolymyxinB_sensitive 4 0 0 1 0.0 0.0 0.6873 11.4302 D9 Plate_4 A. baumannii Ab0 Parental 5 1 0 1 0.0 0.0 0.534 9.12455 1.0 E5 Plate_5 A. baumannii Ab6 PolymyxinB_resistant 1 0 0 1 0.0 0.0 0.477 7.60745 E6 Plate_5 A. baumannii Ab7 PolymyxinB_resistant 2 0 0 1 -0.0043 -0.0043 0.5223 8.694799999999999
E7 Plate_5 A. baumannii Ab8 PolymyxinB_resistant 3 0 0 1 0.0 0.0 0.849 13.422799999999999
E8 Plate_5 A. baumannii Abc PolymyxinB_sensitive 4 0 0 1 0.0 -0.002 0.613 10.0918 E9 Plate_5 A. baumannii Ab0 Parental 5 1 0 1 0.0 0.0 0.478 8.0993 1.0 F6 Plate_6 A. baumannii Ab6 PolymyxinB_resistant 1 0 0 1 0.0 0.0 0.5637 9.6152
F7 Plate_6 A. baumannii Ab7 PolymyxinB_resistant 2 0 0 1 -0.0003 -0.004 0.9277 14.21755
F8 Plate_6 A. baumannii Ab8 PolymyxinB_resistant 3 0 0 1 0.0 0.0 0.898 13.142699999999998
F9 Plate_6 A. baumannii Abc PolymyxinB_sensitive 4 0 0 1 0.0 0.0 0.631 11.321199999999997
F10 Plate_6 A. baumannii Ab0 Parental 5 1 0 1 0.0 0.0 0.4803 8.284199999999998 1.0 B7 Plate_7 A. baumannii Ab7 PolymyxinB_resistant 2 0 0 1 0.0013 0.0013 0.4813 8.027899999999999
B8 Plate_7 A. baumannii Ab8 PolymyxinB_resistant 3 0 0 1 0.0443 0.0063 1.0673 18.1379 B9 Plate_7 A. baumannii Abc PolymyxinB_sensitive 4 0 0 1 0.0693 0.0113 0.9223 16.6849 C8 Plate_7 A. baumannii Ab6 PolymyxinB_resistant 1 0 0 1 0.0643 0.0013 0.6443 11.156399999999998
C9 Plate_7 A. baumannii Ab0 Parental 5 1 0 1 0.0663 0.0033 0.941 15.8029 1.0 D9 Plate_8 A. baumannii Ab8 PolymyxinB_resistant 3 0 0 1 0.001 0.001 1.1563 14.475399999999999
D10 Plate_8 A. baumannii Ab0 Parental 5 1 0 1 0.001 0.001 0.709 12.213899999999999 1.0 D11 Plate_8 A. baumannii Abc PolymyxinB_sensitive 4 0 0 1 0.0 0.0 0.6323 10.908899999999997
E10 Plate_8 A. baumannii Ab6 PolymyxinB_resistant 1 0 0 1 0.0 0.0 0.73 12.3199 E11 Plate_8 A. baumannii Ab7 PolymyxinB_resistant 2 0 0 1 0.01 0.0013 0.56 9.303400000000002
C2 Plate_9 A. baumannii Abc PolymyxinB_sensitive 4 0 0 1 0.0017 0.0017 0.4903 7.1297500000000005
D2 Plate_9 A. baumannii Ab6 PolymyxinB_resistant 1 0 0 1 0.0037 0.0037 0.6623 10.61825
D3 Plate_9 A. baumannii Ab0 Parental 5 1 0 1 0.0037 0.0037 1.129 16.40075 1.0 E2 Plate_9 A. baumannii Ab7 PolymyxinB_resistant 2 0 0 1 0.0027 0.0027 0.533 8.668750000000001
E3 Plate_9 A. baumannii Ab8 PolymyxinB_resistant 3 0 0 1 0.0047 0.0047 0.9253 11.747750000000002

Sample_ID Isolate Group Control mean(auc_log) mean(k_log) mean(death_log) mean(gr) mean(dr) mean(td) mean(lagC) mean(lagP) mean(t_k) mean(t_gr) mean(t_dr) std(auc_log) std(k_log) std(death_log) std(gr) std(dr) std(td) std(lagC) std(lagP) std(t_k) std(t_gr) std(t_dr) diauxie MSE norm(mean(auc_log)) norm(mean(k_log)) norm(mean(death_log)) norm(mean(gr)) norm(mean(dr)) norm(mean(td)) norm(mean(lagC)) norm(mean(lagP)) norm(mean(t_k)) norm(mean(t_gr)) norm(mean(t_dr)) 0 A. baumannii Ab6 1 0 85.98840943933985 4.362074184945453 0.3269830372848596 0.6439205277871125 -0.14520766148413145 1.0854854205370552 0.6148154768113446 0.0 16.26 2.195 23.465 1.1625358529200425 0.06844039755779131 0.19773602706217144 0.06027613777160853 0.08689925987597152 0.09824797377178597 0.21005187746326304 0.0 4.36207253122604 0.9946701397290896 2.68859826044365 0.0 1.1301529788787634
1 A. baumannii Ab7 2 0 95.8164998014529 4.852753233145662 0.20893239946225395 0.7757253590419698 -0.1218919723251473 0.8964821174045938 0.9935476733965054 0.0 20.025 3.36 23.17 0.9462669680889692 0.07018792447256901 0.15760788729742126 0.04476746173662795 0.09269936199270262 0.051516129476102165 0.19493547066846464 0.0 2.980894550840791 0.3832345063648197 2.0141168453361056 0.0 0.5697259388665561
2 A. baumannii Ab8 3 0 94.13485176090357 4.913269023545607 0.2432473590283452 0.5810358954884709 -0.060163580706575986 1.203288890408893 0.12072212567413805 0.0 18.125 0.54 22.59 1.0420238971737736 0.06900575609941258 0.17418957537887753 0.054454592418774254 0.051827651055172785 0.112418351384717 0.24794763503848044 0.0 2.771039807754774 0.7709367252527505 2.5270056540810457 0.0 1.001633277578324
3 A. baumannii Abc 4 0 91.5916018246506 4.5824255955962965 0.3255313891182939 0.6633037516962338 -0.11279283177715035 1.054917384323647 0.26876832501408 0.0 15.185 1.66 22.455 1.1280584705718393 0.08213524055239274 0.2070151460239248 0.06570638403984684 0.08774114521753364 0.10209467053075112 0.2259358393433357 0.0 4.536415507486684 0.9869860253019866 3.688019780178775 0.0 1.0862933703273443
4 A. baumannii Ab0 5 1 100.7119569608968 4.877339032782131 0.36243535536285615 0.8203634514032027 -0.1351630549361422 0.8581138918322685 -0.10753723770101024 0.0 15.975 0.34 22.96 1.3432469127096645 0.08327820121994098 0.2315587143350565 0.10395125994621536 0.10218395378121002 0.10633283372990514 0.277563980890048 0.0 4.817044129070663 0.5216523862891959 3.4080119504698687 0.0 1.604857610598914 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0

Plate_ID    Isolate Comments    Group   Control

E5 Plate_5 A. baumannii Ab6 PolymyxinB_resistant 1 0 E6 Plate_5 A. baumannii Ab7 PolymyxinB_resistant 2 0 E7 Plate_5 A. baumannii Ab8 PolymyxinB_resistant 3 0 E8 Plate_5 A. baumannii Abc PolymyxinB_sensitive 4 0 E9 Plate_5 A. baumannii Ab0 Parental 5 1

Plate_ID    Isolate Comments    Group   Control

D4 Plate_3 A. baumannii Ab6 PolymyxinB_resistant 1 0 D5 Plate_3 A. baumannii Ab7 PolymyxinB_resistant 2 0 D6 Plate_3 A. baumannii Ab8 PolymyxinB_resistant 3 0 D7 Plate_3 A. baumannii Abc PolymyxinB_sensitive 4 0 D8 Plate_3 A. baumannii Ab0 Parental 5 1

Plate_ID    Isolate Comments    Group   Control

D5 Plate_4 A. baumannii Ab6 PolymyxinB_resistant 1 0 D6 Plate_4 A. baumannii Ab7 PolymyxinB_resistant 2 0 D7 Plate_4 A. baumannii Ab8 PolymyxinB_resistant 3 0 D8 Plate_4 A. baumannii Abc PolymyxinB_sensitive 4 0 D9 Plate_4 A. baumannii Ab0 Parental 5 1

Plate_ID    Isolate Comments    Group   Control

B2 Plate_1 A. baumannii Ab6 PolymyxinB_resistant 1 0 B3 Plate_1 A. baumannii Ab7 PolymyxinB_resistant 2 0 B4 Plate_1 A. baumannii Ab8 PolymyxinB_resistant 3 0 B5 Plate_1 A. baumannii Abc PolymyxinB_sensitive 4 0 B6 Plate_1 A. baumannii Ab0 Parental 5 1

Plate_ID    Isolate Comments    Group   Control

C3 Plate_2 A. baumannii Ab6 PolymyxinB_resistant 1 0 C4 Plate_2 A. baumannii Ab7 PolymyxinB_resistant 2 0 C5 Plate_2 A. baumannii Ab8 PolymyxinB_resistant 3 0 C6 Plate_2 A. baumannii Abc PolymyxinB_sensitive 4 0 C7 Plate_2 A. baumannii Ab0 Parental 5 1

Plate_ID    Isolate Comments    Group   Control

F6 Plate_6 A. baumannii Ab6 PolymyxinB_resistant 1 0 F7 Plate_6 A. baumannii Ab7 PolymyxinB_resistant 2 0 F8 Plate_6 A. baumannii Ab8 PolymyxinB_resistant 3 0 F9 Plate_6 A. baumannii Abc PolymyxinB_sensitive 4 0 F10 Plate_6 A. baumannii Ab0 Parental 5 1

Plate_ID    Isolate Comments    Group   Control

B7 Plate_7 A. baumannii Ab7 PolymyxinB_resistant 2 0 B8 Plate_7 A. baumannii Ab8 PolymyxinB_resistant 3 0 B9 Plate_7 A. baumannii Abc PolymyxinB_sensitive 4 0 C8 Plate_7 A. baumannii Ab6 PolymyxinB_resistant 1 0 C9 Plate_7 A. baumannii Ab0 Parental 5 1

Plate_ID    Isolate Comments    Group   Control

D9 Plate_8 A. baumannii Ab8 PolymyxinB_resistant 3 0 D10 Plate_8 A. baumannii Ab0 Parental 5 1 D11 Plate_8 A. baumannii Abc PolymyxinB_sensitive 4 0 E10 Plate_8 A. baumannii Ab6 PolymyxinB_resistant 1 0 E11 Plate_8 A. baumannii Ab7 PolymyxinB_resistant 2 0

Plate_ID    Isolate Comments    Group   Control

F2 Plate_10 A. baumannii Ab8 PolymyxinB_resistant 3 0 F3 Plate_10 A. baumannii Abc PolymyxinB_sensitive 4 0 F4 Plate_10 A. baumannii Ab0 Parental 5 1 G3 Plate_10 A. baumannii Ab7 PolymyxinB_resistant 2 0 G4 Plate_10 A. baumannii Ab6 PolymyxinB_resistant 1 0

Plate_ID    Isolate Comments    Group   Control

C2 Plate_9 A. baumannii Abc PolymyxinB_sensitive 4 0 D2 Plate_9 A. baumannii Ab6 PolymyxinB_resistant 1 0 D3 Plate_9 A. baumannii Ab0 Parental 5 1 E2 Plate_9 A. baumannii Ab7 PolymyxinB_resistant 2 0 E3 Plate_9 A. baumannii Ab8 PolymyxinB_resistant 3 0

firasmidani commented 1 year ago

It is a bit hard to look at these files (I think attaching the files using a browser instead of email would have worked better). Still, I think I see the issue. I think the issue here may be simply a misunderstanding of how the Group and Control columns can be used.

It seems that you have five groups (1,2,3,4,5) and your parental strains are in group 5 and you are considering parental strain to be the control. However, each group needs a control. Here, groups 1, 2, 3, and 4 do not have any controls, so, AMiGA cannot normalize them to anything. The key idea here is that AMiGA can normalize summary stats within each group. In practice, it should look inside group 1, find the treatment samples (where Control=0) and normalize to the control samples (where Control=1). Then, it does the same for group 2, 3, ..., etc.

If this is still not a clear, please re-visit the Meta-Data Documentation and specifically look at the third table for an example. In this example, I want to look at how the growth by wild-type strains (Group 1) on trehalose normalized to background growth on minimal media (i.e. negative control) and I want to compare this to the growth of the knock-out strain (Group 2) on trehalose normalized to background growth.

I don't know your experimental design but I am guessing that you are simply trying to compare all of the strains to the parental strain under the same condition. If so, then, they should all be in the same group. What do you think? Do you think this would work in your case?

ofleitas commented 1 year ago

Hello

You are right, I am trying to compare the strains against the parental under the same condition. Then I followed your suggestion and included the parental and the strains in the same group. That fixes the problems of the fold changes and the normalization. The first option of normalization that you suggested worked when I changed --group-by "Isolate" by --group-by "Group". The second option did not work. Now that all the strains are in the same group, how can I use the compare and test commands to compare a strain against the parental ? I tried to compare a strain against the parent strain but it did not work, it did not generate the output file. Thank you very much in advance for your help.

PS C:\Users\ofm83\Documents\Projects_python\amiga-master> python amiga.py compare -i C:\Users\ofm83\Documents\A.baumannii_amiga\summary\Fit_pool_summary.txt -o Ab6_vs_Ab0 -s "Isolate:A. baumannii Ab6,A. baumannii Ab0" --confidence 95 --verbose

-----------------------------------------------------

User provided the following command-line arguments:

-----------------------------------------------------

input..........['C:\Users\ofm83\Documents\A.baumannii_amiga\summary\Fit_pool_summary.txt'] output.........Ab6_vs_Ab0 subset.........Isolate:A. baumannii Ab6,A. baumannii Ab0 confidence.....95.0 verbose........True

PS C:\Users\ofm83\Documents\Projects_python\amiga-master>

On Wed, May 31, 2023 at 7:21 PM Firas Midani @.***> wrote:

It is a bit hard to look at these files (I think attaching the files using a browser instead of email would have worked better). Still, I think I see the issue. I think the issue here may be simply a misunderstanding of how the Group and Control columns can be used.

It seems that you have five groups (1,2,3,4,5) and your parental strains are in group 5 and you are considering parental strain to be the control. However, each group needs a control. Here, groups 1, 2, 3, and 4 do not have any controls, so, AMiGA cannot normalize them to anything. The key idea here is that AMiGA can normalize summary stats within each group. In practice, it should look inside group 1, find the treatment samples (where Control=0) and normalize to the control samples (where Control=1). Then, it does the same for group 2, 3, ..., etc.

If this is still not a clear, please re-visit the [Meta-Data Documentation] (https://firasmidani.github.io/amiga/doc/metadata.html) and specifically look at the third table for an example. In this example, I want to look at how the growth by wild-type strains (Group 1) on trehalose normalized to background growth on minimal media (i.e. negative control) and I want to compare this to the growth of the knock-out strain (Group 2) on trehalose normalized to background growth.

I don't know your experimental design but I am guessing that you are simply trying to compare all of the strains to the parental strain under the same condition. If so, then, they should all be in the same group. What do you think? Do you think this would work in your case?

— Reply to this email directly, view it on GitHub https://github.com/firasmidani/amiga/issues/17#issuecomment-1571183153, or unsubscribe https://github.com/notifications/unsubscribe-auth/AM624NDEUVV3ON4CCMG4BB3XI7VBZANCNFSM6AAAAAAYV3ZZBU . You are receiving this because you authored the thread.Message ID: @.***>

firasmidani commented 1 year ago

First, did you look at the directory where you ran the command? AMiGA compare saves output in the folder where it is run, so, in your case, that would be amiga-master.

Second, when you ran the command that generated Fit_pool_summary.txt file, did you use the --sample-posterior argument as shown at the end of the first command in the Compare Parameters documentation?

ofleitas commented 1 year ago

Hello

You are right, the output file was in amiga-master. I hadn't looked in that folder. The command worked well. Yes, I used --sample-posterior. I finished all the analyses. Now I ran all the commands again and I don't know why the normalize command is not working (I ran the same script that worked in the first analysis), but I get the output with all the columns of normalization in blank. My input contains the Group and Control columns. This was the script

C:\Users\ofm83\Documents\Projects_python\amiga-master> python amiga.py normalize --input C:\Users\ofm83\Documents\A.baumannii_amiga\summary\Fit_pool_summary.txt --group-by "Group" --normalize-method "division" --normalize-by "Control:1" --verbose

Thank you very much for your help.

On Fri, Jun 2, 2023 at 1:51 PM Firas Midani @.***> wrote:

First, did you look at the directory where you ran the command? AMiGA compare saves output in the folder where it is run, so, in your case, that would be amiga-master.

Second, when you ran the command that generated Fit_pool_summary.txt file, did you use the --sample-posterior argument as shown at the end of the first command in the Compare Parameters https://firasmidani.github.io/amiga/doc/comparing.html documentation?

— Reply to this email directly, view it on GitHub https://github.com/firasmidani/amiga/issues/17#issuecomment-1574237283, or unsubscribe https://github.com/notifications/unsubscribe-auth/AM624NH7K4IOZFNYVTEP7W3XJI75PANCNFSM6AAAAAAYV3ZZBU . You are receiving this because you authored the thread.Message ID: @.***>

firasmidani commented 1 year ago

Your command looks fine. I would suggest that you look at your mapping file and make sure that you only have one Group, and that you look at the Fit_pool_summary.txt and make sure that there is nothing wrong with it (check the Group and Control columns there too).

ofleitas commented 1 year ago

Hello I checked the mapping and the Fit_pool_summary.txt and they look fine. I have only one group (Group 1) and one control. I ran the analysis again but the normalization is not working . I also performed the Test Hypotheses step, I got the Log_BF value but I didn't get the M1_FDR_cutoff and M0_FDR_cutoff values. I need these values to consider Bayes factor scores as significant. Best regards

On Mon, Jun 5, 2023 at 8:45 AM Firas Midani @.***> wrote:

Your command looks fine. I would suggest that you look at your mapping file and make sure that you only have one Group, and that you look at the Fit_pool_summary.txt and make sure that there is nothing wrong with it (check the Group and Control columns there too).

— Reply to this email directly, view it on GitHub https://github.com/firasmidani/amiga/issues/17#issuecomment-1576939525, or unsubscribe https://github.com/notifications/unsubscribe-auth/AM624NCBTGZG66MXYCYK6ATXJXWHDANCNFSM6AAAAAAYV3ZZBU . You are receiving this because you authored the thread.Message ID: @.***>

firasmidani commented 1 year ago

If you share both your mapping file and summary file, I can take a second look.

ofleitas commented 1 year ago

Fallows the files. Plate_1.txt Plate_2.txt Plate_3.txt Plate_4.txt Plate_5.txt Plate_6.txt Plate_7.txt Plate_8.txt Plate_9.txt Plate_10.txt Fit_pool_summary.txt Fit_pool_summary_normalized.txt

firasmidani commented 1 year ago

Ok, I see that there is a minor bug with AMiGA, which I will have to fix. Here, there is an issue because you are using the Control column to tell AMiGA what to normalize by. There is a simple workaround that you can implement immediately while I continue to fix the problem. For your case, instead of the following:

--normalize-by "Control:1"

please do the following

--normalize-by "Isolate:A. baumannii Ab0"

Let me know if this workaround makes normalization work and hypothesis testing work.

ofleitas commented 1 year ago

Hello

It did not work. Fit_pool_summary_normalized.txt

firasmidani commented 1 year ago

Ok, hmm, it worked for me. I downloaded the "Fit_pool_summary.txt" that you shared two hours ago, and ran the following command:

python amiga.py normalize -i Fit_pool_summary.txt --group-by "Group" --normalize-method "division" --normalize-by "Isolate:A. baumannii Ab0" --verbose

I got the following: Fit_pool_summary_normalized.txt

So, I just want to double check that we ran the same command. In your case, it would:

C:\Users\ofm83\Documents\Projects_python\amiga-master> python amiga.py
normalize --input
C:\Users\ofm83\Documents\A.baumannii_amiga\summary\Fit_pool_summary.txt
--group-by "Group" --normalize-method "division" --normalize-by "Isolate:A. baumannii Ab0"
--verbose

Can you please confirm that you used the above command?

ofleitas commented 1 year ago

It worked. I saw my error, I had --group-by "Isolate". The hypothesis testing worked too. Thank you very much for your help!!!

firasmidani commented 1 year ago

Glad to hear it. Thanks for pointing out this issue. I will keep this issue open until I address the related bug in AMiGA.

At the moment, for the --normalize-by option , you will have to use a value that is a string/text such as "A. baumanii Ab0" (not a number such as "1"). I will close this issue once I solve this problem.

ofleitas commented 1 year ago

Ok. Thank you very much for your help!!!

On Wed, Jun 7, 2023 at 8:58 AM Firas Midani @.***> wrote:

Glad to hear it. Thanks for pointing out this issue. I will keep this issue open until I address the related bug in AMiGA.

At the moment, for the --normalize-by option , you will have to use a value that is a string/text such as "A. baumanii Ab0" (not a number such as "1"). I will close this issue once I solve this problem.

— Reply to this email directly, view it on GitHub https://github.com/firasmidani/amiga/issues/17#issuecomment-1581001017, or unsubscribe https://github.com/notifications/unsubscribe-auth/AM624NF72H2ARTCFIPCLXG3XKCJHRANCNFSM6AAAAAAYV3ZZBU . You are receiving this because you authored the thread.Message ID: @.***>

ofleitas commented 1 year ago

One last question, does AMiGA perform the comparison of parameters with the normalized parameters?

On Wed, Jun 7, 2023 at 11:11 AM osmel fleitas @.***> wrote:

Ok. Thank you very much for your help!!!

On Wed, Jun 7, 2023 at 8:58 AM Firas Midani @.***> wrote:

Glad to hear it. Thanks for pointing out this issue. I will keep this issue open until I address the related bug in AMiGA.

At the moment, for the --normalize-by option , you will have to use a value that is a string/text such as "A. baumanii Ab0" (not a number such as "1"). I will close this issue once I solve this problem.

— Reply to this email directly, view it on GitHub https://github.com/firasmidani/amiga/issues/17#issuecomment-1581001017, or unsubscribe https://github.com/notifications/unsubscribe-auth/AM624NF72H2ARTCFIPCLXG3XKCJHRANCNFSM6AAAAAAYV3ZZBU . You are receiving this because you authored the thread.Message ID: @.***>