EddyRivasLab / hmmer

HMMER: biological sequence analysis using profile HMMs
http://hmmer.org
Other
305 stars 69 forks source link

Hmmbuild returns different results for a3m and sto #321

Closed Augustin-Zidek closed 4 months ago

Augustin-Zidek commented 6 months ago

Hmmbuild gives different results depending on the input format. For Stockholm, it gives the output of the same length as the aligned sequences. For A3M, it sometimes does not.

In order to get the correctly aligned result, we therefore have to convert a3m -> sto - adding the #=GC RF line while doing so - then run Hmmbuild with the converted Stockholm file.

However, in our convertor, the #=GC RF line is generated trivially:

I realise we are seeing this issue most likely because we are not specifying the reference annotation when using a3m. Is there a way to specify the reference annotation in the a3m case? If not, would it be possible to add that option so that the conversion is not needed?

Reproduction

Input a3m created by removing all columns where the query had a gap and also removing deletions (lower-case letters) in non-query sequences so that everything has exactly the same length (input.a3m):

>query
FHQECSLQSCTQHQPYVVDDPCPIHFYSKWYIRVGARKSAPLIELCVDEAGSKSPIQYIDIGNYTVSCLPFTINCQEPKLGSLVVRCSFYEDFLEYHDVRVVLDFI
>UniRef90_A0A2R3SUZ9/16-121 [subseq from] Uncharacterized protein n=1 Tax=Bat SARS-like coronavirus TaxID=1508227 RepID=A0A2R3SUZ9_CVHSA
FHQECSLQSCAQHQPYVVDDPCPIHFYSRWYIRVGARKSAPLIELCVDEVGSKSPIQYIDIGNYTVSCSPFTINCQEPKLGSLVVRCSYYEDFLEYHDIRVVLDFI
>UniRef90_A0A023PSW4/17-121 [subseq from] Protein 10b n=1 Tax=Rhinolophus affinis coronavirus TaxID=1487703 RepID=A0A023PSW4_CVHSA
-HKECTIQECCENQPYILEDPCPIHYYSDWYLKIGPRKSARLLQLCAGEYGKRLPVQYEKLGNYTINCEPFEINCQTPPVGSLIVRCSYDYDFIEYHDVRVVLDFI
>UniRef90_U5WLD0/17-121 [subseq from] Uncharacterized protein n=11 Tax=Betacoronavirus TaxID=694002 RepID=U5WLD0_CVHSA
-HKECSIQECCENQPFQLEDPCPIHYYSDWFVKIGPRKSARLVQLCAGEYGHRVPIHYEMFGNYTISCEPLEINCQNPPVGSLIVRCSYDVDFMEYHDVRVVLDFI
>UniRef90_Q0Q469/17-121 [subseq from] Non-structural protein 8 n=17 Tax=Betacoronavirus TaxID=694002 RepID=NS8_BC279
-HKECSIQECCENQPYQIEDPCPIHYYSDWFIKIGSRKSARLVQLCEGDYGKRIPIHYEMFGNYTISCEPLEINCQAPPVGSLIVRCSYDYDFVEHHDVRVVLDFI
>UniRef90_A0A0U1UYY4/17-121 [subseq from] Uncharacterized protein n=1 Tax=BtRs-BetaCoV/HuB2013 TaxID=1503302 RepID=A0A0U1UYY4_CVHSA
-HKECSIQECCEIQPSQIEDPSPIHYYSDWFIQIGYRKSARLVQLCEGDYGKRIPIHYEMFGNYSMYCEPLEINCQAPPVGRLIVRWLHDYESAEHHDVRVVLDFI
>UniRef90_D5HJV2/17-51 [subseq from] Uncharacterized protein n=1 Tax=Bat SARS coronavirus HKU3-8 TaxID=742005 RepID=D5HJV2_BCHK3
-HKECSIQECCENQPYQIEDPCPIHYYSDWFIKIGS----------------------------------------------------------------------
>UniRef90_Q3ZTE5/20-117 [subseq from] Orf10 n=10 Tax=Human SARS coronavirus TaxID=694009 RepID=Q3ZTE5_CVHSA
------VQRCASNKPHVLEDPCPTGYQPEWNIRYKTRGNTSTAWLCA--LGKVLPFHRW--HTMVQTCTPVTINCQDPAGGALIARCWYLHETAAFRDVLVVL---
>UniRef90_A0A0K1Z007/20-103 [subseq from] Uncharacterized protein n=6 Tax=Human SARS coronavirus TaxID=694009 RepID=A0A0K1Z007_CVHSA
------VQRCVSNTPYVLENPCPTGYRPEWNIRYNTRGNTNTARLCA--LGK--VLSFHRWHTMVQACTPITINCQDPVGGALVARCWYFHK--------------

Converted input Stockholm (input.sto):

# STOCKHOLM 1.0

#=GS query_0 DE <EMPTY>
#=GS UniRef90_A0A2R3SUZ9/16-121_1 DE [subseq from] Uncharacterized protein n=1 Tax=Bat SARS-like coronavirus TaxID=1508227 RepID=A0A2R3SUZ9_CVHSA
#=GS UniRef90_A0A023PSW4/17-121_2 DE [subseq from] Protein 10b n=1 Tax=Rhinolophus affinis coronavirus TaxID=1487703 RepID=A0A023PSW4_CVHSA
#=GS UniRef90_U5WLD0/17-121_3 DE [subseq from] Uncharacterized protein n=11 Tax=Betacoronavirus TaxID=694002 RepID=U5WLD0_CVHSA
#=GS UniRef90_Q0Q469/17-121_4 DE [subseq from] Non-structural protein 8 n=17 Tax=Betacoronavirus TaxID=694002 RepID=NS8_BC279
#=GS UniRef90_A0A0U1UYY4/17-121_5 DE [subseq from] Uncharacterized protein n=1 Tax=BtRs-BetaCoV/HuB2013 TaxID=1503302 RepID=A0A0U1UYY4_CVHSA
#=GS UniRef90_D5HJV2/17-51_6 DE [subseq from] Uncharacterized protein n=1 Tax=Bat SARS coronavirus HKU3-8 TaxID=742005 RepID=D5HJV2_BCHK3
#=GS UniRef90_Q3ZTE5/20-117_7 DE [subseq from] Orf10 n=10 Tax=Human SARS coronavirus TaxID=694009 RepID=Q3ZTE5_CVHSA
#=GS UniRef90_A0A0K1Z007/20-103_8 DE [subseq from] Uncharacterized protein n=6 Tax=Human SARS coronavirus TaxID=694009 RepID=A0A0K1Z007_CVHSA

query_0                      FHQECSLQSCTQHQPYVVDDPCPIHFYSKWYIRVGARKSA-PLIELCVDEAGSKSPIQYIDIGNYTVSCLP-FTINCQEPKLGSLVVRCSFYED---FLEYHDVRVVLDFI
UniRef90_A0A2R3SUZ9/16-121_1 FHQECSLQSCAQHQPYVVDDPCPIHFYSRWYIRVGARKSA-PLIELCVDEVGSKSPIQYIDIGNYTVSCSP-FTINCQEPKLGSLVVRCSYYED---FLEYHDIRVVLDFI
UniRef90_A0A023PSW4/17-121_2 -HKECTIQECCENQPYILEDPCPIHYYSDWYLKIGPRKSA-RLLQLCAGEYGKRLPVQYEKLGNYTINCEP-FEINCQTPPVGSLIVRCSYDYD---FIEYHDVRVVLDFI
UniRef90_U5WLD0/17-121_3     -HKECSIQECCENQPFQLEDPCPIHYYSDWFVKIGPRKSA-RLVQLCAGEYGHRVPIHYEMFGNYTISCEP-LEINCQNPPVGSLIVRCSYDVD---FMEYHDVRVVLDFI
UniRef90_Q0Q469/17-121_4     -HKECSIQECCENQPYQIEDPCPIHYYSDWFIKIGSRKSA-RLVQLCEGDYGKRIPIHYEMFGNYTISCEP-LEINCQAPPVGSLIVRCSYDYD---FVEHHDVRVVLDFI
UniRef90_A0A0U1UYY4/17-121_5 -HKECSIQECCEIQPSQIEDPSPIHYYSDWFIQIGYRKSA-RLVQLCEGDYGKRIPIHYEMFGNYSMYCEP-LEINCQAPPVGRLIVRWLHDYE---SAEHHDVRVVLDFI
UniRef90_D5HJV2/17-51_6      -HKECSIQECCENQPYQIEDPCPIHYYSDWFIKIGS---------------------------------------------------------------------------
UniRef90_Q3ZTE5/20-117_7     ------VQRCASNKPHVLEDPCPTGYQPEWNIRYKTRGNTYSTAWLCA--LGKVLPFHRW--HTMVQTCTPNVTINCQDPAGGALIARCWYLHEGHQTAAFRDVLVVL---
UniRef90_A0A0K1Z007/20-103_8 ------VQRCVSNTPYVLENPCPTGYRPEWNIRYNTRGNTYNTARLCA--LGK--VLSFHRWHTMVQACTPNITINCQDPVGGALVARCWYFHK-----------------
#=GC RF                      xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.xxxxxxxxxxxxxxxxxxxxxx...xxxxxxxxxxxxxx
//

We then run the following pairs of commands:

  1. hmmbuild --amino output_a3m.hmm input.a3m hmmbuild --amino output_sto.hmm input.sto
  2. hmmbuild --amino --fast output_fast_a3m.hmm input.a3m hmmbuild --amino --fast output_fast_sto.hmm input.sto
  3. hmmbuild --amino --hand output_hand_a3m.hmm input.a3m - fails with "Error: build failed: Alignment input has no reference annotation line" hmmbuild --amino --hand output_hand_sto.hmm input.sto - works

The respective pairs for cases 1 and 2 don't match. E.g. running diff --side-by-side --color --suppress-common-lines output_{a3m,sto}.hmm will produce:

RF    no                                                      | RF    yes
DATE  Fri Feb 16 11:54:10 2024                                | DATE  Fri Feb 16 11:54:02 2024
CKSUM 1322260715                                              | CKSUM 3959945637
STATS LOCAL VITERBI  -10.4812  0.71643                        | STATS LOCAL VITERBI  -10.5324  0.71643
STATS LOCAL FORWARD   -3.7799  0.71643                        | STATS LOCAL FORWARD   -3.7782  0.71643
  COMPO   2.65942  3.36555  2.98721  2.74018  3.31101  3.0457 |   COMPO   2.65955  3.36878  2.98604  2.74013  3.31171  3.0413
      1   2.94412  4.76874  3.08059  2.83649  3.25246  3.4078 |       1   2.94412  4.76874  3.08059  2.83649  3.25246  3.4078
      2   2.86089  5.02345  2.94392  2.52467  4.36519  3.4511 |       2   2.86089  5.02345  2.94392  2.52467  4.36519  3.4511
      3   2.91195  5.01036  2.46992  1.09721  4.35073  3.2437 |       3   2.91195  5.01036  2.46992  1.09721  4.35073  3.2437
      4   2.53348  1.04554  4.03974  3.76892  3.77244  3.1659 |       4   2.53348  1.04554  4.03974  3.76892  3.77244  3.1659
      5   2.20661  4.14685  3.20731  2.95360  4.10684  2.9126 |       5   2.20661  4.14685  3.20731  2.95360  4.10684  2.9126
      6   3.11141  4.42526  4.76966  4.23723  3.39752  4.4551 |       6   3.11141  4.42526  4.76966  4.23723  3.39752  4.4551
      7   2.96016  4.94743  2.96944  2.72630  4.12871  3.4315 |       7   2.96016  4.94743  2.96944  2.72630  4.12871  3.4315
      8   2.70746  4.98837  2.77625  1.84866  4.33244  3.3724 |       8   2.70746  4.98837  2.77625  1.84866  4.33244  3.3724
      9   2.70037  0.76303  4.24129  4.00351  3.97320  3.3022 |       9   2.70037  0.76303  4.24129  4.00351  3.97320  3.3022
     10   1.92268  2.23051  3.96584  3.48706  3.63377  3.2425 |      10   1.92268  2.23051  3.96584  3.48706  3.63377  3.2425
     11   2.73575  5.08166  2.53040  1.74689  4.40504  3.3069 |      11   2.73575  5.08166  2.53040  1.74689  4.40504  3.3069
     12   2.66473  4.76249  2.91590  2.49547  3.78339  3.4063 |      12   2.66473  4.76249  2.91590  2.49547  3.78339  3.4063
     13   2.75517  4.94840  2.91613  2.48751  4.29659  3.4263 |      13   2.75517  4.94840  2.91613  2.48751  4.29659  3.4263
     14   2.88413  4.62467  3.50191  3.35541  4.34124  3.3067 |      14   2.88413  4.62467  3.50191  3.35541  4.34124  3.3067
     15   2.92129  4.55839  3.68622  3.19552  2.25164  3.7956 |      15   2.92129  4.55839  3.68622  3.19552  2.25164  3.7956
     16   2.69154  4.41594  3.51451  2.98821  3.50907  3.6368 |      16   2.69154  4.41594  3.51451  2.98821  3.50907  3.6368
     17   3.14046  4.47065  4.74032  4.19758  3.27852  4.4413 |      17   3.14046  4.47065  4.74032  4.19758  3.27852  4.4413
     18   2.92476  5.44429  1.83424  1.25844  4.73571  3.2239 |      18   2.92476  5.44429  1.83424  1.25844  4.73571  3.2239
     19   2.91218  5.36765  1.19719  2.13137  4.68681  3.1964 |      19   2.91218  5.36765  1.19719  2.13137  4.68681  3.1964
     20   2.88413  4.62467  3.50191  3.35541  4.34124  3.3067 |      20   2.88413  4.62467  3.50191  3.35541  4.34124  3.3067
     21   2.24254  1.35910  3.95416  3.62905  3.87783  3.0102 |      21   2.24254  1.35910  3.95416  3.62905  3.87783  3.0102
     22   2.88413  4.62467  3.50191  3.35541  4.34124  3.3067 |      22   2.88413  4.62467  3.50191  3.35541  4.34124  3.3067
     23   2.77440  4.31601  4.07605  3.60967  3.52856  3.8106 |      23   2.77440  4.31601  4.07605  3.60967  3.52856  3.8106
     24   2.70748  4.73939  2.87655  2.60901  3.90751  2.2196 |      24   2.70748  4.73939  2.87655  2.60901  3.90751  2.2196
     25   3.52675  4.82237  4.45989  4.11886  1.60810  4.2872 |      25   3.52675  4.82237  4.45989  4.11886  1.60810  4.2872
     26   2.76516  4.67273  3.25859  2.74237  3.37714  3.5622 |      26   2.76516  4.67273  3.25859  2.74237  3.37714  3.5622
     27   2.31963  4.29859  3.16387  2.88813  4.23521  3.0147 |      27   2.31963  4.29859  3.16387  2.88813  4.23521  3.0147
     28   2.75224  5.23597  1.85790  1.84537  4.54188  3.2920 |      28   2.75224  5.23597  1.85790  1.84537  4.54188  3.2920
     29   3.50091  4.84038  4.24225  3.97277  2.77096  3.8410 |      29   3.50091  4.84038  4.24225  3.97277  2.77096  3.8410
     30   2.93644  4.55178  3.70668  3.23846  1.84033  3.8094 |      30   2.93644  4.55178  3.70668  3.23846  1.84033  3.8094
     31   3.12614  4.45366  4.73780  4.22182  3.39661  4.4407 |      31   3.12614  4.45366  4.73780  4.22182  3.39661  4.4407
     32   3.00865  5.16345  3.38888  2.72984  4.60418  3.6420 |      32   3.00865  5.16345  3.38888  2.72984  4.60418  3.6420
     33   2.79692  4.25221  4.16650  3.61441  3.10590  3.9203 |      33   2.79692  4.25221  4.16650  3.61441  3.10590  3.9203
     34   2.66117  4.83255  2.65798  2.41775  4.34135  1.8124 |      34   2.66117  4.83255  2.65798  2.41775  4.34135  1.8124
     35   2.16933  4.39371  3.18674  2.68542  3.74483  3.3093 |      35   2.16933  4.39371  3.18674  2.68542  3.74483  3.3093
     36   3.07872  4.91328  3.57600  3.07365  4.28426  3.5542 |      36   3.07872  4.91328  3.57600  3.07365  4.28426  3.5542
     37   2.73126  4.84487  2.89920  2.55892  4.36044  2.2841 |      37   2.73126  4.84487  2.89920  2.55892  4.36044  2.2841
     38   2.45276  4.51299  2.80137  2.58837  4.24017  3.0859 |      38   2.45276  4.51299  2.80137  2.58837  4.24017  3.0859
     39   1.36483  4.13917  3.51074  3.18850  4.08415  2.9983 |      39   1.36483  4.13917  3.51074  3.18850  4.08415  2.9983
                                                              >           2.68620  4.42227  2.77521  2.73125  3.46356  2.4051
                                                              >           0.18564  1.83286  4.65950  0.31818  1.30003  0.4857
                                                              >      40   2.59981  4.78802  2.92470  2.45796  4.19674  3.3330
     40   2.59981  4.78802  2.92470  2.45796  4.19674  3.3330 |      41   2.72651  4.37086  3.81592  3.35942  3.40099  3.6537
     41   2.72651  4.37086  3.81592  3.35942  3.40099  3.6537 |      42   2.19751  4.28385  4.16978  3.62888  3.45828  3.9605
     42   2.19751  4.28385  4.16978  3.62888  3.45828  3.9605 |      43   2.65830  4.82635  3.02322  2.20606  4.05832  3.4415
     43   2.65830  4.82635  3.02322  2.20606  4.05832  3.4415 |      44   3.22795  4.67108  4.29325  3.93346  3.16072  4.0641
     44   3.22795  4.67108  4.29325  3.93346  3.16072  4.0641 |      45   2.66074  0.81820  4.19486  3.94925  3.92577  3.2700
     45   2.66074  0.81820  4.19486  3.94925  3.92577  3.2700 |      46   1.86592  4.46407  3.10222  2.32318  3.82945  3.3500
          2.68618  4.42225  2.77519  2.73123  3.46354  2.4051 <
          0.02940  3.93715  4.65950  0.61958  0.77255  0.4857 <
     46   1.86592  4.46407  3.10222  2.32318  3.82945  3.3500 <
     47   2.63754  4.79839  1.86127  2.27747  4.37728  1.8497 |      47   2.63754  4.79839  1.86127  2.27747  4.37728  1.8497
     48   2.82435  5.27823  1.80631  1.50862  4.56652  3.1510 |      48   2.82435  5.27823  1.80631  1.50862  4.56652  3.1510
     49   2.47524  4.25660  3.79275  3.24552  3.09824  3.7034 |      49   2.47524  4.25660  3.79275  3.24552  3.09824  3.7034
     50   2.74636  4.51539  3.43338  3.34100  4.48211  0.6359 |      50   2.74636  4.51539  3.43338  3.34100  4.48211  0.6359
     51   2.74843  4.93918  2.93139  2.48899  4.25232  3.4222 |      51   2.74843  4.93918  2.93139  2.48899  4.25232  3.4222
     52   2.75327  4.80092  3.24914  2.64794  4.10731  3.5030 |      52   2.75327  4.80092  3.24914  2.64794  4.10731  3.5030
     53   2.64715  4.24001  3.78399  3.23889  3.34264  3.6780 |      53   2.64715  4.24001  3.78399  3.23889  3.34264  3.6780
     54   2.44882  4.31107  3.40996  3.06280  3.84160  3.1991 |      54   2.44882  4.31107  3.40996  3.06280  3.84160  3.1991
     55   3.07089  4.41316  4.66872  4.11392  2.64390  4.3354 |      55   3.07089  4.41316  4.66872  4.11392  2.64390  4.3354
     56   2.70601  4.89845  2.81315  2.42763  4.07309  3.3810 |      56   2.70601  4.89845  2.81315  2.42763  4.07309  3.3810
     57   2.91242  4.50605  3.78428  3.27360  2.15229  3.8112 |      57   2.91242  4.50605  3.78428  3.27360  2.15229  3.8112
     58   2.58732  4.48962  3.18579  2.16860  3.53350  3.4758 |      58   2.58732  4.48962  3.18579  2.16860  3.53350  3.4758
     59   2.58927  4.73576  2.54558  2.42178  3.96985  3.3845 |      59   2.58927  4.73576  2.54558  2.42178  3.96985  3.3845
     60   2.86624  4.27570  4.30507  3.75705  2.07770  3.9547 |      60   2.86624  4.27570  4.30507  3.75705  2.07770  3.9547
     61   2.65604  4.68655  2.84820  2.59124  3.94599  1.7084 |      61   2.65604  4.68655  2.84820  2.59124  3.94599  1.7084
          2.68618  4.42225  2.77519  2.73123  3.46354  2.4051 <
          0.02940  3.93715  4.65950  0.61958  0.77255  0.4857 <
     62   2.50231  4.55131  2.83812  2.58854  4.18112  3.1529 <
     63   2.88802  4.43079  3.91928  3.41469  2.67519  3.8475 |      62   2.50231  4.55131  2.83812  2.58854  4.18112  3.1529
     64   2.37026  4.18114  3.57236  3.13131  3.76391  3.2040 |      63   2.88802  4.43079  3.91928  3.41469  2.67519  3.8475
     65   2.68503  4.33039  3.64856  3.08440  3.35689  3.6996 |      64   2.37026  4.18114  3.57236  3.13131  3.76391  3.2040
     66   2.25962  4.46869  3.08503  2.61031  3.82294  3.3023 |      65   2.68503  4.33039  3.64856  3.08440  3.35689  3.6996
     67   2.66074  0.81820  4.19486  3.94925  3.92577  3.2700 |      66   2.25962  4.46869  3.08503  2.61031  3.82294  3.3023
     68   2.59068  4.69360  2.87630  1.99115  3.99695  3.3463 |      67   2.66074  0.81820  4.19486  3.94925  3.92577  3.2700
     69   2.83824  4.58482  3.45240  3.30078  4.29038  3.2686 |      68   2.59068  4.69360  2.87630  1.99115  3.99695  3.3463
     70   3.05410  4.41546  4.61245  4.04979  2.25479  4.2748 |      69   2.83824  4.58482  3.45240  3.30078  4.29038  3.2686
                                                              >           2.68620  4.42227  2.77521  2.73125  3.46356  2.4051
                                                              >           0.18564  1.83286  4.65950  0.31818  1.30003  0.4857
                                                              >      70   3.05410  4.41546  4.61245  4.04979  2.25479  4.2748
     71   2.65326  4.77553  2.69084  1.84870  4.18823  3.2790 |      71   2.65326  4.77553  2.69084  1.84870  4.18823  3.2790
     72   3.08149  4.48239  4.42721  4.04553  3.41802  4.1360 |      72   3.08149  4.48239  4.42721  4.04553  3.41802  4.1360
     73   2.77741  4.77598  2.74482  2.63684  4.14967  3.2276 |      73   2.77741  4.77598  2.74482  2.63684  4.14967  3.2276
     74   2.66074  0.81820  4.19486  3.94925  3.92577  3.2700 |      74   2.66074  0.81820  4.19486  3.94925  3.92577  3.2700
     75   2.93252  4.92005  2.94611  2.70184  4.09716  3.4089 |      75   2.93252  4.92005  2.94611  2.70184  4.09716  3.4089
     76   2.33767  5.12191  2.00208  1.93476  4.43485  3.2635 |      76   2.33767  5.12191  2.00208  1.93476  4.43485  3.2635
     77   2.83824  4.58482  3.45240  3.30078  4.29038  3.2686 |      77   2.83824  4.58482  3.45240  3.30078  4.29038  3.2686
     78   2.29798  4.54318  3.08829  2.59682  3.90570  3.3372 |      78   2.29798  4.54318  3.08829  2.59682  3.90570  3.3372
     79   2.55843  4.25267  3.63823  3.14978  3.49893  2.5066 |      79   2.55843  4.25267  3.63823  3.14978  3.49893  2.5066
     80   2.74636  4.51539  3.43338  3.34100  4.48211  0.6359 |      80   2.74636  4.51539  3.43338  3.34100  4.48211  0.6359
     81   1.88499  4.36793  3.17701  2.76583  4.07428  3.1287 |      81   1.88499  4.36793  3.17701  2.76583  4.07428  3.1287
     82   3.22795  4.67108  4.29325  3.93346  3.16072  4.0641 |      82   3.22795  4.67108  4.29325  3.93346  3.16072  4.0641
     83   3.04975  4.39085  4.68139  4.18695  3.50129  4.3496 |      83   3.04975  4.39085  4.68139  4.18695  3.50129  4.3496
     84   1.93906  4.21937  3.93044  3.52997  3.71105  3.4515 |      84   1.93906  4.21937  3.93044  3.52997  3.71105  3.4515
     85   3.07872  4.91328  3.57600  3.07365  4.28426  3.5542 |      85   3.07872  4.91328  3.57600  3.07365  4.28426  3.5542
     86   2.66601  1.54657  4.04515  3.55144  3.04922  3.5407 |      86   2.66601  1.54657  4.04515  3.55144  3.04922  3.5407
     87   2.61383  4.33146  3.52836  3.00436  3.08671  3.5222 |      87   2.61383  4.33146  3.52836  3.00436  3.08671  3.5222
     88   3.33350  4.76025  4.12755  3.71678  1.81568  4.1148 |      88   3.33350  4.76025  4.12755  3.71678  1.81568  4.1148
     89   2.66631  4.52125  2.40981  2.73914  2.72623  3.5483 |      89   2.66631  4.52125  2.40981  2.73914  2.72623  3.5483
     90   2.62479  4.63595  3.04114  2.25905  3.57528  3.4498 |      90   2.62479  4.63595  3.04114  2.25905  3.57528  3.4498
     91   2.83947  5.37032  1.62621  1.77491  4.66039  3.2335 |      91   2.83947  5.37032  1.62621  1.77491  4.66039  3.2335
          2.68618  4.42225  2.77519  2.73123  3.46354  2.4051 |           2.68621  4.42228  2.77522  2.73126  3.46357  2.4050
          0.10526  3.93715  2.52064  0.61958  0.77255  0.4857 |           0.18564  2.41891  2.52064  0.83433  0.56946  0.4857
     92   2.49778  4.20437  3.57021  3.11809  2.05672  3.3868 |      92   2.49778  4.20437  3.57021  3.11809  2.05672  3.3868
     93   2.25576  4.28627  3.99142  3.43146  3.30410  3.8772 |      93   2.25576  4.28627  3.99142  3.43146  3.30410  3.8772
     94   2.31136  4.89111  2.48274  1.49202  4.30598  3.2230 |      94   2.31136  4.89111  2.48274  1.49202  4.30598  3.2230
     95   3.17967  4.66981  3.96732  3.52285  1.78944  3.9929 |      95   3.17967  4.66981  3.96732  3.52285  1.78944  3.9929
     96   2.82267  4.85009  3.10071  2.64896  3.76022  3.4747 |      96   2.82267  4.85009  3.10071  2.64896  3.76022  3.4747
     97   2.98011  5.10692  0.93090  2.32951  4.48275  3.2039 |      97   2.98011  5.10692  0.93090  2.32951  4.48275  3.2039
     98   2.93391  4.35583  4.41650  3.94115  3.46609  4.1090 |      98   2.93391  4.35583  4.41650  3.94115  3.46609  4.1090
     99   2.83102  4.73355  3.44527  2.85799  3.81220  3.5583 |      99   2.83102  4.73355  3.44527  2.85799  3.81220  3.5583
    100   2.80107  4.34618  4.11297  3.77587  3.54362  3.7146 |     100   2.80107  4.34618  4.11297  3.77587  3.54362  3.7146
    101   2.80107  4.34618  4.11297  3.77587  3.54362  3.7146 |     101   2.80107  4.34618  4.11297  3.77587  3.54362  3.7146
    102   3.14533  4.61026  4.17519  3.81504  3.13239  3.9708 |     102   3.14533  4.61026  4.17519  3.81504  3.13239  3.9708
    103   2.90028  5.00810  1.05095  2.28961  4.37453  3.1483 |     103   2.90028  5.00810  1.05095  2.28961  4.37453  3.1483
    104   3.15141  4.57971  4.11879  3.81891  1.16527  3.8573 |     104   3.15141  4.57971  4.11879  3.81891  1.16527  3.8573
    105   2.96366  4.40980  4.20729  3.82186  3.31837  3.9684 |     105   2.96366  4.40980  4.20729  3.82186  3.31837  3.9684

Happy to provide extra details!

cryptogenomicon commented 6 months ago

Those two alignments don't appear to contain the same sequence data, so I wouldn't expect them to give the same profile. They aren't just the same alignment in two different formats.

Where you say "Input a3m created by removing all columns where the query had a gap and also removing deletions (lower-case letters) in non-query sequences so that everything has exactly the same length", you mean you're deleting residues from the (Stockholm version of the) alignment? That's changing the sequences; the two alignments are different after those steps are taken.

I've only glanced at this quickly, so if I've misunderstood something, let me know.

Augustin-Zidek commented 6 months ago

Thanks for the blazingly fast reply!

Where you say "Input a3m created by removing all columns where the query had a gap and also removing deletions (lower-case letters) in non-query sequences so that everything has exactly the same length", you mean you're deleting residues from the (Stockholm version of the) alignment? That's changing the sequences; the two alignments are different after those steps are taken.

Yes, you are right that the inputs are not exactly the same. However, if I use the original (uncleaned) a3m:

>query
FHQECSLQSCTQHQPYVVDDPCPIHFYSKWYIRVGARKSAPLIELCVDEAGSKSPIQYIDIGNYTVSCLPFTINCQEPKLGSLVVRCSFYEDFLEYHDVRVVLDFI
>UniRef90_A0A2R3SUZ9/16-121 [subseq from] Uncharacterized protein n=1 Tax=Bat SARS-like coronavirus TaxID=1508227 RepID=A0A2R3SUZ9_CVHSA
FHQECSLQSCAQHQPYVVDDPCPIHFYSRWYIRVGARKSAPLIELCVDEVGSKSPIQYIDIGNYTVSCSPFTINCQEPKLGSLVVRCSYYEDFLEYHDIRVVLDFI
>UniRef90_A0A023PSW4/17-121 [subseq from] Protein 10b n=1 Tax=Rhinolophus affinis coronavirus TaxID=1487703 RepID=A0A023PSW4_CVHSA
-HKECTIQECCENQPYILEDPCPIHYYSDWYLKIGPRKSARLLQLCAGEYGKRLPVQYEKLGNYTINCEPFEINCQTPPVGSLIVRCSYDYDFIEYHDVRVVLDFI
>UniRef90_U5WLD0/17-121 [subseq from] Uncharacterized protein n=11 Tax=Betacoronavirus TaxID=694002 RepID=U5WLD0_CVHSA
-HKECSIQECCENQPFQLEDPCPIHYYSDWFVKIGPRKSARLVQLCAGEYGHRVPIHYEMFGNYTISCEPLEINCQNPPVGSLIVRCSYDVDFMEYHDVRVVLDFI
>UniRef90_Q0Q469/17-121 [subseq from] Non-structural protein 8 n=17 Tax=Betacoronavirus TaxID=694002 RepID=NS8_BC279
-HKECSIQECCENQPYQIEDPCPIHYYSDWFIKIGSRKSARLVQLCEGDYGKRIPIHYEMFGNYTISCEPLEINCQAPPVGSLIVRCSYDYDFVEHHDVRVVLDFI
>UniRef90_A0A0U1UYY4/17-121 [subseq from] Uncharacterized protein n=1 Tax=BtRs-BetaCoV/HuB2013 TaxID=1503302 RepID=A0A0U1UYY4_CVHSA
-HKECSIQECCEIQPSQIEDPSPIHYYSDWFIQIGYRKSARLVQLCEGDYGKRIPIHYEMFGNYSMYCEPLEINCQAPPVGRLIVRWLHDYESAEHHDVRVVLDFI
>UniRef90_D5HJV2/17-51 [subseq from] Uncharacterized protein n=1 Tax=Bat SARS coronavirus HKU3-8 TaxID=742005 RepID=D5HJV2_BCHK3
-HKECSIQECCENQPYQIEDPCPIHYYSDWFIKIGS----------------------------------------------------------------------
>UniRef90_Q3ZTE5/20-117 [subseq from] Orf10 n=10 Tax=Human SARS coronavirus TaxID=694009 RepID=Q3ZTE5_CVHSA
------VQRCASNKPHVLEDPCPTGYQPEWNIRYKTRGNTySTAWLCA--LGKVLPFHRW--HTMVQTCTPnVTINCQDPAGGALIARCWYLHEghqTAAFRDVLVVL---
>UniRef90_A0A0K1Z007/20-103 [subseq from] Uncharacterized protein n=6 Tax=Human SARS coronavirus TaxID=694009 RepID=A0A0K1Z007_CVHSA
------VQRCVSNTPYVLENPCPTGYRPEWNIRYNTRGNTyNTARLCA--LGK--VLSFHRWHTMVQACTPnITINCQDPVGGALVARCWYFHK--------------

I get the following error:

Alignment input parse error:
   sequence UniRef90_Q3ZTE5/20-117 has alen 111; expected 106
   while reading aligned FASTA file orig_input.a3m
   at or near line 17
cryptogenomicon commented 6 months ago

Our format autodetector is detecting the file as aligned FASTA. If you give it --informat a2m or make the file have a suffix of .a2m instead of .a3m, it should work fine.

more detail: It's not possible for our format detector to accurately distinguish aligned FASTA from A2M (or A3M) formats, because we have a requirement that we must not misread either the alignment or any associated annotation. There are ambiguous cases where the alignment is valid both as AFA or A2M/A3M, but A2M/A3M also implies annotation of consensus columns and AFA does not.

So we only autodetect AFA format, and are unable to autodetect A2M/A3M. A2M/A3M requires some sort of other affirmative hint from the user; either the file suffix is .a2m or --informat a2m is provided as a commandline option.

A3M format is the same as "dotless" A2M format, and unfortunately we're a little behind the times (our parsers predate the naming of A3M, I think) and we currently call both formats A2M. I've made a note to update the code so we can see the .a3m suffix too, which would also fix the problem you're seeing.

Augustin-Zidek commented 6 months ago

Oh, I see, thank you very much! --informat a2m indeed addressed the error I was seeing.

However, I think we will have to stick with using Stockholm nevertheless as a2m/a3m doesn't allow us to define consensus columns as far as I know.

I.e. we need the consensus columns to be all columns that don't have a gap in the query (the first sequence in the input MSA).

> query
AB--C
> seq 1
ABMMC
> seq 2
ABMMCdd
> seq 3
A--MkC

We can easily achieve that by setting GC RF=xx..x in Stockholm.

Is there a way to do this using a2m/afa?

cryptogenomicon commented 6 months ago

Yes, a2m/a3m use uppercase and '-' to denote consensus positions, whereas lowercase and '.' are insertions or gaps aligned to insertions. Each aligned sequence must have the same number of uppercase and '-' characters; that number is the number of consensus positions in the profile.

Our parser picks up a reference annotation line from a2m/a3m format based on this.

Augustin-Zidek commented 4 months ago

Thanks again, closing this issue.