`IndexError: index 0 is out of bounds for axis 0 with size 0` in `--use-fft` mode

[corneliusroemer]

On apparently OK data, treetime 0.10.0 (master) throws IndexError: index 0 is out of bounds for axis 0 with size 0 when run as part of augur refine in --use-fft mode:

❯ augur refine --tree results/hmpxv1/tree_fixed.nwk --alignment results/hmpxv1/masked.fasta --metadata results/hmpxv1/metadata.tsv --output-tree results/hmpxv1/tree.nwk --timetree --root MK783032 MK783030 --precision 3 --keep-polytomies --clock-rate 5.7e-05 --clock-std-dev 2e-5 --use-fft --output-node-data results/hmpxv1/branch_lengths.json --coalescent opt --date-inference marginal --date-confidence --clock-filter-iqd 0

augur refine is using TreeTime version 0.10.0

191.21  TreeTime.reroot: with method or node: ['MK783032', 'MK783030']

258.67  ###TreeTime.run: INITIAL ROUND

272.38  TreeTime.reroot: with method or node: ['MK783032', 'MK783030']

274.37  ###TreeTime.run: rerunning timetree after rerooting

322.01  ###TreeTime.run: ITERATION 1 out of 2 iterations

367.97  ###TreeTime.run: ITERATION 2 out of 2 iterations
Traceback (most recent call last):
  File "/opt/homebrew/Caskroom/miniforge/base/envs/py11/lib/python3.11/site-packages/treetime/treetime.py", line 57, in run
    return self._run(**kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniforge/base/envs/py11/lib/python3.11/site-packages/treetime/treetime.py", line 330, in _run
    self.calc_rate_susceptibility(rate_std=vary_rate, params=tt_kwargs)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/py11/lib/python3.11/site-packages/treetime/clock_tree.py", line 874, in calc_rate_susceptibility
    self.make_time_tree(**params)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/py11/lib/python3.11/site-packages/treetime/clock_tree.py", line 374, in make_time_tree
    self._ml_t_marginal()
  File "/opt/homebrew/Caskroom/miniforge/base/envs/py11/lib/python3.11/site-packages/treetime/clock_tree.py", line 732, in _ml_t_marginal
    res, res_t = NodeInterpolator.convolve_fft(msg_parent_to_node, node.branch_length_interpolator,
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniforge/base/envs/py11/lib/python3.11/site-packages/treetime/node_interpolator.py", line 234, in convolve_fft
    if Tmin<Tres_cropped[0] and left_slope<0:
            ~~~~~~~~~~~~^^^
IndexError: index 0 is out of bounds for axis 0 with size 0

To reproduce, run:

curl -L -o hmpxv1_fft_treetime_bug.tar.zst  https://github.com/neherlab/treetime/files/11530736/hmpxv1_fft_treetime_bug.tar.zst.txt
unzstd hmpxv1_fft_treetime_bug.tar.zst 
augur refine --tree results/hmpxv1/tree_fixed.nwk --alignment results/hmpxv1/masked.fasta --metadata results/hmpxv1/metadata.tsv --output-tree results/hmpxv1/tree.nwk --timetree --root MK783032 MK783030 --precision 3 --keep-polytomies --clock-rate 5.7e-05 --clock-std-dev 2e-5 --use-fft --output-node-data results/hmpxv1/branch_lengths.json --coalescent opt --date-inference marginal --date-confidence --clock-filter-iqd 0

hmpxv1_fft_treetime_bug.tar.zst.txt

[corneliusroemer]

Without --use-fft it seems to work and not take much longer, completing in 10m, as opposed to erroring after 7m15s.

[corneliusroemer]

With treetime 0.9.6, this didn't seem to cause an error but result in a warning instead:

$ augur refine --tree results/hmpxv1/tree_fixed.nwk --alignment results/hmpxv1/masked.fasta --metadata results/hmpxv1/metadata.tsv --output-tree results/hmpxv1/tree.nwk --timetree --root MK783032 MK783030 --precision 3 --keep-polytomies --clock-rate 5.7e-05 --clock-std-dev 2e-5 --use-fft --output-node-data results/hmpxv1/branch_lengths.json --coalescent opt --date-inference marginal --date-confidence --clock-filter-iqd 0 
augur refine is using TreeTime version 0.9.6

71.35   WARNING: Previous versions of TreeTime (<0.7.0) RECONSTRUCTED sequences of
        tips at positions with AMBIGUOUS bases. This resulted in unexpected
        behavior is some cases and is no longer done by default. If you want to
        replace those ambiguous sites with their most likely state, rerun with
        `reconstruct_tip_states=True` or `--reconstruct-tip-states`.

188.75  TreeTime.reroot: with method or node: ['MK783032', 'MK783030']

254.55  ###TreeTime.run: INITIAL ROUND

271.18  TreeTime.reroot: with method or node: ['MK783032', 'MK783030']

273.23  ###TreeTime.run: rerunning timetree after rerooting

320.28  ###TreeTime.run: ITERATION 1 out of 2 iterations
Warning: evaluating log probability of a delta distribution.

366.03  ###TreeTime.run: ITERATION 2 out of 2 iterations
Warning: evaluating log probability of a delta distribution.
Warning: evaluating log probability of a delta distribution.
Warning: evaluating log probability of a delta distribution.
Warning: evaluating log probability of a delta distribution.

Inferred a time resolved phylogeny using TreeTime:
        Sagulenko et al. TreeTime: Maximum-likelihood phylodynamic analysis
        Virus Evolution, vol 4, https://academic.oup.com/ve/article/4/1/vex042/4794731

updated tree written to results/hmpxv1/tree.nwk
node attributes written to results/hmpxv1/branch_lengths.json

[rneher]

should be fixed in v0.10.1