Is this a bug or not? (Non-deterministic result for 1-reticulation network, sometimes bad reticulation probabilities)

[lutteropp]

NetRAX ran twice on the same dataset (with LikelihoodType.BEST and BrlenLinkageType.SCALED). First time it worked perfectly fine, second time it returned a nonsense network where the reticulation had probability 1.0...: cmd_output.txt

The only non-deterministic parts in NetRAX, when given a start network:

• brlen opt
• reticulation opt
• model opt
• randomly inserting a reticulation when going from n->n+1 reticulations in the "search in waves" mode

Out of these, the fourth point (the randomly added reticulation) seems most likely to be causing the issue.

The question is: Is it (sometimes returning a nonsense network with probability-1-reticulations) really a bug? Or are the current horizontal moves not "strong" enough to relocate a reticulation?

[lutteropp]

Maybe we need to add another horizontal move type: The combination of an ArcRemoval move with an ArcInsertion move. (That move type would benefit a lot from the whiteboard discussion (https://github.com/lutteropp/NetRAX/issues/30#issuecomment-741810684) we had about figuring out which places are best for putting a reticulation to).

I remember that @celinescornavacca did not like randomly placing a reticulation.

Maybe instead of randomly putting that reticulation somewhere, one should put it to the most likely place among all reticulation candidates... But maybe it is enough to try with a DeltaPlus move then, since there are too many general ArcInsertion moves.

[lutteropp]

The first run returned this network:

Best optimized 1-reticulation network loglikelihood: -4608.81
Best optimized 1-reticulation network BIC score: 9478.27
IMPROVED BEST SCORE FOUND SO FAR: 9478.27
(((8:0.00279279)#0:0.0126592::0.526265,(9:0.0134016,#0:1.32952e-06::0.473735):1.47159e-06):1.47159e-06,(5:0.246491,6:0.0326704):0.00638227);

The second run returned this network:

Best optimized 1-reticulation network loglikelihood: -4611.94
Best optimized 1-reticulation network BIC score: 9484.53
IMPROVED BEST SCORE FOUND SO FAR: 9484.53
(5:0.010708,((((8:0.00969668,9:0.013151):0.00253654)#0:0.00344504::1,6:0.0320626):0.127318,#0:3.91008e-05::0):0.114586);

[lutteropp]

The network returned by the first run in a picture:

The network returned by the second run in a picture (with the reticulation having prob 0/1):

[lutteropp]

I'm pretty much convinced by now that the problem is that the current horizontal moves are not "strong" enough to arbitrarily relocate a badly-placed reticulation in a single step. That's my working hypothesis at least.

[lutteropp]

Oh... but also if I look at this "true" simulated network, probably the best inferred network should have been a tree here. The simulated network has a weird reticulation, where actually both displayed tree topologies are the same one (only difference is maybe 1 differing branch length)

If switching from scaled to unlinked branch lengths mode, the inferred network should definitely be a 0-reticulation tree here!

[lutteropp]

I redid the runs with unlinked branch lengths mode, and indeed NetRAX inferred a zero-reticulation tree in this case.

[lutteropp]

I made a new issue for discussing the "true" simulated network and the effects of branch length linkage here: https://github.com/lutteropp/NetRAX/issues/33

[lutteropp]

And what likely caused the high variation in the resulting inferred networks is described in this issue: https://github.com/lutteropp/NetRAX/issues/34

[lutteropp]

The only question here remains: How did NetRAX end up with a reticulation with probability 1.0 in that second run? Was it because of so much better brlen optimization than it had for the initial tree, so much better model optimization than it had for the initial tree, or some bug (in re-rolling the network after undoing a move) leading to wrongly computed loglikelihoods?

[lutteropp]

In order to cancel out the bug possibility regarding un-rolling/re-rolling changes, I decided to finally implement network checkpointing: https://github.com/lutteropp/NetRAX/issues/35

[lutteropp]

Now that I have created child-issues for all aspects of this problem, there is nothing left anymore to discuss in this issue here. Closing it soon.

[stamatak]

Those LnL differences are not large

On 17.12.20 16:08, Sarah Lutteropp wrote:

The first run returned this network:

Best optimized 1-reticulation network loglikelihood: -4608.81 Best optimized 1-reticulation network BIC score: 9478.27 IMPROVED BEST SCORE FOUND SO FAR: 9478.27 (((8:0.00279279)#0:0.0126592::0.526265,(9:0.0134016,#0:1.32952e-06::0.473735):1.47159e-06):1.47159e-06,(5:0.246491,6:0.0326704):0.00638227);

The second run returned this network:

Best optimized 1-reticulation network loglikelihood: -4611.94 Best optimized 1-reticulation network BIC score: 9484.53 IMPROVED BEST SCORE FOUND SO FAR: 9484.53 (5:0.010708,((((8:0.00969668,9:0.013151):0.00253654)#0:0.00344504::1,6:0.0320626):0.127318,#0:3.91008e-05::0):0.114586);

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/lutteropp/NetRAX/issues/32#issuecomment-747459632, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGXB6SAY7RY4RKJ7OOXJ23SVIGENANCNFSM4U7VSDLA.

-- Alexandros (Alexis) Stamatakis

Research Group Leader, Heidelberg Institute for Theoretical Studies Full Professor, Dept. of Informatics, Karlsruhe Institute of Technology

www.exelixis-lab.org

[stamatak]

what is the meaning of the labels and numbers in the network viz?

On 17.12.20 16:25, Sarah Lutteropp wrote:

The network returned by the first run in a picture: 1 https://user-images.githubusercontent.com/1059869/102499841-f4f84b00-407b-11eb-845d-a6e3beeabd48.png

The network returned by the second run in a picture (with the reticulaton having prob 0/1): 2 https://user-images.githubusercontent.com/1059869/102499861-fa559580-407b-11eb-9ec7-48799e01e164.png

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/lutteropp/NetRAX/issues/32#issuecomment-747469221, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGXB6T47ZIQYPJ3DF4H4B3SVIIEXANCNFSM4U7VSDLA.

-- Alexandros (Alexis) Stamatakis

Research Group Leader, Heidelberg Institute for Theoretical Studies Full Professor, Dept. of Informatics, Karlsruhe Institute of Technology

www.exelixis-lab.org

[lutteropp]

what is the meaning of the labels and numbers in the network viz?

Those are simply the clv indices of the nodes in the network. If a node has a label, then I visualize it in the form clv_index : node_label. The taxon labels I got from Celine's simulator are 5,6,8,9.

Similarly, for the edges I am printing their pmatrix_index.

... It originates from this visualization originally being used for debugging the operations array for loglikelihood computation.

[celinescornavacca]

Please do not close the issue yet. I would like to concentrate our effort to understand better what is going on here, and find a way to get the true network for this small example.

First, note that the second network you computed was only one tail move from the true one, so we are not so bad. Second, we need the inheritance probabilities from the true network to understand better why we get the 0/1. Also, if you give the true topology, do you get 0/1? (the question underlying all this being, is the algorithm for inheritance probabilities buggy?) And with AVERAGE (and not BEST), how scaled/unlinked perform ? What about the LnL of the true network?

We need some numbers to crunch.

because of so much better brlen optimization than it had for the initial tree

Possibly yes, possibly the branch lengths between the two displayed trees are not very different (please post the newick for the true network)

[lutteropp]

This is the Extended NEWICK for the true simulated network: (((6:0.026220429886411434,(8:0.01232094961373149,9:0.01232094961373149):0.013899480272679942):0.07175112780701653,#H1:0.0070261784688911534::0.6370941658922615):0.09088149459958164,(5:0.09094537922453681)#H1:0.0979076730684728::0.36290583410773847);

The scores for this true network with LikelihoodModel.BEST, BrlenLinkage.LINKED:

sarah@gram-3:~/code-workspace/NetRAX/simulator/src/network/logic$ ../../../../bin/netrax --msa datasets_small_network_single_debug/0_4_taxa_1_reticulations_SimulatorType.CELINE_SamplingType.PERFECT_SAMPLING_2000_msasize_msa.txt --model datasets_small_network_single_debug/0_4_taxa_1_reticulations_SimulatorType.CELINE_SamplingType.PERFECT_SAMPLING_2000_msasize_partitions.txt --start_network datasets_small_network_single_debug/0_4_taxa_1_reticulations_SimulatorType.CELINE_SamplingType.PERFECT_SAMPLING_2000_msasize_true_network.nw --score_only --best_displayed_tree_variant --brlen linked
BIC score after model optimization: 9471.29
Initial, given network:
((5:0.0909454)#H1:0.0979077::0.362906,(#H1:0.00702618::0.637094,((9:0.0123209,8:0.0123209):0.0138995,6:0.0262204):0.0717511):0.0908815);
Initial (before brlen and reticulation opt) BIC Score: 9471.29
Initial (before brlen and reticulation opt) loglikelihood: -4609.82
BIC score after branch length optimization: 9465.21
BIC score after updating reticulation probs: 9465.06
BIC score after model optimization: 9464.67
Network after optimization of brlens and reticulation probs:
((5:0.0909454)#H1:0.117765::0.484997,(#H1:0.013112::0.515003,((9:0.0123209,8:0.00965158):0.0071585,6:0.0288426):0.0717511):0.0908815);
Number of reticulations: 1
BIC Score: 9464.67
Loglikelihood: -4606.51

The scores for this true network with LikelihoodModel.BEST, BrlenLinkage.SCALED:

sarah@gram-3:~/code-workspace/NetRAX/simulator/src/network/logic$ ../../../../bin/netrax --msa datasets_small_network_single_debug/0_4_taxa_1_reticulations_SimulatorType.CELINE_SamplingType.PERFECT_SAMPLING_2000_msasize_msa.txt --model datasets_small_network_single_debug/0_4_taxa_1_reticulations_SimulatorType.CELINE_SamplingType.PERFECT_SAMPLING_2000_msasize_partitions.txt --start_network datasets_small_network_single_debug/0_4_taxa_1_reticulations_SimulatorType.CELINE_SamplingType.PERFECT_SAMPLING_2000_msasize_true_network.nw --score_only --best_displayed_tree_variant --brlen scaled
BIC score after model optimization: 9480.28
Initial, given network:
((5:0.0909454)#H1:0.0979077::0.362906,(#H1:0.00702618::0.637094,((9:0.0123209,8:0.0123209):0.0138995,6:0.0262204):0.0717511):0.0908815);
Initial (before brlen and reticulation opt) BIC Score: 9480.28
Initial (before brlen and reticulation opt) loglikelihood: -4609.82
BIC score after branch length optimization: 9473.96
BIC score after branch length scaler optimization: 9473.96
BIC score after updating reticulation probs: 9473.8
BIC score after model optimization: 9473.41
Network after optimization of brlens and reticulation probs:
((5:0.0923297)#H1:0.119558::0.484985,(#H1:0.0133116::0.515015,((9:0.0125085,8:0.00979849):0.00726746,6:0.0292816):0.0728433):0.0922649);
Number of reticulations: 1
BIC Score: 9473.41
Loglikelihood: -4606.39

The scores for this true network with LikelihoodModel.BEST, BrlenLinkage.UNLINKED:

sarah@gram-3:~/code-workspace/NetRAX/simulator/src/network/logic$ ../../../../bin/netrax --msa datasets_small_network_single_debug/0_4_taxa_1_reticulations_SimulatorType.CELINE_SamplingType.PERFECT_SAMPLING_2000_msasize_msa.txt --model datasets_small_network_single_debug/0_4_taxa_1_reticulations_SimulatorType.CELINE_SamplingType.PERFECT_SAMPLING_2000_msasize_partitions.txt --start_network datasets_small_network_single_debug/0_4_taxa_1_reticulations_SimulatorType.CELINE_SamplingType.PERFECT_SAMPLING_2000_msasize_true_network.nw --score_only --best_displayed_tree_variant --brlen unlinked
BIC score after model optimization: 9552.18
Initial, given network:
((5:0.0909454)#H1:0.0979077::0.362906,(#H1:0.00702618::0.637094,((9:0.0123209,8:0.0123209):0.0138995,6:0.0262204):0.0717511):0.0908815);
Initial (before brlen and reticulation opt) BIC Score: 9552.18
Initial (before brlen and reticulation opt) loglikelihood: -4609.82
BIC score after branch length optimization: 9540.77
BIC score after updating reticulation probs: 9538.97
BIC score after model optimization: 9538.58
Network after optimization of brlens and reticulation probs:
((5:0.17779)#H1:1.68813::0,(#H1:0.0241656::1,((9:0.0131559,8:0.00824083):0.00675589,6:0.0316441):0.0717511):0.48182);
Number of reticulations: 1
BIC Score: 9538.58
Loglikelihood: -4603.02

The scores for this true network with LikelihoodModel.AVERAGE, BrlenLinkage.LINKED:

sarah@gram-3:~/code-workspace/NetRAX/simulator/src/network/logic$ ../../../../bin/netrax --msa datasets_small_network_single_debug/0_4_taxa_1_reticulations_SimulatorType.CELINE_SamplingType.PERFECT_SAMPLING_2000_msasize_msa.txt --model datasets_small_network_single_debug/0_4_taxa_1_reticulations_SimulatorType.CELINE_SamplingType.PERFECT_SAMPLING_2000_msasize_partitions.txt --start_network datasets_small_network_single_debug/0_4_taxa_1_reticulations_SimulatorType.CELINE_SamplingType.PERFECT_SAMPLING_2000_msasize_true_network.nw --score_only --brlen linked
BIC score after model optimization: 9471.29
Initial, given network:
((5:0.0909454)#H1:0.0979077::0.362906,(#H1:0.00702618::0.637094,((9:0.0123209,8:0.0123209):0.0138995,6:0.0262204):0.0717511):0.0908815);
Initial (before brlen and reticulation opt) BIC Score: 9471.29
Initial (before brlen and reticulation opt) loglikelihood: -4609.82
BIC score after branch length optimization: 9465.21
BIC score after updating reticulation probs: 9465.06
BIC score after model optimization: 9464.67
Network after optimization of brlens and reticulation probs:
((5:0.0909454)#H1:0.117765::0.484997,(#H1:0.0131128::0.515003,((9:0.0123209,8:0.00965159):0.00715838,6:0.0288426):0.0717511):0.0908815);
Number of reticulations: 1
BIC Score: 9464.67
Loglikelihood: -4606.51

The scores for this true network with LikelihoodModel.AVERAGE, BrlenLinkage.SCALED:

sarah@gram-3:~/code-workspace/NetRAX/simulator/src/network/logic$ ../../../../bin/netrax --msa datasets_small_network_single_debug/0_4_taxa_1_reticulations_SimulatorType.CELINE_SamplingType.PERFECT_SAMPLING_2000_msasize_msa.txt --model datasets_small_network_single_debug/0_4_taxa_1_reticulations_SimulatorType.CELINE_SamplingType.PERFECT_SAMPLING_2000_msasize_partitions.txt --start_network datasets_small_network_single_debug/0_4_taxa_1_reticulations_SimulatorType.CELINE_SamplingType.PERFECT_SAMPLING_2000_msasize_true_network.nw --score_only --brlen scaled
BIC score after model optimization: 9480.28
Initial, given network:
((5:0.0909454)#H1:0.0979077::0.362906,(#H1:0.00702618::0.637094,((9:0.0123209,8:0.0123209):0.0138995,6:0.0262204):0.0717511):0.0908815);
Initial (before brlen and reticulation opt) BIC Score: 9480.28
Initial (before brlen and reticulation opt) loglikelihood: -4609.82
BIC score after branch length optimization: 9473.96
BIC score after branch length scaler optimization: 9473.96
BIC score after updating reticulation probs: 9473.8
BIC score after model optimization: 9473.41
Network after optimization of brlens and reticulation probs:
((5:0.0923298)#H1:0.119558::0.484985,(#H1:0.0133124::0.515015,((9:0.0125085,8:0.00979851):0.00726734,6:0.0292816):0.0728434):0.0922649);
Number of reticulations: 1
BIC Score: 9473.41
Loglikelihood: -4606.39

The scores for this true network with LikelihoodModel.AVERAGE, BrlenLinkage.UNLINKED:

sarah@gram-3:~/code-workspace/NetRAX/simulator/src/network/logic$ ../../../../bin/netrax --msa datasets_small_network_single_debug/0_4_taxa_1_reticulations_SimulatorType.CELINE_SamplingType.PERFECT_SAMPLING_2000_msasize_msa.txt --model datasets_small_network_single_debug/0_4_taxa_1_reticulations_SimulatorType.CELINE_SamplingType.PERFECT_SAMPLING_2000_msasize_partitions.txt --start_network datasets_small_network_single_debug/0_4_taxa_1_reticulations_SimulatorType.CELINE_SamplingType.PERFECT_SAMPLING_2000_msasize_true_network.nw --score_only --brlen unlinked
BIC score after model optimization: 9552.18
Initial, given network:
((5:0.0909454)#H1:0.0979077::0.362906,(#H1:0.00702618::0.637094,((9:0.0123209,8:0.0123209):0.0138995,6:0.0262204):0.0717511):0.0908815);
Initial (before brlen and reticulation opt) BIC Score: 9552.18
Initial (before brlen and reticulation opt) loglikelihood: -4609.82
BIC score after branch length optimization: 9541.9
BIC score after updating reticulation probs: 9541.74
BIC score after model optimization: 9541.36
Network after optimization of brlens and reticulation probs:
((5:0.0909454)#H1:0.119153::0.484845,(#H1:42.7122::0.515155,((9:0.013156,8:0.00824083):0.00675581,6:0.0316442):0.0717511):0.102199);
Number of reticulations: 1
BIC Score: 9541.36
Loglikelihood: -4604.41

[lutteropp]

And now, the results for NetRAX running with the true network used as starting network (keep in mind that those results currently might be non-deterministic. Also keep in mind that NetRAX search-in-waves currently does not try to search networks with less reticulations than the specified input network!!!):

NetRAX starting from true network with LikelihoodModel.BEST, BrlenLinkage.LINKED (it took 4 rNNI moves):

Best optimized 1-reticulation network loglikelihood: -4606.5
Best optimized 1-reticulation network BIC score: 9464.66
Best found network is:
((5:0.0909454)#H1:0.108363::0.485107,(#H1:0.00702618::0.514893,((9:0.0123209,8:0.00964894):0.0071675,6:0.0288426):0.0770253):0.0908815);

NetRAX starting from true network with LikelihoodModel.BEST, BrlenLinkage.SCALED (it took 4 rNNI moves):

Best optimized 1-reticulation network loglikelihood: -4606.38
Best optimized 1-reticulation network BIC score: 9473.4
Best found network is:
((5:0.0926236)#H1:0.110363::0.485107,(#H1:0.00715583::0.514893,((9:0.0125483,8:0.00982699):0.00729976,6:0.0293748):0.0784467):0.0925585);

NetRAX starting from true network with LikelihoodModel.BEST, BrlenLinkage.UNLINKED (it took 3 rNNI moves, 3 head moves, 2 rSPR1 moves):

Best optimized 1-reticulation network loglikelihood: -4602.96
Best optimized 1-reticulation network BIC score: 9538.46
((((9:0.00622708)#H1:0.00693643::1,8:0.00849187):0.00690408,6:0.0308634):0.0703799,(5:0.200511,#H1:11.5385::0):0.000881952);

NetRAX starting from true network with LikelihoodModel.AVERAGE, BrlenLinkage.LINKED (it took 3 rNNI moves):

Best optimized 1-reticulation network loglikelihood: -4606.58
Best optimized 1-reticulation network BIC score: 9464.81
Best found network is:
((((9:0.0123209,8:0.00966498):0.00717175,6:0.0288426):0.0909454)#H1:0.120394::0.485107,(#H1:0.00702618::0.514893,5:0.0770387):0.0908815);

NetRAX starting from true network with LikelihoodModel.AVERAGE, BrlenLinkage.SCALED (it took 6 rNNI moves):

Best optimized 1-reticulation network loglikelihood: -4606.21
Best optimized 1-reticulation network BIC score: 9473.07
((5:0.0943719)#H1:0.12493::0.485107,(#H1:0.00603547::0.514893,((9:0.0127852,8:0.0100291):0.00744196,6:0.0299293):0.071947):0.0943056);

NetRAX starting from true network with LikelihoodModel.AVERAGE, BrlenLinkage.UNLINKED (it took 2 rNNI moves):

Best optimized 1-reticulation network loglikelihood: -4604.4
Best optimized 1-reticulation network BIC score: 9541.34
Best found network is:
((5:0.0909454)#H1:0.137181::0.485107,(#H1:42.7097::0.514893,((9:0.0131536,8:0.00823747):0.00674504,6:0.0316331):0.0739675):0.286351);

[celinescornavacca]

Thanks, your outputs makes me think of something that I will write in issue #33.

About this specific example, I think that the 0/1 probability may come from the fact that the network is ultrametric, see my drawing:

Notes_201218_181720.pdf

[celinescornavacca]

(note the #H1:11.5385 and #H1:42.7097 branch lengths in the UNLINKED case)

[stamatak]

thanks

On 17.12.20 21:37, Sarah Lutteropp wrote:

what is the meaning of the labels and numbers in the network viz?

Those are simply the clv indices of the nodes in the network. If a node has a label, then I visualize it in the form |clv_index : node_label|. The taxon labels I got from Celine's simulator are 5,6,8,9.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/lutteropp/NetRAX/issues/32#issuecomment-747655387, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGXB6XPFR35MUJW335BIRTSVJMWLANCNFSM4U7VSDLA.

-- Alexandros (Alexis) Stamatakis

Research Group Leader, Heidelberg Institute for Theoretical Studies Full Professor, Dept. of Informatics, Karlsruhe Institute of Technology

www.exelixis-lab.org

[lutteropp]

Today I learned that reticulation probability of 1 or 0 is not a bug. I played around with the Maths, turns out not only BIC restricts the number of reticulations. Even regarding only network loglikelihood, there are situations where a 0/1 reticulation probability leads to a higher network loglikelihood. No matter whether we use LikelihoodModel.AVERAGE or LikelihoodModel.BEST.

Thus, the sometimes occurring 0/1 reticulation probability is not a bug. Instead it says that the search algorithm needs to be improved. Maybe in case of a 0/1 reticulation probability occurring, we need to "downgrade" the network by removing that reticulation and then restart the search from there...

[stamatak]

Hi Sarah,

Does what you write here still hold or has it become obsolete?

Alexis

On 05.01.21 15:09, Sarah Lutteropp wrote:

Today I learned that reticulation probability of 1 or 0 is not a bug. I played around with the Maths, turns out not only BIC restricts the number of reticulations. Even regarding only network loglikelihood, there are situation where a 0/1 reticulation probability leads to a higher network loglikelihood. No matter whether we use LikelihoodModel.AVERAGE or LikelihoodModel.BEST.

Thus, the sometimes occurring 0/1 reticulation probability is not a bug. Instead it says that the search algorithm needs to be improved. Maybe in case of a 0/1 reticulation probability occurring, we need to "downgrade" the network by removing that reticulation and then restart the search from there...

less reticulations through loglh https://user-images.githubusercontent.com/1059869/103649576-ff14b780-4f5e-11eb-8c57-ce3388372ea7.png

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/lutteropp/NetRAX/issues/32#issuecomment-754625627, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGXB6WD23MO7GZX4QDU24LSYMFPXANCNFSM4U7VSDLA.

-- Alexandros (Alexis) Stamatakis

Research Group Leader, Heidelberg Institute for Theoretical Studies Full Professor, Dept. of Informatics, Karlsruhe Institute of Technology

www.exelixis-lab.org

[lutteropp]

The 0/1 reticulation probs issue has been resolved. I had code trying out those extreme values before doing brent optimization of the reticulation probs... that somehow messed things up. I removed that code part and am now only using the standard brent optimization.

[lutteropp]

However, we are still not inferring the wanted network. Instead, NetRAX infers a tree here, as the BIC prefers zero reticulations for some reason. I will post exact loglh and BIC values for all settings (brlen-linkage, likelihood model) here, after I fixed the current bug I am at (failed assertion when using scaled brlens)... If we are lucky, NetRAX will infer the correct number of reticulations after the current ongoing bugfixes.

[lutteropp]

I just realized that I do not have the original MSA for this network anymore, thus I cannot do another run on that specific dataset (it got overridden when re-running the experiments script). :-(

I am closing this issue now. If the problem reappears in another, new dataset, I will create a new issue for it.