Switch from ete3 to ete4

[dengzq1234]

PR Checklist

• [X] Referenced issue is linked
• [X] If you've fixed a bug or added code that should be tested, add tests!
• [ ] Documentation in docs is updated

Description of changes

This PR is the response to the issue https://github.com/etetoolkit/ete/issues/739, which aim to upgrade the usage of etetoolkit in pertpy from ete3 to ete4.

perpty implement ete for the following tasks: a) browsing trees, such as getting leaves, descendants, ancestors, etc. b) annotation of trees, such as adding information to nodes. c) drawing tree, including drawing tree with layouts of nodes' information.

This repo upgrade the syntax in regards to the first two tasks. The third task which include the drawing part will mostly maintain the usage of ete3 version expect for calling the module in new places

Technical details This PR mostly just correct the syntax of calling the methods or features from ete3 to ete4, straight forward and trivial. detailed differences can be found from

• migration noteos: https://github.com/etetoolkit/ete/wiki/3to4
• ete4 official tutorial https://etetoolkit.github.io/ete/tutorial/tutorial_trees.html#browsing-trees-traversing

I used two ways to conduct the test 1) Passed the unitest this folder /home/deng/Projects/metatree_drawer/test_pertpy/pertpy/tests/tools/_coda

2) Be able to recreate tree images from the step-by-step tutorial of tasccoda tree-aggregated analysis from https://pertpy.readthedocs.io/en/latest/tutorials/notebooks/tasccoda.html

Additional context ete4 installation, ete4 currently is released on githup repo as beta version therefore not yet available in pip nor conda. Users can install ete4 via githup pip install https://github.com/etetoolkit/ete/archive/ete4.zip

In regards to c) drawing tree, ete4 still contains the treeview inherited from ete3 which relies on novel PyQt6 module to render tree image. Users can install the depencies by pip install PyQt6 or install ete4 via pip install -e .[treeview] for a local editable installation.

Important information: In ete4 we will stop maintaining nor making new updates in the old treeview module which is used in pertpy, at this moment it is still functional.

Instead ete4 embrace the new smartview module which has abilities to visualize large tree dataset up to thousands of tips interactively and introduce with novel layouts. The new drawing engine has been implemented in some tools such as PhyloCloud https://phylocloud.cgmlab.org/ (https://doi.org/10.1093/nar/gkac324) TreeProfiler https://doi.org/10.1101/2023.09.21.558621

If pertpy has the demands to visualize trees with ete4 smartview, I'm more than welcome to provide a novel PR in regards to this feature.

[Zethson]

Thank you very much!

ete4 installation, ete4 currently is released on githup repo as beta version therefore not yet available in pip nor conda. Users can install ete4 via github pip install https://github.com/etetoolkit/ete/archive/ete4.zip

This is why I was asking for a release of ete4 on pypi. Currently, I cannot merge/accept this PR because I could not distribute pertpy on pypi. Github dependencies are not supported on pypi for obvious reasons.

What is the status of a pypi release of ete4?

Thank you again!

[Zethson]

Important information: In ete4 we will stop maintaining nor making new updates in the old treeview module which is used in pertpy, at this moment it is still functional.

Instead ete4 embrace the new smartview module which has abilities to visualize large tree dataset up to thousands of tips interactively and introduce with novel layouts. The new drawing engine has been implemented in some tools such as PhyloCloud https://phylocloud.cgmlab.org/ (https://doi.org/10.1093/nar/gkac324) TreeProfiler https://doi.org/10.1101/2023.09.21.558621

If pertpy has the demands to visualize trees with ete4 smartview, I'm more than welcome to provide a novel PR in regards to this feature.

@johannesostner could you please comment on this?

[johannesostner]

Hi @dengzq1234, thanks for helping us with porting our code from ete3 to ete4! Regarding the drawing engine, most of our tree structures are quite small (up to 100 tips), so the performance boost from smartview is not that important for us. Nevertheless, I think that replacing treeview with smartview would be preferable to ensure future compatibility.

Some questions:

• How well does smartview integrate with jupyter notebooks? I think most of our users would like to see the plots rendered directly below their code cells
• Do you know a better way to create the last plot from https://pertpy.readthedocs.io/en/latest/tutorials/notebooks/tasccoda.html (tree with a barplot for every leaf?) Currently, the implmentation for this is quite hacky (save the tree as png and import it into a matplotlib plot).

[dengzq1234]

Thank you very much!

ete4 installation, ete4 currently is released on githup repo as beta version therefore not yet available in pip nor conda. Users can install ete4 via github pip install https://github.com/etetoolkit/ete/archive/ete4.zip

This is why I was asking for a release of ete4 on pypi. Currently, I cannot merge/accept this PR because I could not distribute pertpy on pypi. Github dependencies are not supported on pypi for obvious reasons.

What is the status of a pypi release of ete4?

Thank you again!

I see the issue, the current ete4 is undergoing development on one of the major features which is not showing on the github yet therefore we haven't put it on pypi. I will need to coordinate to see if we want to release the beta version on pypi or wait until it is fully ready. I will keep you updated!

[dengzq1234]

Hi @dengzq1234, thanks for helping us with porting our code from ete3 to ete4! Regarding the drawing engine, most of our tree structures are quite small (up to 100 tips), so the performance boost from smartview is not that important for us. Nevertheless, I think that replacing treeview with smartview would be preferable to ensure future compatibility.

Some questions:

• How well does smartview integrate with jupyter notebooks? I think most of our users would like to see the plots rendered directly below their code cells
• Do you know a better way to create the last plot from https://pertpy.readthedocs.io/en/latest/tutorials/notebooks/tasccoda.html (tree with a barplot for every leaf?) Currently, the implmentation for this is quite hacky (save the tree as png and import it into a matplotlib plot).

Hi! If the tree is relatively small then I think treeview at this moment will handle just fine and tree will be shown clearly.

To your questions: 1) Here is how it goes in Jupyter notebook, it provide a interactive interface so users can manipulate the tree.

2) I have seen the barplot drawing in the function, and yes and no for current ete4. Yes we can draw the barplot for every leaf, such as

And no because your barplot is bidirectional barplot with negative, that's something we are currently working on it.

One of the downside of current ete4 is instead of drawing a tree as static image, we open the tree as a browsing and interactive session, users need to active this browsing session in order to output the static png/pdf for tree. For users who need to automate the tree drawing pipeline, it might be a hinder. But as you said if your users would like to see the tree in the jupyter cells, then I would say ete4 smartview fit this purpose.