Scverse extension packages

[adamgayoso]

It should be easy for methods developers to package up their computational methods into a nice package on PyPI that is plug and play with the rest of the ecosystem.

In this issue we can discuss creating a github template repo for developing and deploying packages that connect to the rest of scverse.

The way I see this is that it reduces overhead in dev time, but also provides a readily usable package that doesn't "need" to go into scanpy to be broadly useful. If some approach becomes fundamental in analysis, then I can see it going into scanpy/muon/etc

[giovp]

let's do that, what should this template contain?

• tests?
• docs?
• requirements?

I would refrain from having a CLI (I know cookietemple provide that @Zethson ?). Had a look at https://github.com/YosefLab/scvi-tools-skeleton @adamgayoso and it's really cool! could it borrow from that removing the scvi part?

[grst]

What I would really love to have (and I think cookietemple provides) is some sort of template sync.

Sphinx got updated and the docs broke again? -> Update the template and a PR is automatically submitted to all repositories that use the template.

[Zethson]

I would refrain from having a CLI (I know cookietemple provide that @Zethson ?). Had a look at https://github.com/YosefLab/scvi-tools-skeleton @adamgayoso and it's really cool! could it borrow from that removing the scvi part?

Yeah, no CLI. Nowadays I would remove the cli part from the cli-python package of cookietemple and rather provide a package only option, but back then we wrote more CLI packages ^_^

I'd rather write a template "from scratch" and cookiecutter it. Github templates are slightly annoying because one still has to rename lots of stuff etc. We can then include it into cookietemple for sync + bump-version support etc.

@ivirshup do you have a link to your cookiecutter template draft?

[grst]

let's do that, what should this template contain?

tests?
docs?
requirements?

I would also suggest to add some API structure (.tl, .pl, ...). If someone doesn't want to use it, they can remove it, but it would at least encourage to have similarly structured APIs.

[Zethson]

Think that the eternal battle here is how streamlined we want it to be. I personally and likely @grst like the very structured and homogeneous templates where you get forced into a common standard (hello Black formatter), but others like @ivirshup have argued more for loose and flexible structures.

We have to make a decision here and then we can quickly decide on a feature and technology set. Writing the template up is not a crazy amount of work, but we can spend days discussing what it has to look like :) (we love tech)

[grst]

Like @Zethson pointed out I'm on the "stricter" side of the spectrum. However, I would like to point out that an ecosystem package doesn't have to use the template. It is an offer, but if someone wants to do it differently they can still be part of the ecosystem as long as they use core IO and data structures.