search

jorenham / scipy-stubs

Typing Stubs for SciPy
https://pypi.org/project/scipy-stubs/
BSD 3-Clause "New" or "Revised" License
6 stars 3 forks source link

The great `types-scipy-sparse` merger 🎉 #129

Open jorenham opened 4 hours ago

jorenham commented 4 hours ago

@BarakKatzir has been developing the types-scipy-sparse stub package for scipy.sparse. A large portion appears to more complete that the scipy.sparse stubbs in scipy-stubs.

I have talked with @BarakKatzir about joining forces, and we both agreed that gradually merging types-scipy-sparse into scipy-stubs is not only a good idea, but also a feasible one.

@BarakKatzir, how about we could use this issue as central planning on how we can go about this? So e.g. how we can "chop it up" into PR's, and in which order we do so.

BarakKatzir commented 4 hours ago

Good idea. Currently we talked about the first PR including: 1.Moving a single sparse array type (csc / csc etc.) from types-scipy-sparse package to scipy-stubs. Add inheritance from _spbase class (since in my package _spbase isn't type-annotated and you mentioned that despite being private it is used by users).

  1. I probably have to add generics to _spbase as well, since there are some methods which need it (e.g., real). This might also be in the form of generated code.
  2. Regarding the code generation - I'll start by copying my tool (just for the relevant sparse type in the PR). It's a rough tool, but it's a good starting point IMO.

After the first PR we'll have a better idea of issues in the merger and we can plan ahead.

jorenham commented 3 hours ago
  1. I probably have to add generics to _spbase as well, since there are some methods which need it (e.g., real). This might also be in the form of generated code.

Do you think that it would help if we defer the Generic bits, and re-introduce them once we have the fundamentals in place? You used __ceil__ as example to show how generics lead to unavoidable code duplication. From that perspective, omitting the generics would be like compressing a file before you transfer it, and decompressing it when all files are received.

  1. Regarding the code generation - I'll start by copying my tool (just for the relevant sparse type in the PR). It's a rough tool, but it's a good starting point IMO.

The codegen/ directory seems like the right place for that. But I'm not really sure on what the best approach for integrating this in the CI, and perhaps also pre-commit, would be. The current codegen stuff is mostly meant for one-time only use, and is therefore not part of the CI.

But if we decide to defer the generics, do you still think it would be needed? Because as I mentioned in #47, I plan to integrate unpy in scipy-stubs soon, which also does code-generation. So I'm sure you can imagine that having two codegens together could become a bit messy, if not done right.

I think that the ideal scenario would be to achieve the same DRY result as your template-based approach through unpy, because unpy code is (and probably always will be) syntactically valid Python. This means that IDE's still understand it, and all of our linters and type-checkers can also be used in the same way.

After the first PR we'll have a better idea of issues in the merger and we can plan ahead.

Yea that makes sense. Sometimes I forget that premature optimization is the root of all evil.