Add typing hints

[rbuffat]

Recently it seems that more and more Python libraries include typing hints. Personally, I'm a fan of type hints while developing. Are there any plans to add typing hints to Fiona?

I'm not 100% sure what the best way to deal with the dynamic nature of the dataset schemas is. Maybe the implementation of static typing of psycopg3 could be used as inspiration: https://www.psycopg.org/psycopg3/docs/advanced/typing.html

[sgillies]

There wasn't much point in doing this before the Feature, Geometry, Properties classes. They will help quite a bit. Maybe a Schema class would, too?

[StefanBrand]

In a naive attempt to model the parameters to fiona.open (v1.8) we created this:

from typing import TypedDict

class FionaBaseDict(TypedDict):
    properties: dict[str, Any]

class FionaSchemaWithoutGeometry(FionaBaseDict):
    geometry: None

class FionaSchemaWithGeometry(FionaBaseDict):
    geometry: str

FionaSchema = Union[FionaSchemaWithoutGeometry, FionaSchemaWithGeometry]

class RecordGeometry(TypedDict):
    type: str
    coordinates: tuple

class RecordWithoutGeometry(FionaBaseDict):
    geometry: None

class RecordWithGeometry(FionaBaseDict):
    geometry: RecordGeometry

Record = Union[RecordWithoutGeometry, RecordWithGeometry]

I'm not yet familiar with the new Feature, Geometry, Properties classes. That would probably simplify things. In any case I would be interested to help get type hints into fiona, either by sharing opinion or contributing code.

[sgillies]

@StefanBrand @rbuffat what would you think about collaborating on stubs for 1.9 and then possibly moving the stubs into the code at 2.0?

[rbuffat]

@sgillies I can have a go at it but probably during the next few weeks I don't have too much time available, thus it can take a while. I will also need to dive a bit deeper into the Python typing system and maybe look at how other projects are implementing dynamic types (Sqlalchemy or pydantic come to my mind here, perhaps it would also be worth asking somebody familiar with mypy/pyright for some inputs). Being able to static type check writerecords() or the collection iterator would probably be the most useful thing, but I guess also the hardest to implement.

[StefanBrand]

but probably during the next few weeks I don't have too much time available

Maybe that's not too bad in this case. I myself should have more availability starting from (mid?) April. I haven't much worked with type stubs instead putting type hints directly into .py scripts. It will be an interesting exercise. If time permits I will look into type stub implementation.

[StefanBrand]

Interesting opinion piece by @Tinche, arguing that "Python is two languages now": untyped Python (used for infrastructure code) and typed Python (used in business logic). In order to combine the two, he argues that (bold by me)

we need infrastructure code not to be type-hinted internally, but to provide type-hinted interfaces at the code boundaries. This is exactly where the ecosystem is going, with noteworthy examples being SQLAlchemy 2.0 and a new generation of web frameworks like FastAPI.

@Tinche's argument is in line with @rbuffat's comment:

Being able to static type check writerecords() or the collection iterator would probably be the most useful thing, but I guess also the hardest to implement.

---

On another note, @constantinius and me have submitted a lightning talk proposal to FOSS4G 2023: Adding static type hints to fiona :: FOSS4G 2023 :: pretalx (currently in the community voting process). If it is accepted and you, @rbuffat, and any other contributors will attend FOSS4G, I will also be happy to add you to the list of speakers. :slightly_smiling_face: We are excited to get the ball rolling and make type hints happening. :muscle:

[StefanBrand]

Almost two months have passed. In the meanwhile I have setup a wiki page in our fork and added commands to generate type stubs using several tools. The resulting stubs can be found in branches in our fork.

Here are all instructions and links: https://github.com/EOX-A/Fiona/wiki/Generating-Python-Stubs

Since using tools to generate type stubs resulted in a big amount of type annotations, I welcome any reviews and comments. The initial focus should be on writerecords() and the collection iterator, as mentioned by @rbuffat above:

Being able to static type check writerecords() or the collection iterator would probably be the most useful thing, [...]

PS.: The lightning talk proposal for FOSS4G 2023 was accepted. So I really need to put some flesh on this. :grin:

[StefanBrand]

We are looking for initial feedback on the #1259 draft. Please give your comments. :ok_hand:

[StefanBrand]

Cross reference: https://github.com/jazzband/geojson/issues/167