Optimize dataclasses with __slots__

[nat-n]

Adding __slots__ to a class improves memory usage and performance of attribute access, at the expense of not being able to set arbitrary attributes on the resulting objects.

This project demonstrates how dataclasses can be augmented to use slots, whilst still having class attributes set (normally adding the slots attribute to a class definition would cause an error with dataclassess that declare default values).

Making generated dataclasses use slots in this way would be a small breaking change, though I don't see how it could be too inconvenient to work with, and I expect the performance improvement would be worthwhile.

[Gobot1234]

Gonna give this one a shot 👍

[Gobot1234]

>>> @dataclass
... class Data:
...     t: int
...     m: str
...     n: float
... 
>>> Data(t=3, m=3, n=3).__sizeof__()
32
>>> @dataslots.with_slots
... @dataclass
... class Data:
...     t: int
...     m: str
...     n: float
... 
>>> Data(t=3, m=3, n=3).__sizeof__()
40

Really not seeing much in the ways of benefits here.

Attribute access is marginally faster as is instantiation but this doesn't seem like a worthwhile trade off against the extra overhead and memory size.

I might be seriously missing something or it might be Python 3.9 doing stuff not too sure though.

[nat-n]

I'm curious did you find a measurable speed improvement? Is the memory penalty constant or proportional to the number of fields? (or does it turn to a benefit with enough fields?)

Admittedly I've not done enough investigation to validate this, but my thinking was the if the technique from dataslots were implemented directly in the Message class then it should give the most benefit.

[Gobot1234]

The there didn't appear to be a point at which the un-slotted class used more memory. The speed improvement wasn't noteworthy. Might have to test some more however.

[Gobot1234]

Actually I take this back it appears to be working now 🤔

[Gobot1234]

All tests were ran with timeit.repeat and a sped up custom implementation of dataslots.

So for a single slotted class attribute access 0.11287586920000003s vs unslotted 0.1498976172s

Init appears to be slightly slower slotted 25.6038842112s vs 22.337924385599997s.

Overhead is relatively low at 3.2458480566e-05s per class per decorator directly using the implementation of the above library finishes in 3.7460388750999996e-05s

Memory usage temperamental which I think is caused by the use of betterproto.int32_field(0) etc. So I think it's safe to say the previous statements still apply,

[Gobot1234]

The size issues might be due to using it on instances this sure is very strange.

[Gobot1234]

I'm gonna give a Message metaclass a go for the best performance as you suggested.