Use 3rd-party ddbcereal library for deserialize/serialize.

[JustinTArthur]

Figured I'd put this out here as an option. I've been working on a serializer/deserializer lib to supplement my use of aiobotocore, but it might make sense for this project too. It should be as fast or faster than this one.

Overall, it reduces/simplifies the code, but the initialization of Client is a tad uglier due to ddbcereal needing a pre-constructed Deserializer and Client being frozen. It could be made pretty by unfreezing it or alternatively, making a seperate client-spawning function that passes the Deserializer to the Client constructor.

[ojii]

It should be as fast or faster than this one.

I see you added a benchmark for this, what numbers do you get on your machine?

[JustinTArthur]

On cpython 3.8.5 with ddbcereal 2.1.0, I get

$ pyperf compare_to boto3.json aiodynamo.json ddbcereal.json
Mean +- std dev: [boto3] 1.85 sec +- 0.10 sec -> [aiodynamo] 1.01 sec +- 0.09 sec: 1.83x faster
Mean +- std dev: [boto3] 1.85 sec +- 0.10 sec -> [ddbcereal] 997 ms +- 98 ms: 1.86x faster

With cpython 3.9.4:

$ pyperf compare_to boto3-394.json aiodynamo-394.json ddbcereal-394.json
Mean +- std dev: [boto3-394] 1.84 sec +- 0.10 sec -> [aiodynamo-394] 1.01 sec +- 0.12 sec: 1.83x faster
Mean +- std dev: [boto3-394] 1.84 sec +- 0.10 sec -> [ddbcereal-394] 974 ms +- 111 ms: 1.89x faster

Next release will speed up the base64 conversion a tiny amount.

The benchmarks don't cover construction time. Constructing a ddbcereal deserializer is expensive due to needing to set things up for many possible configurations. Current aiodynamo deserializer funcs only do things one way from the get-go and are built to bytecode at first import instead of needing to construct an object with these decisions. So the tradeoff is a slower Client creation time if you go with ddbcereal.

I won't take offense if you decide against this direction at the moment. If you do want it, I can resolve the couple remaining mypy complaints.

[ojii]

It looks like at the moment, the speed gains are negligible (comparing aiodynamo to ddbcereal), so for now I'd rather not add this dependency.

[JustinTArthur]

No problem, thanks for considering. Maybe a future version will be a better fit.

[dimaqq]

@JustinTArthur perhaps we can learn something from your source code and you from ours?