Reduce compilation time by using large-records library

Haskell-OpenAPI-Code-Generator / Haskell-OpenAPI-Client-Code-Generator

Generate Haskell client code from an OpenAPI 3 specification

46 stars 19 forks source link

Reduce compilation time by using large-records library #79

Closed thomasbach-dev closed 6 months ago

thomasbach-dev commented 2 years ago

The compilation time of generated client libraries is way too long. There is some work from Edsko de Vries from Well-Typed on this: https://well-typed.com/blog/2021/08/large-records/.

The investigations lead to the large-records package. I have not used the package yet, but it seems like it could give us a dramatic improvement in compilation times.

joel-bach commented 2 years ago

Hey there, I have two lines of thought regarding this:

Some OpenAPI specs are huge (like the one from Stripe) and I am not sure if there is actually a way to make compilation really fast since we will always need to generate a huge amount of types etc. Therefore, the way to improve compilation time ATM is to use the flags --operation-to-generate and --white-listed-schema (or --opaque-schema) to specify what you will actually use because usually you do not need everything in an OpenAPI spec.
Thank you for the hint regarding the large records package, I will have a look but I am not sure if I have the time to actually change the implementation, so feel free to submit a PR. Were you able to pin-point the long compilation time being caused by large records? Could you tell me which OpenAPI spec you are using (because not all OpenAPI specs have large records)?

thomasbach-dev commented 2 years ago

Hi @joelfisch,

Regarding your first point: yes, it is probably best to generate client bindings only for things you really need. But from an end-user perspective there are quite a few hoops to jump. Also, wouldn't it be nice to just generate bindings and be able to publish on Hackage?

ATM I'm not as enthusiastic and optimistic as I was, when I opened the feature request -- the reason for slow compilation times is probably manifold. I am looking at output files from -ddump-simpl and in the Types submodules I see some very huge files. These files are the ones where large records are defined. So, yes, I guess there will be an improvement in compilation time using the large-records library, not sure how much impact it can have, though.

I'll do a quick and dirty implementation to see possible improvements.

joel-bach commented 6 months ago

I'll close this as it has gone stale, let me know if there are any updates on it.