search

gql-dart / ferry

Stream-based strongly typed GraphQL client for Dart
https://ferrygraphql.com/
MIT License
593 stars 113 forks source link

Building release throws Out of memory #560

Closed LiLatee closed 7 months ago

LiLatee commented 7 months ago

Hi! It's me again. ferry has generated about 2.2k files and 2.7mln lines of code. The whole lib directory has about 3.7k files. And now we are not able to build the app in release mode. It was running 3 hours (M2 pro, 32GB) and threw Out of memory error.

We are trying to switch from artemis, which generates just 2 files (about 400k lines). It was also problematic to work with such big files, but build_runner takes a few minutes as well as the building of the app in the release mode.

Any ideas for such a problem? 😞

knaeckeKami commented 7 months ago

Hi!

I'm sorry to hear that you have such problems migrating to ferry.

I assume this is due to built_value, which generates a lot of additional boilerplate and also collects all serializers in the serializers.g.dart file, which, I assume will consume a lot of memory when compiling.

I can think of a few avenues how to reduce the size of the generated code;

  1. small optimizations like avoiding genering a .var class for operations without variables
  2. Generalize reuse_fragments to reuse generated classes when multiple queries query the same fields of a type
  3. migrate away from built_value, and generate all code in ferry, like graphql_codegen. that's a lot of work and hard to get right (lots of corner cases in serialization), but will have the greatest effect and likely speed up code generation also by a lot (build_runner spends most time with built_value currently).

I do have a full time job as well, so I can't promise any quick solutions, unfortunately.

LiLatee commented 7 months ago

Thanks a lot for such a quick response 😊

All of the above points are just ideas to improve, not ready to use for now, right?

knaeckeKami commented 7 months ago

Yes.

I implemented a quick workaround that you can try out;

it's on branch workaround/non-serializer-reqs

It removes the xxxxReq classes from the serializers.gql.dart file.

This means, calling toJson() on them won't work anymore, and the offlineMutationLink which persists queries on offline mode to send them later will break because of this, but other than that everything should continue to work.

Maybe that's enough for compilation to succeed.

You can also try disabling when(), maybeWhen() methods on union types if you don't need them.

LiLatee commented 7 months ago

Probably can be closed for now. The issue has been explained here https://github.com/gql-dart/ferry/issues/558#issuecomment-1865189940