search

gql-dart / ferry

Stream-based strongly typed GraphQL client for Dart
https://ferrygraphql.com/
MIT License
602 stars 116 forks source link

Last lists of data causing everything to freeze #434

Open lloydrichards opened 2 years ago

lloydrichards commented 2 years ago

I'm in the process of building a data dashboard for IoT data from our GraphQL endpoint and have been running into several limitations . The most noticeable is that when i query a very large dataset (17,000 rows) then the query causes the whole app to freeze and force me to close the emulator.

Normally i'm not querying all the data, but when something goes wrong and this happens i would expect there to be some error handeling or exemption thrown rather than causing everything to lock up. I don't know the limitations of dart when it comes to list sizes, but JS handles the same query without any trouble so i think the issue might be a bottleneck in Ferry somewhere 🤷

My current solution is to limit the query to less than a 1000 rows, but this could be an issue for larger data visualizations.

knaeckeKami commented 2 years ago

The bottleneck there is likely normalization, which can be expensive for very large datasets.

If you do not need normalization, you can turn it off as described here.

Or you can disable caching for these request (setting the FetchPolicy to .NoCache), since normalization is only done for the cache. The normalized cache is not really suited for large amounts of data.