search

Kemsekov / GraphSharp

GraphSharp is a tool to study and use knowledge of graph theory in C#!
MIT License
39 stars 5 forks source link

Lazy load data #78

Open emdadgar2 opened 6 months ago

emdadgar2 commented 6 months ago

Hi, I have a +300M nodes with +1G edges, loading all data from database to process with your great project, needs too resource and take long time

Is there any way to Lazy load data from data source as needed? Thanks

Kemsekov commented 6 months ago

This is interesting question.

I honestly don't even know what it means to implement such lazy loading in the first place. 😅

You mean to create some graph implementation that known where to take nodes and edges (aka knows how to load them from DB to memory), but does it only once particular node or edge is actually accessed, meanwhile keeping illusion of all data being loaded at once?

There is currently no such implementation.

But i believe this is pretty simple to do.

You may look into interfaces INodeSource and IEdgeSource, implement them with such lazy techniques and then use this sources with graph G.SetSources.

Maybe I will work on this when I will have a bit of free time 😅