search

josiah-wolf-oberholtzer / discograph

Social Graphing for the Discogs Database
MIT License
74 stars 11 forks source link

Optimize relation graphing. #69

Open josiah-wolf-oberholtzer opened 8 years ago

josiah-wolf-oberholtzer commented 8 years ago

Now that labels are being integrated into the graph, great care needs to be taken around labels which comprise thousands, even hundreds of thousands of relationships. Their count should be taken and cached, and used as a barrier against actually retrieving those relations.

josiah-wolf-oberholtzer commented 8 years ago

Still valid. The current bottle-neck - both in terms of complexity and memory usage - is paging.

Paging labels with 10k+ relations is incredibly slow, and probably O(n^2).