Concern from library community regarding scaling
We are talking about order of 10M/1000M rows.
Let’s do some profiling
7/20
Set up extremely basic profiling tools
Generate N random arks
Pull a random sample of 50 arks
Fetch them and see the mean response time
With 10K arks in the database, the response is 300ms
With 20K arks in the database, the response is still 300ms
With 1.1M arks in the database, response is still 300ms idk guys lol
With 10M rows the response goes down to 150ms LOL
Disk size 2271 Mb
Concern from library community regarding scaling We are talking about order of 10M/1000M rows.
Let’s do some profiling
7/20 Set up extremely basic profiling tools Generate N random arks Pull a random sample of 50 arks Fetch them and see the mean response time With 10K arks in the database, the response is 300ms With 20K arks in the database, the response is still 300ms With 1.1M arks in the database, response is still 300ms idk guys lol With 10M rows the response goes down to 150ms LOL Disk size 2271 Mb