By using mmap() rather than read()ing into a buffer, we can avoid the kernel having to copy pages around, and it's also easy to hint that we are going to access the whole file.
This alone, on a m5n.16xlarge EC2 instance doing a reposync of the Amazon Linux 2023 x86-64 repository cuts around 2.5 seconds from the duration of a reposync, as well as about .5 seconds off system time.
When combined with the avoiding libc buffered IO patch, we gain an extra 0.5 seconds of elapsed time.
For reference, my benchmarking has been done on a m5n.16xlarge EC2 instance to the in-region S3 buckets as well as to the CDN repositories. That instance type has 256GB memory, a 75Gbit network connection, and is a 64 core Cascade Lake system. The root volume is a 256GB gp3 EBS volume with 500MB/sec of IO and 3000 IOPs.
The background of this is that a lot of EC2 instances don't live that long (relatively speaking), and never install RPMs except on launch - so all the time-to-install RPMs is time spent scaling up a system that could be better served by running the customer workload.
By using mmap() rather than read()ing into a buffer, we can avoid the kernel having to copy pages around, and it's also easy to hint that we are going to access the whole file.
This alone, on a m5n.16xlarge EC2 instance doing a reposync of the Amazon Linux 2023 x86-64 repository cuts around 2.5 seconds from the duration of a reposync, as well as about .5 seconds off system time.
When combined with the avoiding libc buffered IO patch, we gain an extra 0.5 seconds of elapsed time.
For reference, my benchmarking has been done on a
m5n.16xlargeEC2 instance to the in-region S3 buckets as well as to the CDN repositories. That instance type has 256GB memory, a 75Gbit network connection, and is a 64 core Cascade Lake system. The root volume is a 256GB gp3 EBS volume with 500MB/sec of IO and 3000 IOPs.The background of this is that a lot of EC2 instances don't live that long (relatively speaking), and never install RPMs except on launch - so all the time-to-install RPMs is time spent scaling up a system that could be better served by running the customer workload.
Goes well when paired with https://github.com/rpm-software-management/librepo/pull/294 and https://github.com/rpm-software-management/librepo/pull/295