Imaptest for very old dovecot?

[chloe-alverti]

Hello to all,

I am trying to profile dovecot with memory-mapped IO and I realized that a very old version of it, already uses it for reading e-mails (dovecot-1.0.0). I managed to install it, but now I do not know if and how I can compile imaptest with such an old version.

It fails early on as there is not dovecot-config file in the old dovecot. Even if this is fixed, is this possible to run imaptest with an old version?

Best Regards, Chloe

[siavasht]

Imaptest doesn't have to be running on the same machine as dovecot. You can compile and run it on a separate machine/VM with recent dovecot libraries and point it to the host running the old version of dovecot.

[sirainen]

Easiest is probably to:

• Compile new dovecot: ./configure ---without-shared-libs && make
• Compile imaptest: ./configure --with-dovecot=../dovecot && make

This should build you imaptest binary that is statically linked.

Anyway I remember testing read() vs mmap() a long time ago, and found that read() was taking a lot less CPU. I was assuming it's because page faults aren't so cheap.

[chloe-alverti]

Thank you very much!

Do you perhaps remember which version of dovecot you experimented with when you compared mmap and read? The truth is that I am testing dovecot over Optane Persistent Memory storage, and I was wondering if this reverses the trend that favors POSIX read/writes.

Thank you very much again! Best Regards, Chloe

On Tue, Nov 30, 2021 at 4:12 PM sirainen @.***> wrote:

Easiest is probably to:

• Compile new dovecot: ./configure ---without-shared-libs && make
• Compile imaptest: ./configure --with-dovecot=../dovecot && make

This should build you imaptest binary that is statically linked.

Anyway I remember testing read() vs mmap() a long time ago, and found that read() was taking a lot less CPU. I was assuming it's because page faults aren't so cheap.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/dovecot/imaptest/issues/14#issuecomment-982673465, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJRDLXQKDTAIH5K4YXZR7GLUOTLUTANCNFSM5JBXTAXQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[sirainen]

For emails it was very early in the beginning, v0.99.something. For index files I tested sometimes later, but still probably v1.0.x. For reading dovecot.index files there's:

/* How large index files to mmap() instead of reading to memory. */
#define MAIL_INDEX_MMAP_MIN_SIZE (1024*64)

I remember thinking that this was about a good size where mmap() overhead wasn't too large compared to the benefits of reduced memory usage.

[chloe-alverti]

Thank you very much for all the info and I am sorry for the late reply!

I did manage to run stress-tests on a very old version of dovecot which uses mmio to read mail data.

I would like to kindly ask if you can recommend a meaningful experimentation setup that could realistically stress dovecot performance?

Moreover, is there a reason that in the function that fetches the body of an e-mail you scan it in a char by char basis to figure out the number of lines? This seems to be very expensive.

In any case, thank you very very much for all your help! It has been precious!

Best Regards, Chloe

On Tue, Nov 30, 2021 at 4:36 PM sirainen @.***> wrote:

For emails it was very early in the beginning, v0.99.something. For index files I tested sometimes later, but still probably v1.0.x. For reading dovecot.index files there's:

/ How large index files to mmap() instead of reading to memory. /

define MAIL_INDEX_MMAP_MIN_SIZE (1024*64)

I remember thinking that this was about a good size where mmap() overhead wasn't too large compared to the benefits of reduced memory usage.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/dovecot/imaptest/issues/14#issuecomment-982695333, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJRDLXSYCG75B5ICRCQERJDUOTONTANCNFSM5JBXTAXQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[sirainen]

The "realistic stress test" is rather difficult, but the "profile" parameter is kind of attempting that. Looks like it's missing documentation. But basically run "imaptest profile=profile.conf clients=100"

Which function counts numbers of lines? In Dovecot or imaptest? In Dovecot there is message-size.c which could probably be somewhat more optimized (with i_memcspn()), but we haven't actually seen it use much CPU. It's mainly intended to calculate the message sizes correctly, with CRLF linefeeds even if the input has only LFs.

[slusarz]

Closing, as there is nothing left to do in this ticket. To the extent Timo was commenting on the need for "profile" usage documentation, that has been added.