Data::ULID::XS

[bbrtj]

Hey there!

This issue is about something we may consider in the future.

As you know I have worked hard to make the ulid generation faster, and that's because I want to generate them for anything and everything in my performance-hungry project. Sadly I think we are at Perl limits already and ~130k per second may not cut it.

For this reason I am working on partial XS implementation. I have just uploaded Data::ULID::XS - this is very much work in progress, but I already have working pure C base32 encoding. Even with this little XS, ulid can already compete with Crypt::Misc::random_v4uuid:

            Rate    ulid ulid_xs    uuid
ulid    126879/s      --    -60%    -61%
ulid_xs 320655/s    153%      --     -2%
uuid    327762/s    158%      2%      --

I think I can squeeze even more (but not as much) performance out of it when I reimplement binary_ulid in XS.

This module has the same interface as Data::ULID, but will only work for ulid and binary_ulid called without arguments - in other cases it just calls Data::ULID::ulid or Data::ULID::binary_ulid instead. I don't think anything else other than generation needs that much performance. It will also only work for 64 bit systems.

The actual question is - after I fully implement it and make it stable, would you like to include my XS code in base Data::ULID distribution (making Data::ULID::XS obsolete), or keep Data::ULID implemented as-is and let XS implementation coexist?

Another solution would be to make Data::ULID use backends - like JSON::MaybeXS - but the issue I have with this is that Data::ULID already depends on CryptX, which requires XS compilation, so there's technically no reason for it to just ship Perl when XS solution exists (other than Perl code being more stable).

There's no rush, will take me some time to make it stable. Also, I'm not exactly sure if we should do it, because of the limitations mentioned above. I'm perfectly happy with having a second module with XS implementation, but that's not how XS implementations on CPAN usually work.

[bbrtj]

A teaser: these are the results I get on another box after implementing binary_ulid in XS:

                Rate   perl_text perl_binary     xs_text   xs_binary
perl_text   100748/s          --        -68%        -83%        -88%
perl_binary 316649/s        214%          --        -48%        -63%
xs_text     606233/s        502%         91%          --        -29%
xs_binary   848745/s        742%        168%         40%          --

[bbrtj]

After some thought I don't think it's a good idea. I managed to get very good results in my XS implementation, but it contains some hackery which I wouldn't want to be a part of the main implementation. Additionally, I wrote Types::ULID, so I made it work sort of like JSON::MaybeXS does... detecting XS backend if available.

Closing.