valkey-io / valkey

A flexible distributed key-value datastore that is optimized for caching and other realtime workloads.
https://valkey.io
Other
17.21k stars 648 forks source link

Investigate integrating with fast_float #1069

Open madolson opened 1 month ago

madolson commented 1 month ago

I've seen a couple of posts recently about how both dragonflyDB and Redis integrated with https://github.com/fastfloat/fast_float to improve parsing of doubles. We use strtod (exposed as getDoubleFromObject) in sorted set computation. The library was implemented in C++, but we should be able to extract out the needed code that solves our specific use case.

swaingotnochill commented 1 month ago

Hi, I would like to work on this one. Additionally, this will be my first time contributing to Valkey so it would be great if someone could review my work alongside.

PingXie commented 1 month ago

go for it, @swaingotnochill. You can tag us in your PR when it is ready for review.

swaingotnochill commented 1 month ago

Thanks. I will start working on it and add you guys when I have a draft PR ready.

swaingotnochill commented 1 month ago

@madolson @PingXie I looked at the fast_float, they have scripts to get it in a single header file to use. Then it's possible to create an interface to expose specific functions just like other dependencies in "deps" folder. And looking at Redis code, they have done something similar. Is the plan to replace the strtod in the sorted_set implementation Or all of the occurrences of strtod? (From the benchmarks of fast_float, its 5 times faster than strtod for parsing random floating numbers)

parthpatel commented 1 month ago

How would you expose an interface for a function with templating? Are you proposing simply initializing a version of that function for a type and exposing an interface for it?

template <typename T, typename UC = char,
          typename = FASTFLOAT_ENABLE_IF(is_supported_float_type<T>())>
FASTFLOAT_CONSTEXPR20 from_chars_result_t<UC>
from_chars(UC const *first, UC const *last, T &value,
           chars_format fmt = chars_format::general) noexcept;
swaingotnochill commented 1 month ago

How would you expose an interface for a function with templating? Are you proposing simply initializing a version of that function for a type and exposing an interface for it?

template <typename T, typename UC = char,
          typename = FASTFLOAT_ENABLE_IF(is_supported_float_type<T>())>
FASTFLOAT_CONSTEXPR20 from_chars_result_t<UC>
from_chars(UC const *first, UC const *last, T &value,
           chars_format fmt = chars_format::general) noexcept;

Something like this: https://stackoverflow.com/questions/2744181/how-to-call-c-function-from-c (look at the top answer). Basically, create a wrapper over fast_float to use it in C.

madolson commented 1 month ago

@madolson @PingXie I looked at the fast_float, they have scripts to get it in a single header file to use. Then it's possible to create an interface to expose specific functions just like other dependencies in "deps" folder. And looking at Redis code, they have done something similar. Is the plan to replace the strtod in the sorted_set implementation Or all of the occurrences of strtod? (From the benchmarks of fast_float, its 5 times faster than strtod for parsing random floating numbers)

I would say the numeric parsing in sorted_sets is the most important, but I see no reason to not replace all of the usage of strod.