Azure / azure-kusto-python

Kusto client libraries for Python
MIT License
183 stars 109 forks source link

Enhance Dataframe Conversion to Support All Float Data Types (float16, float32, float64) in `dataframe_from_result_table()` #512

Closed thlautenschlaeger closed 3 months ago

thlautenschlaeger commented 10 months ago

Is your feature request related to a problem? Please describe. Currently, the dataframe_from_result_table function in the azure-kusto-python library converts float data types to float64 by default. However, there are scenarios where using float16 or float32 might be more appropriate, such as when working with large datasets where memory efficiency is crucial, or when interfacing with systems that primarily use float32 data types.

Describe the solution you'd like I propose extending the dataframe_from_result_table() function to allow the user to specify the float data type (float16, float32, float64). This can be achieved by adding an optional parameter, such as float_dtype, with a default value of float64. The function would then use this parameter to determine the data type for floating-point columns in the resulting DataFrame.

Describe alternatives you've considered An alternative would be to create a separate function specifically for float32 conversion, but this approach seems redundant and less efficient. Allowing the user to specify the float data type in the existing function provides a more elegant and flexible solution.

Additional context This enhancement would increase the usability of the azure-kusto-python library for a broader range of applications, particularly those dealing with large datasets or specific system requirements. It also aligns with the principles of flexibility and user choice in software design. I have already implemented a version of this feature in my local environment and found it to be extremely useful for my use cases.

thlautenschlaeger commented 10 months ago

I'd like to contribute this feature.

AsafMah commented 10 months ago

Please do, let us know if you have any questions

AsafMah commented 6 months ago

Are you still interested in implementing this?

AsafMah commented 3 months ago

Added in v4.5.0.

Thanks @mayamarom10 !