amazon-ion / ion-python

A Python implementation of Amazon Ion.
https://amazon-ion.github.io/ion-docs/
Apache License 2.0
253 stars 50 forks source link

Enhance ionc_write_big_int method for optimized handling of large integers. #321

Closed linlin-s closed 6 months ago

linlin-s commented 6 months ago

Issue #, if available:

N/A

Description of changes:

The updated method ionc_write_big_int initially attempts to convert the Python object to a Clong long using PyLong_AsLongLongAndOverflow. If the integer fits within a 64-bit range and no error occurs, it proceeds to write the integer directly using ion_writer_write_int64. For integers larger than 64-bit, it will fallback to string conversion. This PR also replaced method PyObject_CallMethod(obj, "__str__", NULL) with PyObject_Str(obj), this also improve the performance.

Here are the benchmark results from benchmarking ion-python writing using service_log_legacy: There is 14.74% performance improvement from the change.

file_size(B) time_min(ns) time_mean(ns) memory_usage_peak(B)
Before 21270622 2943214116.60 3002532105.01 42637008
After 21270622 2520604525.00 2559838239.71 42647684

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.