Optimize file copy in sklearn data table file creation

KhiopsML / khiops-python

The Python library of the Khiops AutoML suite

https://khiops.org

BSD 3-Clause Clear License

10 stars 1 forks source link

Optimize file copy in sklearn data table file creation #219

Open folmos-at-orange opened 3 months ago

folmos-at-orange commented 3 months ago

Description

Currently the Pandas dataframes and Numpy arrays are copied in memory before being written to a file to be consumed by Khiops. This not optimal memory wise.

Questions/Ideas

Implementing the "dump" to CSV manually would solve this.