This is a feature as it updates the algorithm arguments
What does this do?
Changed the internal representation of algorithms and datasets within the service from byte slices to file paths, writing received data directly to temp files. This modification allows for handling potentially large data sets without the need to load them entirely into memory, improving the memory efficiency and scalability of the service. Additionally, it aligns the call signature of external algorithms with the new approach, updating documentation and examples accordingly. Updated the linear regression example for consistency with the new data handling process.
Resolves issues with memory bloat when processing large datasets.
What type of PR is this?
This is a feature as it updates the algorithm arguments
What does this do?
Changed the internal representation of algorithms and datasets within the service from byte slices to file paths, writing received data directly to temp files. This modification allows for handling potentially large data sets without the need to load them entirely into memory, improving the memory efficiency and scalability of the service. Additionally, it aligns the call signature of external algorithms with the new approach, updating documentation and examples accordingly. Updated the linear regression example for consistency with the new data handling process.
Resolves issues with memory bloat when processing large datasets.
Which issue(s) does this PR fix/relate to?
Have you included tests for your changes?
No
Did you document any new/modified feature?
Yes, I have updates docs
Notes
to be merged after #123