Handling Larger size datasets

SelfExplainML / PiML-Toolbox

PiML (Python Interpretable Machine Learning) toolbox for model development & diagnostics

https://selfexplainml.github.io/PiML-Toolbox

Apache License 2.0

931 stars 111 forks source link

Handling Larger size datasets #45

Open yigitcancomlek opened 11 months ago

yigitcancomlek commented 11 months ago

Hello!

Thank you very much for providing this tool!

Is it possible to use any csv files that are more than 10MB in size? Is there a work around to handle larger datafiles (~500MB). Thank you very much in advance for your help!

ZebinYang commented 11 months ago

Hi @yigitcancomlek，

Sure, the 10MB limit is due to the widgets interface.

PiML can handle larger data by pandas dataframe, see the user guide in https://selfexplainml.github.io/PiML-Toolbox/_build/html/guides/data/data_load.html#external-dataset.