Is this behavior intended? Can't we toggle it providing this as additional parameter to the function?
Expected behavior:
In my case there are a lot of rows in my DataFrame that contains NaNs value but are just fine from the application logic perspective, and I would just like to store my entire DataFrame to dolt without dropping anything.
PS: even in case the drop is needed for any reason, it would make sense to print a warning to the user. In my case it was successfully concluding the write with no errors, but the fact that 50% of the content was silently dropped :)
Hi there,
After a lot of debugging I just noticed that the
write_pandaswrapper is silently discarding rows that contain NaN values.Here is the point where this happens.
Is this behavior intended? Can't we toggle it providing this as additional parameter to the function?
Expected behavior: In my case there are a lot of rows in my DataFrame that contains NaNs value but are just fine from the application logic perspective, and I would just like to store my entire DataFrame to
doltwithout dropping anything.PS: even in case the drop is needed for any reason, it would make sense to print a warning to the user. In my case it was successfully concluding the write with no errors, but the fact that 50% of the content was silently dropped :)