search

jTaylo99 / kolumnextractor

A data extractor designed around building your requirements and easily extracting and verifying your data.
4 stars 8 forks source link

Have an option to replace blank values with a default value. #25

Open jTaylo99 opened 1 year ago

jTaylo99 commented 1 year ago

Data sources often have gaps in their data so we should have the option to handle blank values by giving them a default value. This however should be optional and have to be manually decided by the user if they'd allow this.

Qoyyuum commented 1 year ago

Typically, data is handled on a column basis. If following how pandas deal with blank values, the default is NA for Not Applicable. And if the user wants to fill NAs with something else, they can call a function to either back fill or forward fill or provide a default value. Are you looking to implement the same functionality here @jTaylo99 ?

kimessenbergfonterra commented 1 year ago

Can assign to me?

jTaylo99 commented 1 year ago

That's essentially how we'd like it yeah. either that or being strict and saying either we allow N/A or not and the validator raises an exception.