search

zavtech / morpheus-core

The foundational library of the Morpheus data science framework
Apache License 2.0
238 stars 22 forks source link

How to exclude specific row? #104

Open bayraktugrul opened 4 years ago

bayraktugrul commented 4 years ago

I want to exclude first and last rows from my CSV file before I start to process. Is there any options method for it? There are exclude and include column methods. I could not find for rows. Any idea about it?

Zavster commented 4 years ago

You can set a row predicate on the CsvSourceOptions that receives a String[] with the contents of each row - not sure if that would work. It might be worth augmenting the predicate to receive the row number as well as the String[] contents...

bayraktugrul commented 4 years ago

Contents of first and last rows are different for each CSV file. Files will be uploaded by clients. I guess that does not work for this situation. Actually skipping some specific rows are an expected feature. Looks like solution is removing rows before giving as an input to Morpheus.

Zavster commented 4 years ago

I'll look to add an enhancement to facilitate this

bayraktugrul commented 4 years ago

That would be great. Thanks!