arnaudroger / SimpleFlatMapper

Fast and Easy mapping from database and csv to POJO. A java micro ORM, lightweight alternative to iBatis and Hibernate. Fast Csv Parser and Csv Mapper
http://simpleflatmapper.org
MIT License
435 stars 76 forks source link

MalformedInputException when parsing csv file with umlauts #766

Open ryanthara opened 2 years ago

ryanthara commented 2 years ago

I try to parse a csv file from a public geodata server Fixpunkte_BS.csv that contains umlauts like 'ä' and 'ö' in the first line of the file.

Therefore I use the callback approach.

// Callback
try {
    csv = CsvParser
            .separator(separator)
            .forEach(file2Read.toFile(), new ListCollector<>()).getList();
    return true;
} catch (IOException e) {
    LOGGER.error("Comma separated values file '{}' could not be read.", file2Read.toString());
}

Example:

Punktnummer;Punktkategorie;E (LV95);N (LV95);Höhe (LN02);Lage-Genauigkeit;Lage-Zuverlässigkeit;Höhen-Genauigkeit;Höhen-Zuverlässigkeit;Vermarkungsart;Punktart

It is quite equal if the encoding is 'UTF-8' or 'Windows 1252'. It always ends in the

java.nio.charset.MalformedInputException: Input length = 1.

Did I miss something to avoid this?