kbase / staging_service

MIT License
0 stars 9 forks source link

DATAUP-698: Fix ?sv files with missing header items getting strange key names #164

Closed MrCreosote closed 2 years ago

MrCreosote commented 2 years ago

Pandas would replace missing header items with weird names like Unnamed: 2_level_0. Pandas has enough quirks that it's not worth using it - simpler to just use a low level csv parser.

MrCreosote commented 2 years ago

Pandas was probably a mistake for ?sv parsing, where the standard library parser works just fine and Pandas has a number of quirks to work around, but abstracting the differences in handling xlsx and xls files really simplifies the code, so it's probably worth the quirks. I'm going to take a look and see if there's better ways to work around those.

sonarcloud[bot] commented 2 years ago

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 2 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication