Parsing mistake with this sample

[AndydeCleyre]

Hi, thanks for the project! The rainbow column colorizing is great.

I noticed 0.6.1 had trouble parsing the following sample:

Year,Agriculture,Architecture,Art and Performance,Biology,Business,Communications and Journalism,Computer Science,Education,Engineering,English,Foreign Languages,Health Professions,Math and Statistics,Physical Sciences,Psychology,Public Administration,Social Sciences and History
1970,4.22979798,11.92100539,59.7,29.08836297,9.064438975,35.3,13.6,74.53532758,0.8,65.57092343,73.8,77.1,38,13.8,44.4,68.4,36.8
1980,30.75938956,28.08038075,63.4,43.99925716,36.76572529,54.7,32.5,74.98103152,10.3,65.28413007,74.1,83.5,42.8,24.6,65.1,74.6,44.2
1990,32.70344407,40.82404662,62.6,50.81809432,47.20085084,60.8,29.4,78.86685859,14.1,66.92190193,71.2,83.9,47.3,31.6,72.6,77.6,45.1
2000,45.05776637,40.02358491,59.2,59.38985737,49.80361649,61.9,27.7,76.69214284,18.4,68.36599498,70.9,83.5,48.2,41,77.5,81.1,51.8
2010,48.73004227,42.06672091,61.3,59.01025521,48.75798769,62.5,17.6,79.61862451,17.2,67.92810557,69,85,43.1,40.2,77,81.7,49.3

The sample passes csvlint, seems to be parsed ok by csvlens, and is converted to the following JSON by csvkit's csvjson:

Long JSON

```json [ { "Year": 1970, "Agriculture": 4.22979798, "Architecture": 11.92100539, "Art and Performance": 59.7, "Biology": 29.08836297, "Business": 9.064438975, "Communications and Journalism": 35.3, "Computer Science": 13.6, "Education": 74.53532758, "Engineering": 0.8, "English": 65.57092343, "Foreign Languages": 73.8, "Health Professions": 77.1, "Math and Statistics": 38, "Physical Sciences": 13.8, "Psychology": 44.4, "Public Administration": 68.4, "Social Sciences and History": 36.8 }, { "Year": 1980, "Agriculture": 30.75938956, "Architecture": 28.08038075, "Art and Performance": 63.4, "Biology": 43.99925716, "Business": 36.76572529, "Communications and Journalism": 54.7, "Computer Science": 32.5, "Education": 74.98103152, "Engineering": 10.3, "English": 65.28413007, "Foreign Languages": 74.1, "Health Professions": 83.5, "Math and Statistics": 42.8, "Physical Sciences": 24.6, "Psychology": 65.1, "Public Administration": 74.6, "Social Sciences and History": 44.2 }, { "Year": 1990, "Agriculture": 32.70344407, "Architecture": 40.82404662, "Art and Performance": 62.6, "Biology": 50.81809432, "Business": 47.20085084, "Communications and Journalism": 60.8, "Computer Science": 29.4, "Education": 78.86685859, "Engineering": 14.1, "English": 66.92190193, "Foreign Languages": 71.2, "Health Professions": 83.9, "Math and Statistics": 47.3, "Physical Sciences": 31.6, "Psychology": 72.6, "Public Administration": 77.6, "Social Sciences and History": 45.1 }, { "Year": 2000, "Agriculture": 45.05776637, "Architecture": 40.02358491, "Art and Performance": 59.2, "Biology": 59.38985737, "Business": 49.80361649, "Communications and Journalism": 61.9, "Computer Science": 27.7, "Education": 76.69214284, "Engineering": 18.4, "English": 68.36599498, "Foreign Languages": 70.9, "Health Professions": 83.5, "Math and Statistics": 48.2, "Physical Sciences": 41, "Psychology": 77.5, "Public Administration": 81.1, "Social Sciences and History": 51.8 }, { "Year": 2010, "Agriculture": 48.73004227, "Architecture": 42.06672091, "Art and Performance": 61.3, "Biology": 59.01025521, "Business": 48.75798769, "Communications and Journalism": 62.5, "Computer Science": 17.6, "Education": 79.61862451, "Engineering": 17.2, "English": 67.92810557, "Foreign Languages": 69, "Health Professions": 85, "Math and Statistics": 43.1, "Physical Sciences": 40.2, "Psychology": 77, "Public Administration": 81.7, "Social Sciences and History": 49.3 } ] ```

---

The problem is apparent by the empty gaps in the displayed data:

[shshemi]

Hello, Thank you for the bug report. I will fix it in the next version. Meanwhile, I suggest you use --infer-schema full to open the file.

[shshemi]

The bug should be fixed in version 0.6.2. Would kindly you let me know if the problem continues?

[AndydeCleyre]

Thanks! I'll try again later, but trying to upgrade with cargo for now is failing to compile dep polars-arrow.

[shshemi]

Interesting!

Works fine on my machine

Have you tried reinstalling it using

cargo install tabiew

[AndydeCleyre]

That is the only way I've been installing it, aside from sometimes adding --locked to see if that's any better. It's not a problem with tabiew really, but I get 12 errors building polars-arrow.

[AndydeCleyre]

I'm guessing polars-arrow requires a newer rust/cargo than 1.77.1, though it tries anyway.

[shshemi]

I'm using v1.80.0 on my machine

[AndydeCleyre]

I only now realized managing rust via mise is a bad idea.

Now that I'm using my distro's packaged rust (which I wrongly assumed to be older than the mise/asdf plugin's "latest"), I can reproduce the successful fix. Thanks!