Column row/number seems not to correspond to original data

[jwhendy]

I can't use my original data as it's confidential, but wanted to give a general idea as perhaps someone can clarify if I'm doing something wrong. If not, give me some time to try and reproduce via fake data.

At the moment... this is work related, I'm new to this package, and I'd like to understand if I'm missing something silly.

I was running a script which reads in a CSV, and saw an error:

Warning message:
One or more parsing issues, call `problems()` on your data frame for details, e.g.:
  dat <- vroom(...)
  problems(dat) 

From the docs, I see that problem() returns:

A data frame with one row for each problem and four columns: row,col - Row and column number that caused the problem, referencing the original input

In my script, I pass col_types, and so I recreated the read (currently with readr::read_csv():

library(vroom)
col_types <- "TciciccccdcicclicdiildicdiTT"
dat <- vroom(file_path, col_types = col_types)

Now I do problems(dat) and see the first row (data obscured):

> problems(dat)
# A tibble: 31 × 5
       row   col expected   actual  file                                                                             
     <int> <int> <chr>      <chr>    <chr>                                                                            
 1 1647868    20 an integer a_string  C:/path_to_file…

However, column 20 is indeed defined as i, and I find:

> dat[, 20] %>% unique()
# A tibble: 2 × 1
   col_name
  <int>
1    NA
2    16

I recognize the format of a_string, which would put it in column 7, not 20. This is a pretty massive file. Is this somehow about delimiters or e.g. newlines causing something to bump onto the next line?

I read this in again, but used skip=1600000 and then saved it out. The first error is on row 47868, and when I look, everything seems as it should. a_string shown in the output of problems() is in Column G.

Let me know if something stands out I should check into further. It seems like a false alarm, but I wanted to try and investigate to make sure I wasn't missing something which would affect my results. Thanks!

[jennybc]

Hard to say and this doesn't immediately tweak my spidey sense about being connected to some known issue.

Have you looked at that specific line in the raw file or its general neighborhood to see if there's anything "interesting" about it?

[jwhendy]

@jennybc thanks for the skim, and I figured this would probably be a hard one to say much about. I wish I could just attach my file! I didn't see anything interesting. The file was too big to open directly in LibreOffice, so I had do the trick mentioned:

I read this in again, but used skip=1600000 and then saved it out. The first error is on row 47868, and when I look, everything seems as it should. a_string shown in the output of problems() is in Column G.

Could we start simple since I'm new to vroom? These are all just various ways of asking the same thing just to make sure I actually understand the error output correctly:

• when vroom says col 20, is there any other interpretation than "the data is in the 20th column?"
• when vroom says "col 20", is this col 20 of the original file or the loaded data object? [Edit: it says the original input, so I think this is answered...]
• [Edit: since the previous is "the original input", we know the input and loaded object have a disconnect in the actual column, right?] assuming you've seen previous disconnects between reported raw file column and loaded data column... are there any things one could check (I was thinking e.g. \t or \n in a cell or some goofy escape character)
• when I look at dat[problem_row, ], should I see the value it says is there in column 20?
• when I look at unique(dat[, problem_col]), should I see the value it says in the vector?