clementfarabet / lua---csv

A package to read and write CSV. Provides high-level database-like handlers.
46 stars 16 forks source link

Saving CSV: scrambled data #20

Open Alex870 opened 8 years ago

Alex870 commented 8 years ago

In File.lua the function "tocsv" takes a row of data (in a table) and converts it to a string:

local function tocsv(t, separator, nan_as_missing)
local s = ""
   for _,p in pairs(t) do
      if (nan_as_missing and p ~= p) then
         p = ''
      end
      s = s .. separator .. escapeCsv(p, separator)
   end
   return string.sub(s, 2) -- remove first comma
end

However, pairs(t) does not have a guaranteed ordering. As a result, for some (especially large) data sets, the ordering of each row (including the header row) becomes inconsistent from row to row.

Possible solution: ipairs(t) guarantees order, put places constraints on the data format in t (requires it to effectively be a numbered array), and so may not be drop-in replacement without other rework.