Gapminder / ddf-validation

How do you know your DDF dataset is ✅valid?
https://open-numbers.github.io/ddftools.html
GNU General Public License v3.0
7 stars 2 forks source link

file table overflow when validating large files #520

Closed semio closed 6 years ago

semio commented 6 years ago

ddf-validation version: 1.17.0

Following exception raised when validating this dataset: https://github.com/open-numbers/ddf--ilo--ilostat. The file ddf--datapoints--eap_teap_sex_age_edu_nb--by--ref_area--sex--classif1--classif2--time.csv is a 50MB file, so I guess it's because the file is large.

      throw er; // Unhandled 'error' event
      ^

Error: ENFILE: file table overflow, open '/Users/semio/src/work/Gapminder/datasets/open-numbers/ddf--ilo--ilostat/ddf--datapoints--eap_teap_sex_age_edu_nb--by--ref_area--sex--classif1--classif2--time.csv'
Emitted 'error' event at:
    at fs.js:2094:12
    at FSReqWrap.oncomplete (fs.js:169:20)
buchslava commented 6 years ago

@semio please, read this one: https://github.com/reactioncommerce/reaction-cli/issues/11

I suppose the reason is related to OS settings

buchslava commented 6 years ago

... or this one: https://github.com/meteor/meteor/issues/8057

semio commented 6 years ago

@buchslava oh, I didn't think of it is a OS problem. I will have a test and let you know, thanks!

semio commented 6 years ago

tested and confirmed it's an OS problem, close issue now.