search

IDEMSInternational / R-Instat

A statistics software package powered by R
http://r-instat.org/
GNU General Public License v3.0
38 stars 103 forks source link

IMPORTING DATA SET FROM EXCEL INTO R-INSTAT #4267

Open LethahaLes opened 6 years ago

LethahaLes commented 6 years ago

r-instat issue

rdstern commented 6 years ago

This was the error when trying to import a large(ish) set of data into R-Instat from Excel. It is about 450,000 rows. As you see, in the background, it was able to read the Dodoma data.

These data read fine into my machine and into Motebang's.

He has tried with 3 different data sets - all about the same size. A much smaller file - 63,000 rows from Excel imports fine into his machine.

We are trying other options, e.g. can the data go in if it is an R file. It does go in OK as a csv file. So this looks partly like an Excel issue, though the Excel files go in fine on the other computers.

dannyparsons commented 6 years ago

It seems this has been a problem with read_excel using a lot of memory https://github.com/tidyverse/readxl/issues/150, although this is quite old. But it would explain why smaller files and csv files import ok. Is the machine particularly slow? Can you find out how much memory there is? Can you try closing all other programs to see if that frees up enough memory to import the large file?

nassuphis commented 6 years ago

I got the same error: Error in read_fun(path = path, sheet = sheet, limits = limits, shim = shim, : std::bad_alloc

for a sheet while processing a large number of sheets. I do not believe this is a memory problem, because all sheets were identical in size. When saving the offfending ".XLS" sheet as ".XLSM" I got the below error message: image

I beleive that this might have something to do with the error? The ".XLSM" sheet works fine.