search

junyuan-chen / ReadStatTables.jl

Read and write Stata, SAS and SPSS data files with Julia tables
MIT License
40 stars 5 forks source link

type conversion, dangerous Int8 data #43

Open PaulSoderlind opened 3 weeks ago

PaulSoderlind commented 3 weeks ago

My .dta (STATA) file has some columns as Float32, others as Int8, etc. Especially the Int8 casuses lots of problems in the code (overflow...), so they must be converted to Int64.

Is there a simple way of telling readstat() to do that? (I prefer not to create a df.) If not, I would kindly ask for an option safe_types (or something) that would convert all Intx to Int64 and perhaps also all Floatx to Float64.

junyuan-chen commented 3 weeks ago

Hi! It sounds like the problems you mention occur after the file is loaded, right? Currently, the value types are determined based on the variable-specific metadata in the file. So, you are asking for type conversion at the time of reading the file, without building the columns based on the value types suggested by the metadata. I think this would be a feature to be implemented. It's viable although not readily available at this moment.

PaulSoderlind commented 3 weeks ago

Yes, such a feature would be very useful.