[BUGZILLA #17239] Improve Handling of Integer and Floating Point Values in Exported DBFs

MichaelChirico commented 4 years ago

Created attachment 2239 [details] Patch file for improving identification of int and precision of float numbers in DBF files (foreign package)

We were trying to use R to automate the generation of deliverable DBF files as part of a transportation project for Michigan DOT and Detroit's MPO (SEMCOG) and realized that the resulting files did not generate integer and floating point column specs in the DBF header in a way that was consistent with the underlying data. The main issues we saw, and that became sticking points for the client, were:

Some of the integer variables were exported as floating point numbers
The number of decimal places in the exported files was fixed and did not reflect the number of decimal places in the data

To address these issues, we made changes to the dbf.R file in the foreign package that improve the identification of integer data and explicitly compute and specify the precision of floating point data in the generated DBF file header.

We believe that these changes increase the fidelity of DBF files generated by R and would like to see them included in future updates.

We have attached a patch file for this change.

METADATA

Bug author - Marcelo Simas
Creation time - 2017-03-17 15:10:51 UTC
Bugzilla link
Status - UNCONFIRMED
Alias - None
Component - Add-ons
Version - R 3.3.*
Hardware - All All
Importance - P5 enhancement
Assignee - R-core
URL -
Modification time - 2017-04-19 16:18 UTC

MichaelChirico commented 4 years ago

Bump

METADATA

Comment author - Marcelo Simas
Timestamp - 2017-04-18 16:01:04 UTC

MichaelChirico commented 4 years ago

"Bump"ing a report isn't going to make a difference. If nobody replied to your original report, it's likely that nobody is currently interested in supporting that aspect of the package. (I don't even know what a DBF file is. It might help to explain what it is, and why it's important.)

You could also try to figure out who committed the original dbf.R file, and ask them to deal with your issue. But they might not be interested in supporting it any longer, so the solution might be to remove dbf.R. The source repository is at https://svn.r-project.org/R-packages/trunk/foreign.

METADATA

Comment author - Duncan Murdoch
Timestamp - 2017-04-18 23:10:35 UTC

MichaelChirico commented 4 years ago

Sorry, did not want to ruffle feathers but felt like a month was a long time for a bug to get no comments/updates.

My report includes a patch that addresses the issue and has improved the resulting DBF file. R is quickly becoming a popular tool for transportation planners and modelers (both in consulting and government) and several tools used in this area rely on DBF files to store tabular data. Removing support for DBF files would impact the ability of R to interact with these types of tools.

I reported this here b/c the foreign package reports that it is maintained by the core team, but I will follow your suggestion and attempt to contact the original package committers.

Thank you, M

METADATA

Comment author - Marcelo Simas
Timestamp - 2017-04-19 16:18:53 UTC

MichaelChirico / r-bugs