search

scraperwiki / spreadsheet-download-tool

A ScraperWiki plugin for downloading data from a box as a CSV or Excel spreadsheet
BSD 2-Clause "Simplified" License
2 stars 1 forks source link

Large numbers get rounded #70

Closed frabcus closed 10 years ago

frabcus commented 10 years ago

e.g. in_reply_to_status_id of 446521933308637200 shows in the .xlsx download in LibreOffice as 446521933308637000

I've filed this bug here, in case there is a way of storing large numbers in .xlsx, in which case it should do that.

Possibly the Twitter tool should save a string for this field - if so refile a bug there.

drj11 commented 10 years ago

In twitter id is a 64-bit integer. Technically, this is fine for JSON which represents numbers with as many decimal digits as you like, but... a 64-bit number integer can't be stored in a JavaScript float (which is a 64-bit double with 53-bits of precision). That's why the id_str field exists.

sqlite3 can store a 64-bit integer in an INTEGER field (http://www.sqlite.org/datatype3.html), but it seems safer to store them as strings (because other database engines may not handle 64-bit integers).

I couldn't (with about 2 minutes of research) find any way to represent a 64-bit integer in Excel.

drj11 commented 10 years ago

Is the Right Thing for this tool to work out if any given number can be safely stored as a float, and use string if not?

pwaller commented 10 years ago

shudders

StevenMaude commented 10 years ago

@drj11 If you didn't already encounter it, the official advice for Excel appears to be format very large numbers as text.

scraperdragon commented 10 years ago

Fix applied to twitter-*-tool