search

sparkfun / Qwiic_OpenLog

Open source I2C based datalogger. The bigger, better cousin to the original serial OpenLog.
Other
24 stars 6 forks source link

~160x faster write speed, ~2x effective storage size, may break compa… #3

Closed s-m-t closed 5 years ago

s-m-t commented 5 years ago

…tibility with older code

Changed writeFile so it: -doesn't sync to SD card every byte written -doesn't write an extra null character after every byte specified by the user to be written. sizeof(myData) does not* return the length of the string. It returns the size of the type pointer to char, which on the m328p is two bytes. Thus, every time writeFile is called it writes the byte the user supplied and the zeroed out byte right ahead of it. When you take out the sd card and view it on your computer you may not see this because a lot of text editors silently ignore most special ASCII characters. View in VIM or Sublime to see for yourself.

Added syncFile function and neccesary additions to memoryMap, registerMap, valueMap, functionMap: -If user doesn't want for the buffer to get full and the write to sd card to be automatically called you can manually call it to be sure your data will be written before whatever device you have is shutoff, loses battery, implodes, etc.

I couldn't find where lastSyncTime was updated so I updated it with millis() whenever the receiveEvent or requestEvent callbacks are initiated. I'm not really sure of this one because I have done zero testing of the low power mode.

You may want to change some things around like keeping the old writeFile function the same for backwards compatibility and adding a new function and i2c register for this style of fast writing.

s-m-t commented 5 years ago

I have an accompanying fork of the arduino library here: https://github.com/s-m-t/SparkFun_Qwiic_OpenLog_Arduino_Library

nseidle commented 5 years ago

This looks really good. I need to do a deeper review with hardware but initially:

s-m-t commented 5 years ago

Right on, I was wondering why they weren't getting highlighted

AndyEngland521 commented 5 years ago

@s-m-t this doesn't seem to be working for me, I'm using your updated version of the library but nothing seems to be written to the log file. The OpenLog is definitely able to create new log files as I see those on the SD Card but they're all blank. Can you double check the commits that you've uploaded?

s-m-t commented 5 years ago

Ah, yeah, none of the examples will work unless you add a call to myLog.syncFile() or you try opening/appending a new file ( sync is called whenever the current workingfile is closed(), see line 114 of log_creation.ino ) after you have buffered your writes.

I've updated the examples

AndyEngland521 commented 5 years ago

Seems to be in working order, thanks for all of your collaboration on this. I'll pull everything in and update a few of our tutorials to reflect the changes. Thanks a bunch @s-m-t