Final version for 2021 deployments

[jerabaul29]

@jvoermans I think it is time to set up the final version for the 2021 deployments.

I have removed the problematic sonar stuff altogether, and put the final code for the logger in a separate branch.

To use, in terminal:

~/Desktop/Git/Vibration_Logger [master|✔]> git fetch
~/Desktop/Git/Vibration_Logger [master|✔]> git checkout version/no-sonar 
Branch 'version/no-sonar' set up to track remote branch 'version/no-sonar' from 'origin'.
Switched to a new branch 'version/no-sonar'
~/Desktop/Git/Vibration_Logger [version/no-sonar|✔]> git pull
Already up to date.

This should then compile fine, and be the latest code, minus the sonar. The only parameter you may want to adapt is the file duration, though I have already put it to 15 minutes here:

https://github.com/jvoermans/Vibration_Logger/blob/ebec6f10e3c12a05323f1d1867971ab0d5224f7c/material_Jean/Due_SD_high_frequency_logger/src/params.h#L60

Can you confirm that:

• you are able to checkout this branch
• you are able to compile

Then, can we perform a full test?

• can you set it up for 2 hours or so, and send me all files (may be quite big, maybe you want to share with a downloadable link from your univ cloud)
• then I will check that everything looks good
• once you have sent me the 2 hour or so, can you set it up for 1 week and then same procedure, send me the files, and I will check that they look good.

Can you do the two tests in conditions as similar as possible to the real world deployment, i.e. GPS access if possible, no serial plugged in, all sensors, etc?

[jvoermans]

Hi Jean,

Yes will do. Just need to prepare the equipment so I can keep it outside :) I will start recording tmrw

On Thu., 28 Jan. 2021, 21:44 JR, notifications@github.com wrote:

@jvoermans https://github.com/jvoermans I think it is time to set up the final version for the 2021 deployments.

I have removed the problematic sonar stuff altogether, and put the final code for the logger in a separate branch.

To use, in terminal:

~/Desktop/Git/Vibration_Logger [master|✔]> git fetch

~/Desktop/Git/Vibration_Logger [master|✔]> git checkout version/no-sonar

Branch 'version/no-sonar' set up to track remote branch 'version/no-sonar' from 'origin'.

Switched to a new branch 'version/no-sonar'

~/Desktop/Git/Vibration_Logger [version/no-sonar|✔]> git pull

Already up to date.

This should then compile fine, and be the latest code, minus the sonar. The only parameter you may want to adapt is the file duration, though I have already put it to 15 minutes here:

https://github.com/jvoermans/Vibration_Logger/blob/ebec6f10e3c12a05323f1d1867971ab0d5224f7c/material_Jean/Due_SD_high_frequency_logger/src/params.h#L60

Can you confirm that:

• you are able to checkout this branch
• you are able to compile

Then, can we perform a full test?

• can you set it up for 2 hours or so, and send me all files (may be quite big, maybe you want to share with a downloadable link from your univ cloud)
• then I will check that everything looks good
• once you have sent me the 2 hour or so, can you set it up for 1 week and then same procedure, send me the files, and I will check that they look good.

Can you do the two tests in conditions as similar as possible to the real world deployment, i.e. GPS access if possible, no serial plugged in, all sensors, etc?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/jvoermans/Vibration_Logger/issues/44, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKIGDYCASRF6VF2W73FD7VLS4E52HANCNFSM4WW2SQKA .

[jvoermans]

Also, have to send three of these on Monday already to Antarctica. So its gonna be a relatively short test for those.

On Fri., 29 Jan. 2021, 12:11 Joey Voermans, joey.voermans@gmail.com wrote:

Hi Jean,

Yes will do. Just need to prepare the equipment so I can keep it outside :) I will start recording tmrw

On Thu., 28 Jan. 2021, 21:44 JR, notifications@github.com wrote:

@jvoermans https://github.com/jvoermans I think it is time to set up the final version for the 2021 deployments.

I have removed the problematic sonar stuff altogether, and put the final code for the logger in a separate branch.

To use, in terminal:

~/Desktop/Git/Vibration_Logger [master|✔]> git fetch

~/Desktop/Git/Vibration_Logger [master|✔]> git checkout version/no-sonar

Branch 'version/no-sonar' set up to track remote branch 'version/no-sonar' from 'origin'.

Switched to a new branch 'version/no-sonar'

~/Desktop/Git/Vibration_Logger [version/no-sonar|✔]> git pull

Already up to date.

This should then compile fine, and be the latest code, minus the sonar. The only parameter you may want to adapt is the file duration, though I have already put it to 15 minutes here:

https://github.com/jvoermans/Vibration_Logger/blob/ebec6f10e3c12a05323f1d1867971ab0d5224f7c/material_Jean/Due_SD_high_frequency_logger/src/params.h#L60

Can you confirm that:

• you are able to checkout this branch
• you are able to compile

Then, can we perform a full test?

• can you set it up for 2 hours or so, and send me all files (may be quite big, maybe you want to share with a downloadable link from your univ cloud)
• then I will check that everything looks good
• once you have sent me the 2 hour or so, can you set it up for 1 week and then same procedure, send me the files, and I will check that they look good.

Can you do the two tests in conditions as similar as possible to the real world deployment, i.e. GPS access if possible, no serial plugged in, all sensors, etc?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/jvoermans/Vibration_Logger/issues/44, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKIGDYCASRF6VF2W73FD7VLS4E52HANCNFSM4WW2SQKA .

[jvoermans]

@jerabaul29 I think I'm going to skip the temperature probes as well. There is something with the hardware that keeps on struggling and taking too much time to figure it all out (I think I've wasted already two weeks on such problems). In testing phase it works fine, but when build with exactly the same wiring it suddenly doesn't. All wires are checked and properly connected. It is unfortunate but better stick with simply the geophone logger I think. Now occasionally the third temperature probe gives a reading, but the others don't. Already tried with 10K clock speed...

I think in large it lies with the Arduino Due or something I just don't understand of it. Perhaps voltage is just getting too low when multiple things are attached. But with the Mega I don't have any of these issues with the probes. Only reason Due was chosen was because of the 12 bit analog, but sure this can be done with a breakout and the Mega as well...

Perhaps in next deployment I'll know why it is doing difficult :)

[jerabaul29]

Ok, strange about the hardware stuff, sorry to hear that. Yes, this is annoying, same thing with the sonar, when a hardware piece / its software package does not work it does take a lot of time. I think some instruments are not fully "polished" :( .

Ok, good luck! :) .

[jvoermans]

Indeed!

I'll try to finish the first geophone logger tonight, so I can put it on the roof overnight...

[jerabaul29]

Sounds good!

[jvoermans]

But many thanks anyway, impressive how quickly and without hardware you were able to produce a functioning sketch!

[jerabaul29]

Thanks. Actually I feel a bit bad about it - I should have bought a bit of electronics myself and done it in a more self-contained way so that I do not need to ping you all the time for testing... Unfortunately with change of job, corona, etc, my plans got quite disturbed.

You are doing an amazing job with setting these up together, planning the instruments design, etc, so I think this is a nice collective effort :) . Hope everything works well "in production" too, and that we are lucky with the deployments and get good data.

[jvoermans]

Uploading soon. FIle sizes are small and a lot, so guess it just recorded every 15 seconds. I think I uploaded the "15 * 60" duration, but no have to double check...

[jvoermans]

Just send you a link to check the files (though they are probably only 15 s long). In one of the first I jumped a few times near the geophone.

Ok damn, my bad. Seems like I didn't upload the "15*60" value. Not sure though what version I put on. Anyway, I'll swap and put it back on the roof....

[jerabaul29]

Ok, sounds good, will look at these tomorrow.

[jerabaul29]

Yes, sorry, many versions. Remember to git checkout the right branch to get the 'release' version, see the first post higher up :) .

[jvoermans]

I put 4 zip files in the folder. I think you should be able to see all of them?

Test_1: when placed on the roof overnight, but 15 s files (only took the first and last few) Test_2: ~2 hours on the roof 15 min files (note, first file I jumped 3 times or so next to it). Test_3: Same as Test_2 but used a SD-card extender (about 1 m long) Test_4: Same as Test_2 but filled the 128GB card with 50 days of 10MB files (i.e., I renamed one of the files F9999999 and copied-pasted it many times).

[jerabaul29]

Hi @jvoermans ,

It is good that we do some "real testing" with longer logging...

At first I had some problems parsing the data and I got very worried. However a couple of hours of debugging later things look quite good ^^ .

• I think that the ADC data are correctly logged over the whole period of time; there is only 1 question: it looks like ADC channel 2 is by default saturated, and goes down if there is some signal. If I remember well, this is a design choice, in order to have 1 channel with large half-amplitude, is this memory of some old discussions I have true (see attached screenshot)? If this is not the expected behavior, that may be a minor wiring issue of the geoduino, in which case you may want to fix it - or not if it is too much work, as things work well as it is now.

• It looks like the instruments are very sensitive, we can clearly see some signal. The signal has some clear self correlation and some typical frequency, so I do not think that it is noise, I think that it is physical. We are probably measuring vibrations in the building due to wind, people walking, vehicles driving around, lifts etc. This should be promising for deployments on the ice :) .

• I see that there are no instruments other than the geoduinos and the GPS being logged, but I think this is as intended, as you removed the temperature sensors (and of course the sonar), right? :) . As a consequence, I removed not only the sonar but also the temperature sensor logging altogether.

I found a couple of small bugs in the Due code, that were not visible on "short" deployments but become visible when the logger is used for over around 70 minutes. This was the source of the worries I had when starting to look at the data. I spent a few hours on these today, status is:

• these bugs are not too bad, i.e. there is no loss of data and I can fix things in the Python post processing if really needed
• however, fixing things in the python parser is a pain. I pushed a (very simple) fix to the version/no-sonar branch, can you update the code on all Due loggers, perform a new deployment for a few hours, and send me the new files for double checking again? Once this fix is applied, I expect (crossing fingers) that everything will work fine.

FYI, the problem is very minor: I wrote the microseconds timestamps with 9 numbers, while 10 numbers are needed; this means that the microseconds timestamps "wrap up" quite a lot more often than it should in the data files. The fix I applied is simple, see:

https://github.com/jvoermans/Vibration_Logger/commit/0ffd18f35512df7716aa9aa18a32c3fb55e2ed52#diff-13d2fc46ee1865ca83de69e8034264b26ac49f232b8ff8066ab51adf23b4ef43

To upload the code, usual procedure, use the version/no-sonar branch:

~/Desktop/Current/Vibration_Logger [master|✔]> git fetch
~/Desktop/Current/Vibration_Logger [master|✔]> git checkout version/no-sonar
Branch 'version/no-sonar' set up to track remote branch 'version/no-sonar' from 'origin'.
Switched to a new branch 'version/no-sonar'
~/Desktop/Current/Vibration_Logger [version/no-sonar|✔]> git pull

Make sure that you upload the code in the latest version from this branch for the next round of tests :) .

[jerabaul29]

So let me know when you have updated the firmware and done some new outside testing for a couple of hours, and I will have a last look at the data (that should be all good this time).

One more thing: now we are testing the firmware and design "in general" on one instrument, but I suppose we should test individually each Arduino Due logger too, right? When you are at the step of testing each instrument individually, let me know, and I will have one more look at each instrument individually.

But things look good, I think we are getting close. I am very excited about these measurements, as it seems that the geoduinos are super sensitive :) .

[jvoermans]

@jerabaul29 Just uploaded data from the 3 geo recorders. (named 1, 2 and 3). Nr 1 is the one from yesterday, there was a bad solder connection which is why it was saturated.

Unfortunately it is hot and sunny here, so didn't dare to leave them baking in the sun, so they were sheltered. For the first file of each Geo I placed them momentarily in the sun to get a GPS fix, but I'm afraid it disappears once they were moved back in the shadow!

I'm turning them on again to continue recording. There is more shadow now so maybe more GPS connectivity...

[jerabaul29]

That sounds good! I will have a look soon and let you know :) .

[jerabaul29]

Just FYI: the folder Test_5_Geo1 seems to be empty.

[jerabaul29]

(and I got an error message zipping out, so I think this is were the problem lies).

[jerabaul29]

Hi again,

• the data for the logger Geo3 look good, see screenshot, nothing problematic there, everything is as expected, so that is good :)

• the data for the other loggers do not look as good, but I wonder if this is an issue with how the files were zipped and / or the file transfer protocol (like what happened with the corrupted SD card images you got from me over the file transfer), rather than the logger data file:

  • the folder for Geo1 is empty (see screenshot)

• the folder for Geo2 looks like it contains files with the right size (see screenshot)

• but when I unzipp them, I get an error message, and then the file I get have inconsistent sizes:

[jerabaul29]

So I was very worried initially when things did not work as I wanted but now I think this is just an issue with your univ file transfer protocol...

[jvoermans]

Thanks Jean.

Would you mind checking the latest data for me as well? I'm struggling to get all the cabling etc going before the deadline, so would be great if you could check it

I just uploaded them, in separate zip files this time...

Cloudstor is doing a bit difficult I see. I just send you a Dropbox link.

[jerabaul29]

Sounds good, no worries, I will do it now :) I do it in 15 minutes :) .

[jerabaul29]

(and good luck with the cabling! Which cabling is taking long time / are you busy with?).

[jerabaul29]

The file download worked perfectly this time :) . Could you / maybe you should (when you have better time :) ) report to your univ that their file sharing solution is broken...

Parsing now, will take a few minutes, everything looks perfect so far :) .

[jvoermans]

Cables of the sensors. Needs to be waterproof, everything. I've tried to do it too tidely and too user friendly by trying to use as few cables as possible and paying for that now unfortunately...

[jerabaul29]

Ok, sorry to hear that... Good luck!

[jerabaul29]

I think that things look good. One of the loggers lost GPS fix at some point, but I suppose this is expected if under a roof? I will be busy in 30 minutes now, but I investigate in more detailed the lost GPS fix after that :) .

[jerabaul29]

Everything looks good. Quite bad GPS reception quality, but from what you said I understand this is expected :) .

[jerabaul29]

I can confirm that everything looks good for all loggers three Due :) The logger 3 lost GPS contact after a few tens of minutes, maybe double check that the GPS antenna is well connected and engaged (both the SMA and the uFL connectors; possible the SMA connector is not tight enough, or that hte uFL popped out, this happens sometimes), but that may be due to bad reception too.

Otherwise all ready to go :) .