ODM2 / ODM2DataSharingPortal

A Python-Django web application enabling users to upload, share, and display data from their environmental monitoring sites via the app's ODM2 database. Data can either be automatically streamed from Internet of Things (IoT) devices, manually uploaded via CSV files, or manually entered into forms.
BSD 3-Clause "New" or "Revised" License
31 stars 8 forks source link

Fix broken "Add New Sensor" for some variables #342

Closed aufdenkampe closed 2 years ago

aufdenkampe commented 5 years ago

While teaching a group today, we all got errors and couldn't move forward when trying to add specific variables, including:

Maybe there are others?

@jcaraballo17, @Maurier, any idea what might be happening? Is this a database issue related to the migration, similar to what we've seen previously?

cc: @horsburgh, @SRGDamia1

jcaraballo17 commented 5 years ago

Can't think of any reason that would happen. I'll take a look at it and try to figure out what the issue is in a bit.

horsburgh commented 5 years ago

@aufdenkampe - can you describe the steps and the specific error?

horsburgh commented 5 years ago

I just tried to add these two sensors to one of my sites, and when I click the button to add them I just get a small message at the bottom that says "Failed to add sensor" and it just leaves me in the modal window. @aufdenkampe - have you had issues with any others?

SRGDamia1 commented 5 years ago

I got it yesterday with a Zebra Tech DO in ppm, though, oddly, not the other DO variable..

jcaraballo17 commented 5 years ago

I figured out what the problem is, I'll try to fix it tomorrow or the beginning of next week. Do we want to create a hotfix for this one? I don't think we should wait until the next release to fix this bug.

SRGDamia1 commented 5 years ago

Please, please fix this ASAP!

aufdenkampe commented 5 years ago

@horsburgh and @SRGDamia1, thanks for digging in and providing extra details.

@jcaraballo17, thanks for figuring out the problem. Please fix this as a hot fix as soon as possible, as it is a big bug that stops us from deploying new stations (which we're still doing this winter).

Once you get it fixed, it will be important for you to describe the bug to @steveskrip, @htaolimno and me, so that we can avoid these kinds of bugs in the future.

SRGDamia1 commented 5 years ago

@aufdenkampe - You can set up the station with the wrong variable and then go back and change the variable metadata that's tied to that UUID once this problem is fixed. It's a PITA though.

jcaraballo17 commented 5 years ago

@aufdenkampe @SRGDamia1 Fixed!

aufdenkampe commented 5 years ago

@gcutrell, @fisherba, and I all just discovered that this bug has again appeared for at least this sensor:

It seems to work, however, for the Bosch_BME280_Temp sensor.

@jcaraballo17, is there anything else that you did with the databases records to fix this? It seems that the code fix #342 would not have caused or fixed the problem alone, given that it was only for a few of the sensors. It would be very helpful to describe what you might have done with the database records to fix this.

dbressler75 commented 5 years ago

Ensign and I had the same/similar problem recently. Decagon CTD depth and conductivity could be added but not add Decagon CTD temp - "Failed to add new sensor".

Same issue when trying to add Mayfly data logger temp (and others) - "Failed to add new sensor"

dbressler75 commented 5 years ago

Time offset issue - no UUID for time offset when uploading new data/new sensors. No instructions in current manual for how to deal with Time offset and any UUID that may be necessary for this parameter

dbressler75 commented 5 years ago

Seemingly all new sensor additions/data uploads give the message "Failed to upload"; however, data actually do upload (for particular sensors, but not others - see previous comments)

aufdenkampe commented 5 years ago

@dbressler75, thanks for providing additional input on this issue. we'll add Decagon CTD temp to the list of sensor/variables that are somehow unlinked from the database.

Regarding Time offset, that is timezone information that when added to the local date time value allows one to convert between timezones. Therefore, there is no need for a UUID. UUIDs are only given to measured sensor variables, and also as unique identifies for sites (which are a type of Sampling Feature, according to the conventions of the Open Geospatial Consortium's Observations and Measurements standard).

SRGDamia1 commented 5 years ago

Is there any chance this will be fixed?

aufdenkampe commented 5 years ago

Yes! We just received a bit of funding and will get started in the next few days.

Sent from my iPhone

On Apr 24, 2019, at 5:48 PM, Sara Damiano notifications@github.com<mailto:notifications@github.com> wrote:

Is there any chance this will be fixed?

- You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHubhttps://github.com/ODM2/ODM2DataSharingPortal/issues/342#issuecomment-486456100, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABHNHVFGRR3F34CLGO6BS3DPSDPTBANCNFSM4GKJSAHQ.

SRGDamia1 commented 5 years ago

Oh, thank goodness!

aufdenkampe commented 5 years ago

@gcutrell has been having this bug with adding these sensor variables:

But it works with:

@jcaraballo17 and @htaolimno, I think this could be potential database issue.

aufdenkampe commented 5 years ago

For https://monitormywatershed.org/sites/RVSMRT%232/ we want Davis_VantagePro2_5minRainTotal in inches, so I tried @SRGDamia1's suggested workaround (see https://github.com/ODM2/ODM2DataSharingPortal/issues/342#issuecomment-447498516) and decided to temporarily set up the station with Davis_VantagePro2_10minRainTotal. However, I wanted units of inches, but after adding that as an instrument output variable via the Django Admin panel, I now get a Failed to update sensor error when I select:

cc: @Volk3rJ

aufdenkampe commented 5 years ago

Fixed with https://github.com/ODM2/ODM2DataSharingPortal/commit/298bc70093bf90773708e7f221fae8db4baebf35, which is in the Master branch. We'll try a hot-fix.

jcaraballo17 commented 5 years ago

Fixed.

fisherba commented 5 years ago

I just tested and the sensors I couldn't add yesterday worked today (all were the temp variable for each sensor). Thanks!

jcaraballo17 commented 5 years ago

Perfect! Adding the close me tag.

aufdenkampe commented 5 years ago

These sensor/variables still do not work:

@fisherba said she had to log out and then log back in, but that didn't work for me. I'm about to try a different browser.

aufdenkampe commented 5 years ago

@fisherba and I just did some extensive testing, and what we found was that there is a very persistent caching situation, where it all worked when we each when to a browser that we haven't recently used.

However, for the browser we each most often use (Safari for me, Chrome for Beth), no amount of refreshing the browser, restarting the browser, or logging out would solve the issue.

I'll try again in a few days.

aufdenkampe commented 5 years ago

From @Volk3rJ:

I had luck with adding new sensors after clearing all my browsing data in Chrome (for the last hour – incl. history, cookies and cache). I had to do this for each sensor I wanted to add.

I just did a hard "Empty Caches" command in Safari, and finally got it to work. I'm surprised that the softer cache reseting of refreshing the browser there times didn't work.

aufdenkampe commented 5 years ago

I just had this issue again on production with Digi XBee3 signal percent and RSSI

aufdenkampe commented 2 years ago

This should be permanently fixed with: