Allow super-users to add new variables and sensors

[SRGDamia1]

Include a basic form for a super user to add this information.

[SRGDamia1]

Also, possibly, allow them to modify current info - ie, change Adafruit to AOSong for re-sold sensors. Though this could be very dangerous if anyone already has these sensors in use. Perhaps we additionally need a table of synonyms for identical sensors sold by multiple vendors who don't necessarily make it clear that the product is from a different manufacturer.

[aufdenkampe]

We discussed prioritizing this in our 2017-08-10 EnviroDIY Data Portal call. Here are the notes from that call:

• Adding new sensors
  • ​​I'm kind of thinking the current naming convention needs to be restructured, so that it is more generic and matches the library names in https://github.com/EnviroDIY/ModularSensors
• Adding new variables (for the new sensors)
  • Allow super-users to add variables (ugly django-generated forms fine)
  • Button for submission from any user akin to http://vocabulary.odm2.org/ . We could have users fill in a form, but this would then send a notification to the system admins who would then vett the submissions and add the new submission via the admin interface. No need for a full moderation system built in Django.

[aufdenkampe]

Regarding changing existing sensor names, as mentioned by @SRGDamia1 and in notes, perhaps we should first just provide a revised sensor table. I can do that.

[aufdenkampe]

The intent of revising the EnviroDIY Sensor Station Parts List is to revise the sensor Make_Model information, to match the Make_Model labels used in https://github.com/EnviroDIY/ModularSensors

[horsburgh]

At this point, any changes we make to existing sensor names, etc. would need to cascade to existing instances in the database (e.g., any use of the old names would be updated to use the new names in the database). I would very much prefer to not maintain a list of synonyms. I think it is unnecessarily complicated to do so.

So, to summarize what needs to be done on this issue:

• [ ] @SRGDamia1 and @aufdenkampe to get USU a revised sensor table
• [ ] @jcaraballo17 to update the sensor/variable names in the production database. The new names will replace the old names.
• [ ] @jcaraballo17 and @Maurier to create a form for admin users to create a new sensor and its corresponding output variables. They go together and can't be separated, so the new form has to enable creation of both together.

As part of #73 @Maurier will create a simple "Request new sensor" form that will enable a user to request addition of a new sensor to the database. See details in that issue.

[aufdenkampe]

@horsburgh, Thanks for breaking this down into three separate tasks. I'll work on the first, soon.

I had been hoping that by updating 1-2 tables in the ODM2 instance, that all existing records would then be auto-updated. I agree that a list of synonyms would be a pain, but I really do want to change the sensor Make & Model info for some of the existing sensors. I'll do that in the update, and @jcaraballo17 & @Maurier can let us know if updating existing sensors causes huge headaches or not.

[SRGDamia1]

I was trying to work on the google doc, and it's crashing for me. Is anyone else able to open it?

[aufdenkampe]

Yes, I can get into the 'sensors_for_websdl_v3' tab in the EnviroDIY Sensor Station Parts List.

BTW, I'm creating a separate issue for the task of revising that, issue #94.

[horsburgh]

@aufdenkampe and @SRGDamia1 see my latest comment on #73. We will implement an email request for users to request a new sensor. Those email requests will have to be handled by a super user who can assemble all of the information needed to add a new sensor, specify output variables, units, etc. @jcaraballo17 has been working on a Django Admin page that will enable submission of new sensors by super users.

[aufdenkampe]

@horsburgh and @jcaraballo17, what you describe above and in #73 all sounds awesome. Thanks!

[SRGDamia1]

How to I access this?

[aufdenkampe]

Click on "Admin" menu item in black bar, upper right.

[SRGDamia1]

Hm. I see no such thing.

[Maurier]

This will only show up to staff accounts.

@jcaraballo17 can you make sure @SRGDamia1's account is flagged as staff?

[SRGDamia1]

I still cannot see any of this on the sandbox.

[horsburgh]

@SRGDamia1 - do you have more than one user account? I wonder if @jcaraballo17 tagged the right account to the group that has access to this functionality. It seems to be working for Anthony, and the link shows up when I log in, so it's likely just an association with your account.

[SRGDamia1]

I don't think I do...

[jcaraballo17]

Sorry, that was my bad. I gave the user SGDamiano1 permissions to add and edit the data, but gave staff status to the user sdamiano. You should be able to see it now from both of them.

[SRGDamia1]

Oops. Can you just delete user sdamiano so I don't have two users on the system. I think I've registered all the sites as SGDamiano1.

[SRGDamia1]

Also, is there any sort of verification on the site so the same email isn't used for multiple users on the future, if not, set should open a new issue for that.

[SRGDamia1]

Now being able to look at the admin panel, I love it.

[aufdenkampe]

I agree with @SRGDamia1 that this is awesome!

I have one question about variables (see http://envirodiysandbox.usu.edu/admin/dataloader/variable/). Would the underlying software logic break if we changed variable codes?

As we discussed in https://github.com/ODM2/ODM2WebSDL/issues/74#issuecomment-332660214 and https://github.com/ODM2/ODM2WebSDL/issues/74#issuecomment-332719453, in this EnviroDIY implementation of ODM2, for our VariableCode list are actually a using concatenation of SensorMake_SensorModel_Variable (i.e. AOSong_AM2315_Humidity), rather than the typical use of VariableCode as given by the Term in http://vocabulary.odm2.org/variablename/.

Therefore, if we changed the VariableCode of both AOSong_AM2315_Humidity and Bosch_BME280_Humidity to relativeHumidity (as given by http://vocabulary.odm2.org/variablename/relativeHumidity/), would the code break and produce database errors?

My motivation for discussing this is that the EnviroDIY deployment of ODM2 will showcase a lot of ODM2 capabilities, especially via it's web services such as WOFpy (http://data.envirodiy.org/wofpy/). The current use of VariableCode produces WOF results that are not consistent with other WOF service implementations, nor with our greater ODM2 goal of showcasing data interoperability.

This greater issue does not need to be resolved in this release, but it would be important for Sara and I to know the answer to my question of whether the website code would produce errors.

cc @emiliom & @lsetiawan

[horsburgh]

@aufdenkampe - the VariableName terms in the CV were never intended to be used as VariableCodes. VariableCodes are chosen by the user. So, it is not "typical" to use those terms as VariableCodes. VariableCodes need to be unique for our purposes.

What is the WOFPy inconsistency?

[emiliom]

I'm with @horsburgh:

• VariableNames are different from VariableCodes; the latter are not CV terms
• What's the WOFpy inconsistency? @lsetiawan and I (with @horsburgh's help) have invested substantial effort in implementing ways to maximize WOFpy's ODM2 vocabulary compatibility with WaterML/ODM 1.1 vocabularies. It's never going to be a perfect alignment, though.

However, if you want my and @lsetiawan's attention, please open a separate issue that's actually focused on this narrower topic rather than the original issue. Otherwise it's hard to keep track of "side" discussions.

[aufdenkampe]

The new Admin for Super Users now provides them with the ability to add or edit:

• Equipment Models
• Instrument Output Variables
• Organizations
• Units
• Variables It's completely awesome, so let's close this issue. Thanks!

@horsburgh, it appears that the answer to my question -- "If we changed the VariableCode of both AOSong_AM2315_Humidity and Bosch_BME280_Humidity to relativeHumidity (as given by http://vocabulary.odm2.org/variablename/relativeHumidity/), would the code break and produce database errors?" -- is YES.

Therefore, we won't change these VariableCodes.

I would like to have a broader conversation about how we use VariableCodes, in general, especially as we look to future proposals that might include a component that moves forward our EnviroVariableNames-ODM2TeamIdeas, but I'll take talk @emiliom's suggestion to move that in a more dedicated thread.

[aufdenkampe]

@horsburgh, @Maurier, @jcaraballo17 I just noticed that my personal and LimnoTech accounts don't have Admin permissions on the new Django Administration backend panel at http://data.envirodiy.org/admin/, now that it is all on production. I do have super-user privileges on the front-end, and I successfully deleted Juan's staging test site in the DR and can see the button to the backend admin panel. However, when I get to that Admin panel, I get the notice that "You don't have permission to edit anything.​"

Let me know when you fix that, so that we can edit and add sensors.

This could be related to #187.

[aufdenkampe]

Fixed! Thanks @jcaraballo17!