search

kartoza / django-bims

https://testing.healthyrivers.kartoza.com
GNU Affero General Public License v3.0
12 stars 19 forks source link

Rivers Database - additional tables and data for importing #1610

Closed helendallas closed 4 years ago

helendallas commented 5 years ago

The following tables and data still need to be imported from the Rivers Database: All data are at the level of Sampling Date (SIteVisit) and not Site.

Site Visit Level Changes

CanopyCover - Open, Closed, Partially Open

Chem - ChemCode, ChemDescription. This is linked to ChemUnit and to SiteVistiChem

Embeddeddness - 0-25, 26-50, 51-75, 76-100

StreamDimension - only import Water Surface Width (m). This is linked to StreamDimensionCategory

WaterLevel - Dry, Isolated pools, Low flow, Moderate flow, High flow, Flood

WaterTurbidity - Clear, Discoloured, Opaque, Silty

Site Changes

Then ChannelType - Bedrock, Mixed bedrock and alluvial - dominant type(s), Alluvial with dominant type(s) - this is linked to ChannelTypeDetail - Boulder, Cobble, Gravel, Sand

These are currently at the Sampling Date (SiteVisit) level - but needs be be changed to the Site Level

Link to ticket #1479

timlinux commented 5 years ago

These are at site visit level and not the site

dimasciput commented 5 years ago

Only Chem related tables have been imported.

We still need to import these tables :

CanopyCover - Open, Closed, Partially Open

Embeddeddness - 0-25, 26-50, 51-75, 76-100

StreamDimension - only import Water Surface Width (m). This is linked to
StreamDimensionCategory

WaterLevel - Dry, Isolated pools, Low flow, Moderate flow, High flow, Flood

WaterTurbidity - Clear, Discoloured, Opaque, Silty
dimasciput commented 5 years ago

image

Water level, water turbidity, canopy cover, and channel type are already in SiteVisit.

dimasciput commented 5 years ago

@helendallas , I couldn't find a link between site visit and stream dimension image

image

dimasciput commented 5 years ago

@helendallas I also couldn't find a relationship between stream dimension -> stream dimension category

streamdimensionid                   |streamdimension         |User                                |bankheightindicator|streamdimensionindicator|datefrom           |datemodified       |dateto|
------------------------------------|------------------------|------------------------------------|-------------------|------------------------|-------------------|-------------------|------|
FC04BFC5-7B42-4E03-97C8-22124B0821B5|Left Bank Height (m)    |952D4F51-118A-4550-8C05-4140C5264708|                  1|                       0|2007-09-04 08:24:58|2007-09-04 08:24:58|      |
D87A8D8D-7EB3-466D-8072-78360513ADDC|Water Surface Width (m) |952D4F51-118A-4550-8C05-4140C5264708|                  0|                       1|2007-09-04 08:24:58|2007-09-04 08:24:58|      |
CB67C936-6382-4C8D-95F9-8F5F6C2EE9E5|Macro-channel Width (m) |952D4F51-118A-4550-8C05-4140C5264708|                  0|                       1|2007-09-04 08:24:58|2007-09-04 08:24:58|      |
A5745C8D-A1EF-404A-AB9C-A1283E398ECE|Active-channel Width (m)|952D4F51-118A-4550-8C05-4140C5264708|                  0|                       1|2007-09-04 08:24:58|2007-09-04 08:24:58|      |
3A61FD2F-16CC-4BA5-98FF-A9A2054BBB84|Right Bank Height (m)   |952D4F51-118A-4550-8C05-4140C5264708|                  1|                       0|2007-09-04 08:24:58|2007-09-04 08:24:58|      |
streamdimensioncategoryid           |streamdimensioncategory|bankheightindicator|streamdimensionindicator|displayorder|User                                |datefrom           |datemodified       |dateto|
------------------------------------|-----------------------|-------------------|------------------------|------------|------------------------------------|-------------------|-------------------|------|
AA16A24B-47F8-4286-8A84-0D30636A93E4|20-50                  |                  0|                       1|           8|952D4F51-118A-4550-8C05-4140C5264708|2007-09-04 08:33:01|2007-09-04 08:33:01|      |
E86E3182-2EBC-4716-8703-136A4AB76C0E|>100                   |                  0|                       1|          10|952D4F51-118A-4550-8C05-4140C5264708|2007-09-04 08:33:01|2007-09-04 08:33:01|      |
3082427A-D080-45CA-9FEF-267BDDCA30B8|2-5                    |                  0|                       1|           5|952D4F51-118A-4550-8C05-4140C5264708|2007-09-04 08:33:01|2007-09-04 08:33:01|      |
7FF7899E-26FB-486B-B55F-489C9D760521|1-2                    |                  0|                       1|           4|952D4F51-118A-4550-8C05-4140C5264708|2007-09-04 08:33:01|2007-09-04 08:33:01|      |
C4E3DEAB-1FB0-4EF1-B0B0-4A770E6D9B99|10-20                  |                  0|                       1|           7|952D4F51-118A-4550-8C05-4140C5264708|2007-09-04 08:33:01|2007-09-04 08:33:01|      |
6DA31347-F10A-49B9-B2F8-5ED04C5DBF37|1-3                    |                  1|                       0|           2|952D4F51-118A-4550-8C05-4140C5264708|2007-09-04 08:33:01|2007-09-04 08:33:01|      |
141FD096-6DA3-4FAF-B447-6CF26D9317E5|50-100                 |                  0|                       1|           9|952D4F51-118A-4550-8C05-4140C5264708|2007-09-04 08:33:01|2007-09-04 08:33:01|      |
03752723-D5C2-4B02-94A5-75CAF3140B29|5-10                   |                  0|                       1|           6|952D4F51-118A-4550-8C05-4140C5264708|2007-09-04 08:33:01|2007-09-04 08:33:01|      |
5FD732BD-B94A-41F6-A670-777F4016057E|< 1                    |                  1|                       1|           1|952D4F51-118A-4550-8C05-4140C5264708|2007-09-04 08:33:01|2007-09-04 08:33:01|      |
4306E1EC-7AEB-4241-A68A-DB8482E8B0B3|>3                     |                  1|                       0|           3|952D4F51-118A-4550-8C05-4140C5264708|2007-09-04 08:33:01|2007-09-04 08:33:01|      |
helendallas commented 4 years ago

@dimasciput image

StreamDimension - only import Water Surface Width (m). This is linked to StreamDimensionCategory

helendallas commented 4 years ago

It would also be good to see this data somewhere on the SASS Record: Is it still possible to add this data? At the moment having the Site Code, Date, Owner, Accredited at time of sampling, Site Image - are all squashed to the right side of the Biotopes sampled. It would be better to have it all above. Then additional info as per this ticket, then biotopes, then the SASS taxon capture form as it is.

Maybe the SASS Record and Add SASS Data needs a slight redesign to have it more aligned with the Fish and Invertebrate Data form - more spacious, with map, @jeremyshelton Can we work on this together please. @dimasciput Is this an option?

Screenshot_2019-11-14 FBIS SASS

timlinux commented 4 years ago

Step 1: Add the map, lat, lon, date and owner in same layout as fish form

Step 2: Add new widgets for:

  1. Canopy cover : Open, Closed, Partially Open
  2. Water surface width (m) : <1; 1-2; 2-5; 5-10; 10-20; 20-50; 50-100; >100
  3. Embeddeddness - 0-25, 26-50, 51-75, 76-100
  4. Water level - Dry, Isolated pools, Low flow, Moderate flow, High flow, Flood
  5. Water turbidity - Clear, Discoloured, Opaque, Silty

@jeremyshelton to confirm if these should be added on fish and invert (and algae when it comes) forms

timlinux commented 4 years ago

Also make the button rounding the same

image

image

timlinux commented 4 years ago

We will wait for feedback from FRC before proceeding

helendallas commented 4 years ago

@dimasciput @timlinux

On reflection lets keep things simple.

For all data capture (algae, invertebrates, SASS and fish) we had envisaged capturing biodiversity data first, then prompting the user to Add abiotic data. They would then go to the Abiotic Data Form, where they would have five Categorical variables populated with a dropdown list; and could then also add data from a list of continuous abiotic variables. If they had no abiotic data, they would just move straight on to adding the Source reference for records.

We envisaged these data being stored in the same way as Rivers DB and Biobase data is, and then served in abiotic graphs (DO, Cond, pH and Temp) and in the csv file.

Categorical variables populated with a dropdown list | Dropdown list

Water surface width : <1; 1-2; 2-5; 5-10; 10-20; 20-50; 50-100; >100 m Water level : Dry, Isolated pools, Low flow, Moderate flow, High flow, Flood Water turbidity : Clear, Discoloured, Opaque, Silty Canopy cover : Open, Closed, Partially Open Embeddeddness : 0-25, 26-50, 51-75, 76-100

Continuous variables populated with dropdown/predicted text? Sorted alphabetically | Unit Aluminium | mg/l Ammonia nitrogen | mg/l Ammonium nitrogen | mg/l Bicarbonate | mg/l Biological Oxygen Demand | mg/l Calcium | mg/l Carbonate | mg/l Chemical Oxygen Demand | mg/l Chloride | mg/l Chlorophyll a | mg/l Conductivity | mS/m Depth | m Dissolved inorganic nitrogen | mg/l Dissolved organic nitrogen | mg/l Dissolved oxygen | mg/l Dissolved Oxygen: % saturation | % Fluoride | mg/l Iron | mg/l Kjeldahl nitrogen | mg/l Lead | mg/l Magnesium | mg/l Near bed velocity | m/s Nitrate nitrogen | mg/l Nitrite nitrogen | mg/l Particulate organic matter | mg/l pH |   Potassium | mg/l Silica | mg/l Sodium | mg/l Soluble reactive phosphate (= PO4-P) | mg/l Submerged macrophytes | % Sulphate | mg/l Temperature | deg C Total alkalinity | meq/l Total dissolved solids | mg/l Total inorganic carbon | mg/l Total nitrogen | mg/l Total organic carbon | mg/l Total organics in TSS | mg/l Total phosphorus | mg/l Total suspended solids | mg/l Turbidity | NTU Undercut bank | % Woody debris | %

Excel data file of the above attached. This is a simplified version of the abiotic datafile from Biobase. Abiotic Data Variables Master.xlsx

@dimasciput Please decide if this should form part of 2.6 or be held over for the next release.

dimasciput commented 4 years ago

@helendallas @jeremyshelton for the new abiotic form, let's held over for next release. For version 2.6, I will match the SASS form layout to be similar to other forms. What do you think?

helendallas commented 4 years ago

@dimasciput That is fine, but please can we make the next interim release soon in 2020, as we would like to be able to capture abiotic data with existing SASS data that is being captured.

helendallas commented 4 years ago

I am happy, thanks @dimasciput

helendallas commented 4 years ago

@dimasciput Abiotic data form - I cant see this anymore. I am sure you did it before release V2.6.

Additional abiotic data still needs to be pulled into SASS record, so that it is visible when ones clicks on the SASS record. Layout to be as per the form designed last year.....really cant find it.

Please advice

helendallas commented 4 years ago

@dimasciput please update me on this ticket

dimasciput commented 4 years ago

we have abiotic data form, it just we need to add new collection/sass record before go to abiotic form

helendallas commented 4 years ago

@dimasciput But what happened to the Categorical variables populated with a dropdown list | Dropdown list

Water surface width : <1; 1-2; 2-5; 5-10; 10-20; 20-50; 50-100; >100 m Water level : Dry, Isolated pools, Low flow, Moderate flow, High flow, Flood Water turbidity : Clear, Discoloured, Opaque, Silty Canopy cover : Open, Closed, Partially Open Embeddeddness : 0-25, 26-50, 51-75, 76-100

All we have a re the Continuous variables

image

see ticket Add Abiotic data form #1819

dimasciput commented 4 years ago

@helendallas yes haven't added that, because I was thinking that was for version 3. Can we please close this ticket then, and continue the discussion to other ticket instead.

helendallas commented 4 years ago

@dimasciput Okay, will do. To confirm - has the data in the Categorical variables above been imported from Rivers Database - but are currently not show yet? If this is the case then I will close the ticket as the data will be visible one Add Abiotic data form #1819 is done - Do I understand this correctly?

dimasciput commented 4 years ago

@helendallas yes, for example: http://freshwaterbiodiversity.org/admin/sass/sitevisit/137/change/?_changelist_filters=p%3D111, image

dimasciput commented 4 years ago

closing this ticket, the data has been imported but still not displayed. Continue to https://github.com/kartoza/django-bims/issues/1819