XS Tool Preparation

[gmerritt123]

AS OF v2.html

• [x] Hovertool Updates: WellName displayed in place of sys_loc_code on:
  • Plan view Lith
  • Plan view Picks
  • Plan View Boreholes
  • XS Lith
  • XS Picks
  • XS WLs/Screens
  • [x] DESC lith field added to BHViewer hover
  • [x] WellName now displayed in BHViewer on click --> pull request for XS lib req'd to pass additional fields to pv_src in BHViewer
  • [x] Additional Materials added to mat lith tables in XS repo --> pull request
  • [x] 2020 Avg wls averaged (side script export from Hydrographs)
  • [x] "Kill switch" setup dev'd and added
  • [x] Downloaded and using most recent model from Karl (V3.dac)
  • [x] Truncated ROW lith to only Moraine area (i.e. locations NOT in cambridge model)

AS OF v3.html

• [x] Full QA of pick data:
  1. Identifying most recent pick for each sys_loc_code for each unit by parsing consultant_reference field to get a Date
  2. Identifying shallowest available pick for each sys_loc_code for each pick since some consultants picked multiple intervals across the hole for the same pick
  3. Identification of sys_loc codes where ii. is true and the duplicate picks are not in sequence with each other --> e.g. AFB1 is picked, then ATB2, then AFB1 again
  4. JL then looked over these and determined whether taking shallowest was appropriate and if not, overriding manually
  5. With this custom filter applied, only duplicate picks (same depth, same unit, same date) remain, which can be safely removed from the dataset

AS OF v4.html

• [x] URGENT: get missing borehole location data :

  • "1_BasicWellInfo_wRefElev_v12_UR_20220412_183325" from eWRAS provides XY locations for wells
  • "1_Reference_ELEV_v7_20220412_183332" from eWRAS provides reference elevations for wells
  • "1_SQL_X-YInfo_v4_20220518_200909" from eWRAS provides XY locations for all locations in eWRAS (boreholes and wells), but does not have ref elev.
  • Currently this approximately halves the total number of lithologies/picks we can display
  • Need to discuss with Karl to obtain this data
  • Karl: Use "Coordinates & Ground Elevations.xlsx" -->
  • With this applied:
    • 109 sys_loc_codes with lith_data missing location data
    • 1275 sys_loc_codes with loc_data missing lith_info
    • 4,852 sys_loc codes with both lith data and location data

• [x] Picks and Liths: Populated WellName with sys_loc_code if WellName absent

• [x] XS How to doc --> Joelle completed -- > GM completed minor edits and now up on team site accompanying XSv2

[gmerritt123]

As of v6.html

• [x] Incorporate Landfill lith data
  • [x] Stantec has prepped and provided table (email from Kelly)
  • [x] Address missing coords --> Correspondence with Kelly
  • [x] ID and inspect sys_loc_codes, lith, screens, GSElev, and coords for locations already already in eWRAS. ID any issues/discrepancies
  • [x] Merge the data, overwriting existing where appropriate based on inspection
• [x] Incorporate Parkway
  • [x] Stantec has prepped and provided table (email from Kelly)
  • [x] Address missing coords --> Correspondence with Kelly
  • [x] ID and inspect sys_loc_codes, lith, screens, GSElev, and coords for locations already already in eWRAS. ID any issues/discrepancies
  • [x] Merge the data, overwriting existing where appropriate based on inspection
• [x] Moved all this data prep work to separate script, pickling output for XS builder

Stantec Feature Requests --> v6:

• [x] Highlight production wells - Squares in plan view and bigger now

• [x] Highlight wells without picks - Red in plan view now means it has picks, Blue it does not

  • [x] Callback added such that color will change in plan view when you add well picks to a well that didn't before
  • [x] Same callback added to import export of picks

• [x] Extra column in pick table to mark "reviewed"

  • Added ability to add "reviewed_field" with bold conditional formatting applied to non zero values --> https://github.com/gmerritt123/XS/pull/32

• [x] GroupFilter/checkbox filter for Point vector groups

  • [x] Attribute sys_loc_codes by "Type" (Pumping, Observation, WWIS) and "Date" (Unknown,Pre 2009, Post 2008)
  • [x] All point vector based renderers (picks, lith, wells) on all plots are filtered by these checkboxes
  • [x] Incorporate into layout and add Div Titles for the two checkbox groups

• [x] Ability to add picks other than model layers

  • Won't have any color but will suffice (EDIT --> fixed in v7)
  • [x] Added AFC2, ATC2, ATG1
  • [x] Renamed Bedrock to Top of Bedrock

• [x] Ability to click items in XS and load them in XS:

  • Added add_xsfig_sel_cb method to codebase and incorporated --> https://github.com/gmerritt123/XS/pull/32

• [x] Some layout size adjustments/improvements

[gmerritt123]

As of v7.html

• [x] Incorporate WWIS lith data
  • [x] Compilation of data/extraction from DB
  • [x] Mapping of MAT1,MAT2,MAT3 to ROW material --> Completed by Jon
  • [x] Height transfer from DEMs to calculate elevations
    • [x] Develop tiling routine for massive raster w gdal
    • [x] Develop height transfer routine within tiling routine
    • [x] Heights transferred to Well_IDs (wwis_loc_gdf in Water Level Data Comp Notebook)
    • [x] Join to Jon's Lith Data table
      • [x] Post 2008 WWIS data only
      • [x] Omitting WWIS data if WWR is listed in eWRAS construction events
      • [x] Omitting WWIS data if within 25 m of eWRAS datapoint (even after the above two filters, there is considerable overlap due to eWRAS inconsistencies)
      • [x] Incorporate to XS tool w filters
• [x] Cambridge model build:
  • [x] Geounit color theme via previous reporting/Jinhui
  • [x] Routine to trim dataset to only Cambridge model area
  • [x] Map OB layers in Cambridge model --> Key provided by Jinhui
• [x] Improved pick color theme. Separate color theme between picks and layers --> Not all picks are model layers and vice versa. --> https://github.com/gmerritt123/XS/commit/f4391341e8286d1422cf83fc2ea018629c48fcd2
  • [x] Colors ascribed to every possible formation pick in EWRAS
  • [x] Available subset of picks to choose for new picks developed separately between Cambridge and Moraine model builds
  • [x] All color theming now occurs in the XS build script itself, not the data prep portion.

[gmerritt123]

As of v8.html

• [x] P16/Fountain St data included in Moraine XS
• [x] Update WLs with calibration dataset --> Moraine Model only
  • [x] Include Observation

[gmerritt123]

As of v9.html

• [x] Implementation of Smart Label on plan view map (https://github.com/gmerritt123/BokehUtil/commit/9277d5f290183abc21d5f47f3e8f93a6502eac2a)
• [x] Added Control Point Pick Observation Type

[gmerritt123]

AS of v10.html

Geophysics Data

• [x] Branch XS repo and add functionality to BHViewer class
• [x] Merge back to main --> relevant PR https://github.com/gmerritt123/XS/pull/35
• [x] Data Processing
  • [x] Processed Gamma data
• [x] Special marker for locations with geophysics data

Data Export

• [x] Relevant PRs --> https://github.com/gmerritt123/XS/pull/34 and https://github.com/gmerritt123/BokehUtil/commit/f68031925c39a87b7c4758c8a589398c21e7b704
• [x] Export all lith data
• [x] Export data currently on section

[gmerritt123]

As of v11.html

• [x] Removed checkbox filters for now due to required bug fixes
• [x] v6 of Moraine Model now to build surfaces --> Includes dundee N merge and fix of Baden update area perimeter
• [x] Relevant PR -> https://github.com/gmerritt123/XS/pull/36

Geophysics Data

• [x] replace CustomJSFilter bug with CustomJS + Index Filter routine
• [x] "Quiet Cockpit" gui improvement --> only show geophysics figure where we have it, otherwise hide from view
• [x] Replace <0 gamma values as nan (i.e. minor data cleaning step)

Picks Routine Bug Fixes -- relevant PR

• [x] Github issue and Discourse thread: https://github.com/bokeh/bokeh/issues/12227 + https://discourse.bokeh.org/t/filtering-cdsviewing-streaming/9359/1
• [x] Replace CustomJSFilter bug with CustomJS + IndexFilter
• [x] Clearing of pick table toggles table visibility too because of graphical display bugs
• [x] Update initialization of new pick source, drawing from lith data, hard coded initial values( for Omit, Comment etc.) and pv_src for auxiliary fields like well name etc.)
• [x] Update commit button CB, utilitizing CDS.stream and a more intuitive populating of the data to be appended
• [x] Implement Bryan (Bokeh founder) suggested workaround for the rendering issue
• [x] Revise main XS drawing callback to not mess with selection of point vector items --> was messing with things and never served any purpose anyway
• [x] Inefficient but effective callback for commit button if xs_obj is passed: will redraw the existing xs to get the pick filter updated on XS
  • Means the xs_obj passed requires a reset_button and a draw/don't draw xs button to exist.
• [x] Import of pick data to trigger same callback as above

Other

• [x] Correct the BHViewer's location dataframe
• [x] Hide visibility of plan view point vectors (only BHViewer Loc now)
• [x] Update datasets to only show locations that have lith data
• [x] Update Import XS line button to also trigger reset button prior to import/drawing the imported XS
  • XS line IO button requires a reset button now.
• [x] Update Reset Button to also reset the BHViewer
• [x] Export XS polygons --> export current xs geometries to geojson Relevant PR -->https://github.com/gmerritt123/XS/pull/37/commits/b15d197a942e808c3f9436e90b640dd333dbb885

[gmerritt123]

As of v12.html:

• [x] Implemented sys_loc_code lookup using new Bokeh_Util function --> https://github.com/gmerritt123/BokehUtil/commit/60002d2553ad6a5eddf2d2e7c309a13717d46182