dxwell
commented
5 years ago @ajtroup Thanks for the feedback. Please see the comments below and we can discuss further.
The way it is described here, I do not think this is the right way to approach this and I am very concerned that there is not enough data captured. Why are we at a stage where this is being defined in the weeks before the solution vendor is being brought on? Surely what we want should have been specified in the tender, otherwise how do we know that the vendor can actually deliver a suitable system. If they’re just proposing to replicate SARIG for Queensland, that’s a backwards step of about 10 years, not a forwards step.
This concept of borehole data management has been discussed with, developed by, and supported by the GDMP subject matter experts group for the last 18 months.
The tender documentation and the vendor briefings presented the borehole requirements as they are defined here.
The focus of the design is to capture the primary metadata (the stuff that people use to search with) in a structured database. All other data is still captured in a structured way that can be read, updated, harvested, reported on. Just not using a relational database, instead using modern techniques such as document databases and columnar databases.
Terminology • While accurate, I have never seen anyone bother to describe a borehole as ‘a narrow shaft bored in[to] the ground.
• Water is a liquid
What is this comment referring to?
• Petroleum covers both oil and gas
What is this comment referring to?
• There are other synonyms missing for borehole (e.g. corehole)
Can you please provide a reference that mentions corehole? I can only see it as a colloquial term, not something that is defined in a standard.
Background • Actors?
A term used in business analysis and software development: "specifies a role played by a user or any other system that interacts with the subject." See https://en.wikipedia.org/wiki/Actor_(UML)
• GSQ’s borehole dataset is the point of truth for QLD government the others leverage off it
I don't think that is correct. e.g. water bores?
I understand that OGIA source their boreholes from multiple places and then perform QA that never makes it back to MERLIN.
The borehole register • From what is described overall in this document/page, the borehole register is not replacing MERLIN, it is only replacing a part of the MERLIN boreholes table and not even the minimum amount at that. It describes what amounts to the first two ‘screens’ in MERLIN borehole and has to expand beyond this. Otherwise, the solution is not fit for purpose.
See comments elsewhere in this response.
• Linked datasets should be those where data comes in an additional defined format – e.g. las or dlis for wireline logs.
Yes, e.g. a las file is a data resource that can be linked to.
• Core and cuttings intervals are primary attributes of a borehole
These intervals would be in the core and cuttings register as part if the samples database. Need to discuss this given what is captured in the reporting guidelines. This data has not been updated in MERLIN since August 2017.
• Sample intervals are primary attributes of a borehole that should then be linked to the results from the sampling/analysis
See above.
Simplifed data model diagram • Hylogger data is linked data, not result data
In the samples ontology, Hylogging is the observation activity. The results of the hylogging is results.
But, yes it will be recorded as linked data.
• Status is entity data (it’s the activity level of the borehole)
Yes, could be.
• Producing stats is probably more engineering data, maybe operational data
No, I think it is result data because it is the outcome of the well, not the inputs.
• Permit that the borehole is drilled on is primary metadata and can then be linked to further data available relating to the tenure. This should be attributed, rather than constantly derived, as you’ll have to take time as well as space into account.
Yes, agree. The model is intended to convey that, please let me know if it doesn't.
• Reports contain data, they aren’t data themselves
A report is considered a dataset. Where you see data it is typically meant as dataset
• Cores and cuttings are intervals, not results
These are samples, the enduring result of the one-off drilling event.
• Result includes what (if anything it discovered)
MERLIN has a DRILL_RESULT_CODE but nothing is captured in the new reporting guidelines. Let's discuss further.
• Are intervals engineering intervals? Or production intervals? Or sample intervals? Lithological intervals? Orientation (azimuth and inclination) intervals? Casing intervals? This is really ambiguous
Agree - need to disambiguate.
• From the general description, it seems that all you’re proposing is that the borehole register contains just the entity data – this isn’t enough.
The new software design will feature a relational database for the primary metadata, with the remaining metadata and data being stored as key-value pairs. For an example of this, load this borehole data extract into the online tool http://jsoneditoronline.org/.
This method of software design is modern, effective and efficient to both implement and change.
We should not be losing any data - we can capture anything as a key-value pair, but there is no need to replicate the MERLIN database structure or create a complex database system with many Create-Read-Update-Delete forms.
Conceptual data model • Purpose and sub-purpose would map to type and subtype and these don’t have a start and end date.
A start and end date is recorded for purpose and sub-purpose to track any purpose changes over the life of a borehole, e.g. is converted to a water bore.
Otherwise there are issues around confidentiality periods when they are defined by activity.
What issues are there? Aren't the dates based on rig release date for P&G?
The Status should have start and end dates (e.g. producing or not).
Yes, see status_date in Status table in the conceptual model (this would be implemented as start date and end date).
• Azimuth and inclination are likely to be a log in a petroleum or CSG well. Not sure if minerals and coal are running deviation logs, though I have seen some coal deviation logs.
Coal and mineral reporting templates capture both azimuth and inclination as:
Azimuth: The angle (in degrees) of clockwise departure from true north to the drillhole direction.
Inclination: The angle (in degrees) of drillhole deviation away from the vertical. 0 degrees inclination is horizontal and -90 degree inclination is vertical (downward).
Single field not enough for storing this.
I would think anything beyond the azimuth and inclination mentioned above would be in the geometry fields? What other data is there that requires something beyond a single field?
Borehole data elements • Need to have eastings and northings and zone for the location information too
Can do - however, Mineral reporting template does not capture zone (it's the only template that captures eastings and northings). Suggest we use the same input controls as were built for GEM.
• depth_datum would make more sense as origin_datum – much easier to see the association
Agree - does not align to standards, but they all vary anyway.
• Azimuth and inclination are likely to be an associated log for petroleum and csg wells
P&G Well Card has WELL_DESIGN only.
• Require rig release date for petroleum wells
Can add in.
Borehole data elements that are inferred
• Bhf_wireline_logs is not a good record of the wireline logs and should not be simply migrated o Wireline log information must be scraped from las files where possible and composite log scans where no las files are available
What is meant by "not a good record"?
• Bhf_borehole_survey_plan is not up to date
Noted. Last update was 16-Nov-16. Is there another up-to-date system of record?
• QDEX Reports number is not currently held in bhf_borehole_survey_plan – it should be in bibliography, but there’s a lot of mess in that space – had to be manual update. Link will need to come from QDEX system
bhf_borehole_survey_plan was a typo. QDEX comment noted. We can do a compare of data.
• Results field is a pain and is in desperate need of an update. It’s not a sample or observation though. It’s an interpretation of what was found in a well and is not particularly useful as there is an economic implication in the interpretation
This information is not being submitted in new reporting templates
Other borehole data elements
• QWCRN – needed
Will someone search for this metadata?
• Rig release date – needed for petroleum wells
Can add in.
• 306 have it because it’s more a reflection of how many wells have been hylogged
Suprised that is such a low number
• Total depth logger – needed
This is only being recorded for P&G in new reporting guidelines
• Perforation – this is engineering/production information
This information is not being submitted in new reporting templates.
• Comments – probably needs a lot of vigorous discussion and QA
Remarks are being captured in the new reporting templates, so can be captured, but would not be considered primary metadata.
ajtroup
The way it is described here, I do not think this is the right way to approach this and I am very concerned that there is not enough data captured. Why are we at a stage where this is being defined in the weeks before the solution vendor is being brought on? Surely what we want should have been specified in the tender, otherwise how do we know that the vendor can actually deliver a suitable system. If they’re just proposing to replicate SARIG for Queensland, that’s a backwards step of about 10 years, not a forwards step.
Terminology • While accurate, I have never seen anyone bother to describe a borehole as ‘a narrow shaft bored in[to] the ground. • Water is a liquid • Petroleum covers both oil and gas • There are other synonyms missing for borehole (e.g. corehole)
Background • Actors? • GSQ’s borehole dataset is the point of truth for QLD government the others leverage off it
The borehole register • From what is described overall in this document/page, the borehole register is not replacing MERLIN, it is only replacing a part of the MERLIN boreholes table and not even the minimum amount at that. It describes what amounts to the first two ‘screens’ in MERLIN borehole and has to expand beyond this. Otherwise, the solution is not fit for purpose. • Linked datasets should be those where data comes in an additional defined format – e.g. las or dlis for wireline logs. • Core and cuttings intervals are primary attributes of a borehole • Sample intervals are primary attributes of a borehole that should then be linked to the results from the sampling/analysis
Simplifed data model diagram • Hylogger data is linked data, not result data • Status is entity data (it’s the activity level of the borehole) • Producing stats is probably more engineering data, maybe operational data • Permit that the borehole is drilled on is primary metadata and can then be linked to further data available relating to the tenure. This should be attributed, rather than constantly derived, as you’ll have to take time as well as space into account. • Reports contain data, they aren’t data themselves • Cores and cuttings are intervals, not results • Result includes what (if anything it discovered) • Are intervals engineering intervals? Or production intervals? Or sample intervals? Lithological intervals? Orientation (azimuth and inclination) intervals? Casing intervals? This is really ambiguous • From the general description, it seems that all you’re proposing is that the borehole register contains just the entity data – this isn’t enough. Conceptual data model • Purpose and sub-purpose would map to type and subtype and these don’t have a start and end date. Otherwise there are issues around confidentiality periods when they are defined by activity. The Status should have start and end dates (e.g. producing or not). • Azimuth and inclination are likely to be a log in a petroleum or CSG well. Not sure if minerals and coal are running deviation logs, though I have seen some coal deviation logs. Single field not enough for storing this.
Borehole data elements • Need to have eastings and northings and zone for the location information too • depth_datum would make more sense as origin_datum – much easier to see the association • Azimuth and inclination are likely to be an associated log for petroleum and csg wells • Require rig release date for petroleum wells Borehole data elements that are inferred • Bhf_wireline_logs is not a good record of the wireline logs and should not be simply migrated o Wireline log information must be scraped from las files where possible and composite log scans where no las files are available • Bhf_borehole_survey_plan is not up to date • QDEX Reports number is not currently held in bhf_borehole_survey_plan – it should be in bibliography, but there’s a lot of mess in that space – had to be manual update. Link will need to come from QDEX system • Results field is a pain and is in desperate need of an update. It’s not a sample or observation though. It’s an interpretation of what was found in a well and is not particularly useful as there is an economic implication in the interpretation Other borehole data elements • QWCRN – needed • Rig release date – needed for petroleum wells • 306 have it because it’s more a reflection of how many wells have been hylogged • Total depth logger – needed • Perforation – this is engineering/production information • Comments – probably needs a lot of vigorous discussion and QA