mguyette
commented
6 years ago Doug,
Thank you very much for your frank and thorough response to this request. We look forward to being able to get all calibration data into Aquarius efficiently using the Framework once it is available, but given the quantity of our data and the uncertainty of the timing of the changes, we definitely need to find a workaround.
I had already proposed writing directly into the CALIBRATIONCHECK table, inquiring about whether it would be at all feasible, but I was given similar advice as yours (granted without such a fantastic GIF).
We already coerce our field visit data into the HydroML format, which works well for almost everything. The exception is that it doesn't accommodate some parameters for calibrations. The schema appears to allow for working with "generic" parameters, but despite being entered into the XML, these calibrations do not show up in the calibrations table (and it also doesn't behave quite as we'd like for Dissolved Oxygen, one of the non-generic parameters). This is why we were hoping to use the Framework to get the rest of the calibrations in.
I believe our best bet is actually to store these calibrations in a separate Oracle table (formatted like the CALIBRATIONCHECK table) and use scripting to do the calibration drift calculations we need for those parameters. It's not ideal, because we'd really everything to reside in Aquarius, but we can't wait an indefinite amount of time to quality assure our data.
Again, thanks for your response and your advice.
DougSchmidt-AI
Hi Margaret. Thanks for raising this request.
Short answer
The short answer is "it can't happen now, but we are going to add it". It's on the roadmap, but I don't yet know what future AQTS release will support it.
Longer answer:
You've already figured out that the current data model doesn't let you add calibration check activities.
The long-term goal of this plugin framework is to support every activity and data type that can be entered in the Field Data UI page. But the framework just isn't fully complete yet.
The reason the framework lags behind the UI's capabilities is architectural. The UI ships in lock-step with the Time-Series server code. Both sides (the UI side and the server side) are free to completely change from release to release (even from one build to the next), and they always keep agreeing with each other, since they come from the same codebase. This allows the development team to quickly add new features each release without worrying about breaking any integrations.
But the framework is deployed separately from the Time-Series server software, so the plugins need to work against a stable interface that doesn't break from release to release. We may extend the interface to add new functionality, but we shouldn't break anything that exists. So framework features just take longer to add, since we need to wait for one of the quarterly AQTS releases to add the new activity support.
What workarounds are possible for you?
The only file format that currently supports adding calibration check activities is the USGS HydroML file
If you are able to coerce your field visit data into that format, then you'd be able to get it into AQTS. But the HydroML file format is a complex XML structure, so while this approach is technically possible, it isn't always simple.
If you are able to let a support rep know the type of field data you'd like to ingest, then maybe the transformation into HydroML might be straightfoward.
Another option would be to engage our Professional Services team to get something custom written. But that's not simple or cheap either (I'm not trying to upsell you here, that's always a bit rude). But if you have tens of thousands of calibration checks that simply MUST be in your system, then maybe an engagement might be worthwhile to your organization.
So that leaves you with the current situation. You can write a plugin that supports as many activities that are currently possible, and anything not supported will need to be manually entered.
The one workaround that is tempting, but should be avoided
It's super tempting to think "Why don't I just add rows directly to the database. I can see what I need to add".
That approach works until it doesn't. And then some spectacular failures can occur. It isn't worth the risk.
Direct DB access by anything other than a supported Aquatic Informatics product is a temptation that needs to be resisted.