Open aufdenkampe opened 7 years ago
@SRGDamia1, would you be able to create a relatively simple lookup table that Azavea could use for easy coding of unit conversion, while also adopting standard terms and abbreviations for ODM2 units and ODM2 unit types. I think the number of entries would be small; all we need are the current units used by MMW.
@emiliom might be able to provide us guidance based on Vizer. I would be happy to review.
There are several libraries in both Python and JavaScript that perform unit conversion. I imagine that we if implement this, we would take advantage of one of those libraries.
Some examples:
JS:
https://github.com/gentooboontoo/js-quantities https://github.com/ben-ng/convert-units http://mathjs.org/docs/datatypes/units.html
Python:
https://pint.readthedocs.io/en/latest/ https://pypi.python.org/pypi/units https://pypi.python.org/pypi/numericalunits
We do want to implement this, relatively soon, as it is a very frequently requested feature. However, I would like to do in as simple a way as possible, to save costs.
@caseycesari, A few years ago, we did an extensive review of all the well used Units systems and software libraries, and unfortunately most of them fall short, especially with respect to water quality units. The good news is that I don't think we need to implement generic unit conversion capabilities, and might be able to simply get by with a lookup table.
@SRGDamia1, would you be willing to make a first stab at such a lookup table, using ODM2 Units and UnitTypes (from http://vocabulary.odm2.org/units/ and http://vocabulary.odm2.org/unitstype/).
@ajrobbins, if it is handy, can you point us to a list of all the units that are currently in the system, or the code you currently have in place. If the code is all over the place, we can probably figure it out from using the app.
@aufdenkampe Just curious, when you say that most libraries fell short, do you mean that the libraries were missing some units, or that the unit conversions were done incorrectly, or both?
Also, is there any documentation of the unit conversion library review available? I'd be curious to take a look at it. Thanks.
The problem was missing units, not incorrect conversions. At least, I don't remember seeing any significant conversion errors. I didn't look through the python/java libraries, but a few years a go I spent a lot of time digging through systems like UCUM and QUDT. For most standard units, I imagine any of the libraries you mentioned would work. The problems start appearing when you are looking at less common derived units like isotope ratio delta values or converting between concentrations and densities (ppm to mg/L to mol/L to mg/kg, etc.. concentrations are a nightmare). But barring needing those, I imagine the simplest library that has all the units you need will be the best.
On 10/3/2017 8:50 AM, Casey Cesari wrote:
@aufdenkampe https://github.com/aufdenkampe Just curious, when you say that most libraries fell short, do you mean that the libraries were missing some units, or that the unit conversions were done incorrectly, or both?
Also, is there any documentation of the unit conversion library review available? I'd be curious to take a look at it. Thanks.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/WikiWatershed/model-my-watershed/issues/2264#issuecomment-333831457, or mute the thread https://github.com/notifications/unsubscribe-auth/AFIBjoE-q8Nlbudnv5zSu3e5AeUXj3_4ks5soi2egaJpZM4PbgU3.
-- Sara Geleskie Damiano Technician Stroud Water Research Center 970 Spencer Road Avondale, PA 19311 (610) 268-2153 x 248
@caseycesari, good questions. As @SRGDamia1 pointed out, the issue was that most of the unit systems we reviewed (in 2015) were for more physical systems, and they didn't contain the more chemical unit types used for water quality and environmental science. For example, for MMW, we will be converting from parts per million (ppm) to mg/L concentrations, which many systems can't do out of the box (because they technically don't have the same dimensionality).
The best system we found was QUDT (Quantities, Units, Dimensions and Types; http://www.qudt.org/) with it's water quality extension, but even that was incomplete and had some conceptual errors.
Our team built on that to create http://vocabulary.odm2.org/unitstype/, which is in a SKOS, and http://vocabulary.odm2.org/units/, which is unfortunately not in a SKOS.
You can see how we filled in blanks and merged the "best of the best" in our ODM2_UnitsTypeCV Google Doc and our ODM2_UnitsTable-Combined-QUDT-CUAHSI Google Doc.
This is a very often requested enhancement, recently requested again by Dave Arscott.
Here are some emails on the topic from Friday, 9/15:
From Anthony
From David T
From Arianna:
From Anthony: