search

RDCEP / EDE

MIT License
2 stars 1 forks source link

Operations and units #10

Open njmattes opened 8 years ago

njmattes commented 8 years ago

When we let the user make arbitrary operations on multiple datasets, we're going to have to make that same operations on the units. I think we'll have to use something like sympy to operate on units in tandem with the numpy operations on the numeric arrays.

ghost commented 8 years ago

not sure exactly why we would need sympy since we are doing the operations within the database. to align 2 rasters spatially we will use ST_{Resample,Transform,...}. aligning temporally is a little trickier and im working on it, also what extra tables / columns we might need for that...

njmattes commented 8 years ago

If I'm allowing the user to divide dataset A by dataset B, and A's units are t * ha^-1 * yr^-1 and B's units are w * m^-2 * yr^-1, then I need to be able to tell them the unit of A / B. If these operations are essentially arbitrary, is the database capable of determining the units?

ghost commented 8 years ago

no, it isn't, you're right. then we will do it as follows obviously: do the ordinary operation on the actual data in netcdf_data and add the produced new raster to that table and w.r.t. metdata, in particular the units, we read them into python, use sympy, and create a new row in netcdf_meta (with the right unit in particular) for that new raster. all of that in one transaction of course.

njmattes commented 8 years ago

It may also have implications for ingestion. I'm not sure if there's any real standard for declaring units in a NetCDF file. Joshua's are eg w m-2 yr-1, but that could also be eg w/m**2/yr. Either before or after ingestion, we'll have to have some way to parse those units into sympy.Symbols. sympy may not be the best library for this, it's just the one I'm most familiar with.

ricardobarroslourenco commented 8 years ago

@njmattes as we talked this morning, there is a library for doing physical quantities manipulation named pint(https://pint.readthedocs.org/en/0.7/index.html). On my point of view, we should do a physical evaluation prior to processing, and the user would be prompted to agree if a weird resultant unit prompts.

sympy seems to be very useful for calculations, after the prior unit evaluation is done. It reminds me a lot of Numerical Recipes for C