Move away from eval in vocabulary mapping

[JessyBarrette]

Issue

For some variable vocabularies, an apply_func parameter is defined to derive a new variable from the existing one. The defined apply_func value is then evaluated via eval() to generate a new variable. This can create some security risks which should ideally be mitigated.

This has for objectif to help standardizing the different variables variance to a same standard. Here's a quick list of the different functions applied:

• lambda x: x/10: convert from S/m to mS/cm
• lambda x: x*22.319/31.998 convert oxygen from mg/l to umol/l
• lambda x: x*0.022391 convert form umol/l to ml/l
• lambda x: -1*gsw.z_from_p(x,ds.attrs['latitude']) derive depth from sea pressure
• lambda x: gsw.SP_from_SK(x) convert Knut salinity (g/kg) to practica salinity
• lambda x: gsw.t90_from_t68(x) convert temperature in IPTS-68 to ITS-90
• lambda x: 42.814*x/10 convert conductivity ration CRAT to S/m

The related vocabularies are the different DFO offices related ones:

• ocean_data_parser/vocabularies/dfo_ios_vocabulary.csv
• ocean_data_parser/vocabularies/dfo_nafc_p_files_vocabulary.csv
• ocean_data_parser/vocabularies/dfo_odf_vocabulary.csv

Fix

A few possibilities regarding how to fix this issue:

1. Create a dedicated function that essentially identify those problematic variables and apply a correction. The function can be either specific to each parser or more global. Likely the first may be easier to maintain in the future.
2. Attempt to still use eval or an equivalent while fixing the security issues.

@sjbruce feel free to add anything in there.

[sjbruce]

There are a few options that might be useful as a replacement for using eval() directly, one of the more promising modules is asteval: https://lmfit.github.io/asteval/index.html

It's primarily built to allow for the evaluation of mathematical expressions from a string, which would address the majority of issues with minimal modification.

It also has the ability to build custom symbol tables that appear to be able to accept custom functions as well, which may allow for importing of gsw functions. https://lmfit.github.io/asteval/api.html#utility-functions

This last point needs to be tested as it may be limited to numpy functions. If that doesn't work out the apply_func could be checked for "gsw." and then more explicitly execute the appropriate functions without using any form of eval().

[JessyBarrette]

I think this could potentially be useful if we had to do a lot of different equations. However, since we're only so far handling ~10 equations, maybe we can just create a simple function with the function as str, the dataset and related variable:

def apply_func(apply_func:str,ds:xarray, var:str):

    if apply_func == "lamba x: x*10":
        return ds[variable] * 10
    elif ...
       ...
    raise ValueError(....)

We could potentially reassess if needed. Thoughts?

[sjbruce]

True, it is a small list. We'll need tweak the CSV files to account for some small variations in the text here & there but that's fairly straight-forward.

Alright, let's do that first.