Channel Conversion Error with Exponentiation

[markwaterbury]

Python version

('python=3.10.11 (tags/v3.10.11:7d4cc5a, Apr 5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)]') 'os=Windows-10-10.0.19045-SP0' 'numpy=1.25.2' 'asammdf=7.3.14'

Code

MDF version

4.11

Code snippet

from asammdf import MDF
m=MDF('asammdf_formula_issue.mf4')
m.select({'act'},raw=False)

Traceback

Traceback (most recent call last):
  File "C:\Program Files\Python310\lib\site-packages\asammdf\blocks\v4_blocks.py", line 3278, in convert
    values = evaluate(self.formula.replace("X1", "X"))
  File "C:\Program Files\Python310\lib\site-packages\numexpr\necompiler.py", line 943, in evaluate
    raise e
  File "C:\Program Files\Python310\lib\site-packages\numexpr\necompiler.py", line 851, in validate
    _names_cache[expr_key] = getExprNames(ex, context)
  File "C:\Program Files\Python310\lib\site-packages\numexpr\necompiler.py", line 714, in getExprNames
    ex = stringToExpression(text, {}, context)
  File "C:\Program Files\Python310\lib\site-packages\numexpr\necompiler.py", line 300, in stringToExpression
    ex = eval(c, names)
  File "<expr>", line 1, in <module>
TypeError: unsupported operand type(s) for ^: 'OpNode' and 'int'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Program Files\Python310\lib\site-packages\asammdf\mdf.py", line 3514, in select
    signal.samples = conversion.convert(signal.samples)
  File "C:\Program Files\Python310\lib\site-packages\asammdf\blocks\v4_blocks.py", line 3280, in convert
    values = evaluate3(self.formula.replace("X1", "X"))
  File "C:\Program Files\Python310\lib\site-packages\numexpr\necompiler.py", line 943, in evaluate
    raise e
  File "C:\Program Files\Python310\lib\site-packages\numexpr\necompiler.py", line 851, in validate
    _names_cache[expr_key] = getExprNames(ex, context)
  File "C:\Program Files\Python310\lib\site-packages\numexpr\necompiler.py", line 714, in getExprNames
    ex = stringToExpression(text, {}, context)
  File "C:\Program Files\Python310\lib\site-packages\numexpr\necompiler.py", line 300, in stringToExpression
    ex = eval(c, names)
  File "<expr>", line 1, in <module>
TypeError: unsupported operand type(s) for ^: 'OpNode' and 'int'

Description

File sent via email. It appears that there is an issue processing certain formulas with algebraic equations which use the power operator ^. The issue seems to only be related to the channel conversion, as reading the raw data works without issue:

m.select({'act'},raw=True)

and shows the the conversion formula is (X/4600)^2+10. Other channels which do not use ^ in the formula (ex. X/128) are able to be read without issue. Attempting to plot in the GUI gives the same error. I did not create this file or the channel/formula in question, but am questioning whether this file breaks the MDF spec in some way or if the operator is not correct. Regardless, is it possible for this formula to be processed?

[danielhrisca]

@markwaterbury please try the development branch code

[danielhrisca]

I think that the usage of the ^ operator for the power is not correct. According to the standard this should be used for XOR .

However if the formula is computed using the power operator, then the signal looks ok

CANape 18 on the other hand displays a different value

[markwaterbury]

@markwaterbury please try the development branch code

On the development branch ('asammdf=7.3.15.dev16'), I can read the signal without error now, but it looks like it is just returning the raw values, even with raw=False

[danielhrisca]

I did some changes, you need to install sympy as fallback for computing the algebraic conversion

[markwaterbury]

I just noticed that, and installed it, but the numbers being output aren't correct now. For instance, the first sample -8602 should evaluate to 13.4969, but I am seeing 10.00020132. The last sample, -5506, should evaluate to 11.4327, but I am seeing 9.99871777.

[danielhrisca]

Those are the values shown by ATI Vision?

[markwaterbury]

Yes, 13.4969 and 11.4327. Which are also the numbers I get when calculating by hand.

[danielhrisca]

I understand, unfortunately the formula is written with the wrong operator acording to the specs ( ^ instead of ** )

[markwaterbury]

Interesting, sympy is capable of processing this formula without issue

from sympy import *
str_expr="(x/4600)^2+10"
expr=sympify(str_expr)
expr.subs('x', -8602).evalf()

returns 13.4969000000000

[markwaterbury]

I understand, unfortunately the formula is written with the wrong operator acording to the specs ( ^ instead of ** )

I see this in the General Expression Syntax spec now. So to follow the spec, asammdf would need to treat this operator as a bitwise xor. So while it won't be the result I am expecting, at least no error will be thrown.

[danielhrisca]

You could also programatically replace the operator inside the conversion rule

from asammdf import MDF
import asammdf.blocks.v4_constants as v4c

mdf = MDF(r'asammdf_formula_issue.mf4')
for gp in mdf.groups:
    for ch in gp.channels:
        if ch.conversion and ch.conversion.conversion_type == v4c.CONVERSION_TYPE_ALG:
            ch.conversion.formula = ch.conversion.formula.replace('^', '**')

print(mdf.get('act').samples)

[danielhrisca]

according to https://docs.sympy.org/latest/modules/core.html the default value for convert_xor is True so the operator is used as power not as XOR