FormatEntity returns string for SCPI command. Would like it to return double array.

[raphaelcervantes]

I have configured a node in my dripline runtime-config like so

      - name: na_s21_iq_data
        module: FormatEntity
        get_str: ":CALC:PAR:DEF S21; :CALC1:DATA:SDAT?"
        calibration: "[{}]"
        log_routing_key_prefix: 'na_sensor_double_precision_arr'
        log_interval: 30

The raw value is in a string array, and this is really problematic for inserting into my database. Is there a way to have this FormatEntity log a double array instead of a text array?

I looked into the source code for FormatEntity. https://github.com/driplineorg/dripline-python/blob/master/dripline/implementations/entity_endpoints.py#L90 I don't understand whatget_reply_floatdoes.

[laroque]

Um...:

• get_reply_float doesn't look like it does anything and should be removed (there was a dripline2 features that used it but it wasn't done well).
• You may be able to use extract_raw_regex, this uses a regular expression to extract the raw value from the full device return content. We don't support arbitrary content there though, it needs to get pulled in a single match.

The usual expectation is that the "value_raw" is either exactly the raw result reported from the devices, or extremely close to it. One question is if you're trying to log value_raw or value_cal? It looks like your value_cal is just the value raw wrapped in square braces; and like you're using a dedicated logger (per the existence of a custom log prefix). You could instead use the calibration to do the type conversion; if you're getting back ['1','2','3'] as value raw, your calibration could be '{float(item) for item in {}}' for example. Then when you configure the logger, map the value_cal payload key into your desired database column.

Other options would be to either write a custom class inheriting FormatEntity and have it do your extra conversion/parsing, or modify the logger to support type casting. I believe that sqlalchemy and/or postgres support a certain amount of implicit type casting, (if you had a string representation of a number in a variable and used that to populate a double precision column I think it would get cast for you, but I'm not sure if it goes as far as to try and convert array contents. The number of possibilities there is getting quite large and it is a bit hard to imagine how to add something in a generic way. But I'm open to suggestions.

[raphaelcervantes]

I'm putting my value_raw I think I'm going to use the calibration to caste the raw output into what I need it to be. I was also thinking about a keyword argument that would let me cast the output to whatever I needed it to be, but I don't know how I would implement that or if that would be good design.

'value_raw': '-1.04600565594E-004,+4.57950993568E-004,-1.69978202812E-004,-1.63903595102E-004,+1.15204875394E-004,+4.52699172091E-005,-4.66801287504E-005,-2.56885523173E-004,-4.43908296390E-004,-2.70173888709E-004,-1.69204512470E-004,+1.04284868153E-004,-3.36453185906E-004,-1.93761346585E-005,+1.31158410390E-004,-1.46643710047E-004,+1.09289097877E-004,-1.74496870971E-004,+6.94236533998E-005,-1.05078962205E-004,-4.24703851163E-004,+6.81723096335E-005,+1.47371161813E-004,+2.28639984725E-004,+2.33325816961E-005,-5.04767511731E-005,-4.32081529939E-005,+2.37072043893E-004,+3.89962617351E-005,-5.80958774181E-005,-2.26477899758E-004,-1.94973554986E-004,+3.31197736323E-005,-5.85429569946E-006,+6.57294426792E-005,+6.65448343615E-005,-3.51664571861E-004,+2.37102859173E-004,+2.82527599623E-004,+3.62836476544E-004,-8.07725444568E-004,+3.03516395873E-004,+2.71173655897E-004,-2.60293000281E-004,+4.13029014963E-004,+9.04426024893E-005,-1.34687418683E-004,-4.66330743418E-004,+2.41016626193E-004,-8.17540125593E-005,+6.72094557523E-005,-4.58820888574E-004,-5.10870773135E-004,+7.02939119036E-005,+1.14638252749E-004,-1.88264912369E-004,+1.86092747072E-004,-3.64314308693E-004,+4.02216253447E-004,-1.61101402716E-004,+1.52671712246E-004,-1.76391894274E-005,-1.66462288496E-005,+1.28070325568E-004,-1.62026097983E-004,-2.57254683930E-004,+3.81101619576E-004,-6.51343924624E-005,+9.40168867004E-005,-2.80493552323E-004,-9.05927190783E-005,+3.44904346016E-004,-1.35416374212E-004,+1.47461327690E-004,-5.22831722973E-005,-5.80053862050E-006,+7.81561014265E-005,-2.00181125645E-004,+1.84542173475E-004,-1.56195643139E-004,-9.40823616346E-005,+1.10828973116E-004,+2.02317260929E-004,-3.42752340051E-004,-1.61148840502E-004,+8.21317507462E-005,+1.51716424797E-004,-2.92013139693E-004,+2.36438704116E-004,+3.04451917629E-004,-2.27841158274E-004,+3.43830705205E-004,+1.10622844180E-004,+1.14376773068E-004,+6.67321752457E-005,-1.50778223599E-004,+1.97506953322E-005,+1.53606029148E-004,+9.64969734361E-005,+8.40308364134E-005,+3.32735025430E-004,-1.70396781469E-004,-1.64716743374E-004,-7.29721824862E-006,-3.04097380928E-004,-2.57060103533E-004,-1.18629744549E-004,+1.77476255331E-004,+5.16368773770E-004,+3.77057805156E-005,+3.90186885417E-005,-6.09286327267E-005,+1.34410104739E-005,+2.21727391503E-004,+2.65201227229E-005,+7.98855489708E-006,+6.43802900723E-005,-2.92946267316E-004,-6.19817879087E-006,+4.88606336206E-004,+1.67712298871E-004,+2.58661685627E-004,-1.38006856818E-004,+2.44081189123E-005,+3.27397575678E-004,-1.74054060967E-004,-2.59466060585E-005,-5.91154357068E-004,-2.69849750011E-004,-5.34190848097E-005,+6.37639754728E-005,-3.06714824513E-004,+2.93110021653E-004,+1.57546823997E-004,-3.83509112563E-005,+2.48870510032E-004,+4.66046152891E-004,+5.37255274526E-005,-2.19825123817E-004,-1.94204242999E-004,-3.61401895951E-004,-2.62308452043E-004,+2.95106843906E-004,-2.09152900316E-004,-2.05019939410E-004,-3.57323100417E-004,+2.22124289824E-004,-1.71387638378E-005,+1.09851794262E-005,+1.65558842999E-004,-2.35793335631E-005,-1.05706572763E-004,-4.82394872162E-004,+1.91961198299E-004,+2.62683537792E-004,-2.32812934593E-006,-6.28026088963E-005,-6.89377426421E-005,-2.25028848468E-004,-6.11049126689E-005,+1.94295459372E-005,-3.91233851473E-004,+3.08134434977E-004,+3.03583874782E-004,+2.48059502607E-004,-1.19282164390E-004,+1.81497864630E-005,-2.60972517744E-004,+2.43367112150E-004,-5.41505656235E-005,+1.91052550701E-004,-1.49667420225E-004,-1.97449119760E-004,-5.16781653562E-004,+1.70129252567E-005,+5.94268030272E-005,+2.93927679567E-004,-2.11549901164E-004,+2.32987458731E-005,-3.31140737757E-004,-2.29748575375E-004,+5.20189114871E-005,-2.14524750952E-004,-6.71379611022E-005,+2.96509401539E-004,-5.56906452473E-004,+1.91470490354E-004,-4.28275080676E-004,-1.08395884097E-004,-1.63520584956E-004,-1.85279560283E-004,-2.31539954844E-004,-1.66356224168E-004,-8.94490589840E-005,-7.36538462919E-005,-7.19794312360E-004,-9.80899626241E-005,-3.31162842369E-005,+1.90449240406E-004,-2.92110322099E-004,-6.77342146336E-004,-8.16708839139E-005,+1.21132266887E-004,-3.81501427871E-004,+3.70957688718E-005,-3.97091090680E-004,-5.57927281420E-005,+1.89870497151E-004,-2.73392681753E-004,+1.18500229272E-004,-3.11724776094E-004,-2.22875256682E-004,+6.78777484239E-005,+3.25106193310E-004,+9.45591349793E-005,+2.16744914974E-004,-1.31363505780E-004,+5.88122256394E-005,-5.06400475608E-004,+4.35080443060E-004,+1.97262335191E-004,+4.57093018500E-005,-1.25959680100E-004,+1.55671395372E-004,-1.80751951619E-004,-2.66808052946E-004,-1.78419504256E-004,-6.05807990443E-004,-3.52640015920E-004,+2.52416929691E-004,+3.10521517443E-004,-7.16179103117E-005,+3.63199626064E-004,+3.83259288789E-004,+1.74781807665E-004,+2.96512530051E-004,-4.65031277389E-004,+1.50602031628E-004,+3.42880748380E-004,-9.45545295711E-005,-6.80931387259E-004,+7.20823081499E-005,-2.65439266766E-004,+1.08952147549E-004,+1.00023141488E-003,-4.59002350635E-004,-2.58155045641E-004,-4.44452887741E-004,+1.50049677574E-004,+1.37216273963E-004,+2.03123856890E-004,+3.69738556136E-004,-2.52269778956E-004,+9.86254203073E-005,+2.42602000912E-005,-2.68173435267E-004,+1.97548696638E-004,-1.08093139834E-004,+4.58341643710E-004,+8.05680495339E-005,-8.24457731919E-004,-8.84272322468E-004,-6.45756760166E-004,-2.00322590441E-004,+1.41491079800E-004,+4.41070649131E-004,+3.35187805984E-004,-1.86960516847E-004,+5.15401688494E-004,+6.99166063286E-005,+3.99416725214E-004,+2.10891823179E-004,+6.84480425585E-004,-1.27353217439E-004,-3.54972932568E-004,-1.63535347202E-004,+2.34387385164E-004,-8.12777207714E-005,+3.24300281676E-004,+1.34578128937E-004,-7.68559129416E-005,+3.96600037328E-004,+7.32522756822E-005,-7.63588594293E-004,+1.91093100020E-004,+1.28181411665E-005,-1.91761446983E-005,+2.24644915500E-004,+1.25627953181E-004,-1.59897972593E-004,-2.26687532322E-004,-2.60474829340E-004,-4.49839310183E-004,-2.13466595069E-004,-1.21916890485E-004,+2.41636095545E-004,+1.06074438735E-005,+6.82107238636E-005,-2.86867708062E-005,-1.41560513069E-004,-7.38120736930E-004,-2.52241200909E-004,+3.46569659128E-004,-5.33252943434E-005,-6.41399906701E-005,+2.95717129532E-004,-4.26665310631E-004,+1.39521677470E-004,-7.82817816298E-005,+3.54129813180E-004,+8.07560369657E-005,-2.86148135718E-004,-2.45360708011E-004,-3.19062506537E-004,+1.02081786669E-004,+6.69456506664E-005,+2.18596120698E-004,-2.95197228515E-004,-3.91843119114E-004,+1.92183615334E-004,-5.89089585393E-004,-2.84163516053E-004,+3.47874347598E-004,-4.44310391669E-004,-1.26799291878E-004,+2.50547682956E-004,-2.48168001075E-005,+2.73930702833E-004,+1.89568338925E-004,+1.73028749953E-004,-1.68676928853E-004,-7.75401130221E-004,+2.00912483052E-004,+1.27324083097E-004,-1.77043009601E-004,-2.68704897606E-004,+1.06850859883E-004,-2.26832762511E-004,+2.88448270535E-004,-1.53044637974E-004,+3.04697917724E-004,+1.28872601435E-004,+1.57744252421E-004,-3.98160072717E-004,+2.43497021412E-004,+1.67425807868E-005,+2.05729430335E-004,-4.45906657831E-004,+1.11124555763E-004,+3.29210940064E-004,+2.62710014821E-005,-4.98754727091E-004,-1.62262735818E-005,+6.20150843493E-005,+1.87541083537E-004,+7.48402151150E-004,-3.27175946167E-004,+4.49505561660E-004,-5.77606813653E-004,+3.43866354319E-004,-9.44671737638E-006,+4.71272936663E-004,+1.00844934685E-004,+5.38075522098E-005,+4.17214239368E-004,+4.69314277671E-004,+6.10755677395E-004,+2.55006533737E-004,+6.90471845050E-004,+4.86611790829E-005,+1.92548482196E-004,-3.99394580219E-004,+2.05217896600E-004,+5.74769792575E-004,-6.37091253086E-004,-1.60040368536E-004,+1.02229281753E-004,+3.65655148156E-005,+7.15881778367E-004,-3.56156609547E-005,-5.40992745802E-004,-2.11408847504E-004,+1.26413984155E-004,+2.26424984447E-005,+6.11961950117E-004,+1.87747485016E-004,-2.73956129739E-004,-7.17072872042E-004,+5.45591271399E-004,+3.83086313981E-004,-3.54048057189E-004,-1.24618848651E-004,+1.81528576227E-004,-7.69284995946E-004,-4.88247470071E-004,-5.35006797721E-004,+2.68121207441E-004,+1.78873513963E-004,-2.15658295155E-004,+5.30290974173E-005,+9.04428985035E-005,-9.52146920144E-005'}