UTF-8 decode error caused by hUnitsArray

[amos-jk]

Hello,

Description

I have an issue with the encoding of hUnitsArray.raw as it is returned from the shared library (Linux) version of RefProp. It appears to fail in the trim function, called from ALLPROPSdll. See the traceback below

Traceback:
  ...
  File "/work_dir/refprop/routine.py", line 128, in inlet_props
    output_l = dll.ALLPROPSdll("H,V,M,Z,CP,CV", SI, 0, 0, t1, dl, x)
  File "/usr/local/lib/python3.10/site-packages/ctREFPROP/ctREFPROP.py", line 770, in ALLPROPSdll
    return self._ALLPROPSdlloutput_tuple(to_double_array(Output),trim(hUnitsArray.raw),to_int_array(iUCodeArray),ierr.value,trim(herr.raw))
  File "/usr/local/lib/python3.10/site-packages/ctREFPROP/ctREFPROP.py", line 39, in trim
    return s.replace(b'\x00',b'').strip().decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa8 in position 18: invalid start byte

It seems to be fixable if I edit the ctREFPROP files in use to swap utf-8 for something like ISO 8859-1. Making this substitution seems to allow the library to run as expected. Though I didn't validate the results were correct just that they were returned.

Which leaves me questioning if the issue is in ctREFPROP, or if I just have an incompatibility between the wrapper and the library.

Happy to supply build files or the library if needed to try and track this down.

Versions

REFPROP Version: 10.0 (it doesn't report patch number) Operating System and Version: debian:bullseye-slim via 3.10-slim-bullseye container Access Method: Python using ctREFPROP v0.10.2 from 24 Aug 2021 09:11:51

[ianhbell]

Having never seen this problem before, I am not sure what to say. I run on OSX with this similar config without any problems. Let me have a go with this precise setup (note we have docker containers set up for building REFPROP that can also be used for testing it).

It sounds like a memory corruption somewhere that has resulted in one of the output strings having non-ascii contents.

What is the simplest example that demonstrates this problem? That would be helpful for me to debug.

[wandadars]

I just ran into this bug on my machine

Ubuntu 20.04, Python 3.10, REFPROP10

I was able to get ALLPROPS20dll to work, but not ALLPROPSdll. I passed the code for both to GPT-4 and provided this observation.

Certainly! I've compared the two wrapper functions, ALLPROPS20dll and ALLPROPSdll, and here are the differences I found:

Buffer sizes for hOut:

ALLPROPSdll: hOut = ct.create_string_buffer(hOut.encode('utf-8'),10000)
ALLPROPS20dll: hOut = ct.create_string_buffer(hOut.encode('utf-8'),255)
Buffer sizes for hUnitsArray:

ALLPROPSdll: hUnitsArray = ct.create_string_buffer(10000)
ALLPROPS20dll: hUnitsArray = ct.create_string_buffer(1000)
Output array sizes:

ALLPROPSdll: Output = (200*ct.c_double)()
ALLPROPS20dll: Output = (20*ct.c_double)()
iUCodeArray sizes:

ALLPROPSdll: iUCodeArray = (200*ct.c_int)()
ALLPROPS20dll: iUCodeArray = (20*ct.c_int)()
The main differences between the two functions are related to the buffer sizes and the output array sizes. The ALLPROPSdll function is designed to handle a larger output array and unit array compared to the ALLPROPS20dll function.

The bug you encountered earlier might be related to these differences in buffer sizes, causing unexpected behavior when the data doesn't fit into the expected array sizes. It's also possible that there is a bug within the REFPROP library itself, but without further investigation, it's hard to pinpoint the exact cause.

This snippet contains the code that caused the error to show up.

def get_mixture_critical_properties(mixture_fluids, mixture_composition, units, rp):
    # Set up the mixture
    #ncc, hFiles, z, ierr, herr = rp.SETMIXdll(mixture_fluids, "HMX.BNC", "DEF")
    #if ierr != 0:
     #   print(f"Error in setting up the mixture: {herr}")

    # Set up splines for the mixture
    ierr, herr = rp.SATSPLNdll(mixture_composition)
    if ierr != 0:
        print(f"Error in setting up splines for the mixture: {herr}")

    # Get the critical properties of the mixture
    z = mixture_composition + [0.0] * (20 - len(mixture_composition))
    #output, _, _, ierr, herr = rp.ALLPROPSdll('TC;PC;DC', units, 0, 0, 1, 1, z) #Doesn't work
    output, _, _, ierr, herr = rp.ALLPROPS20dll('TC;PC;DC', units, 0, 0, 1, 1, z)

    if ierr != 0:
        print(f"Error in getting critical properties: {herr}")

    T_critical, P_critical, rho_critical = output[:3]

    return T_critical, P_critical

mixture_composition is a dictionary with {'MMH': 0.5, 'NTO': 0.5} mixture_fluids is a string 'MMH * NTO' and units is the integer 20 (which is the molar SI units flag I believe) rp is the object returned from calling: rp = ct.REFPROPFunctionLibrary(os.environ['RPPREFIX'])

[ianhbell]

Can you give me a very very simple runnable example with REFPROP 10 that demonstrates this problem?

[wandadars]

Sure thing. Do you know why the print statment returns a version number that is just ** in that code?

import os
import ctREFPROP.ctREFPROP as ct

def setup_refprop():
    #Refprop setup
    rp = ct.REFPROPFunctionLibrary(os.environ['RPPREFIX'])  
    rp.SETPATHdll(os.environ['RPPREFIX'])
    units = rp.GETENUMdll(0,'MOLAR BASE SI').iEnum
    print('REFPROP Version: ' + str(rp.RPVersion()) )

    #Setting Peng-Robinson EOS for all fluids
    flags=rp.FLAGSdll('PR',3)
    print('Flags: ', flags.herr)
    return rp, units

def get_mixture_critical_properties(mixture_fluids, mixture_composition, units, rp):
    # Set up the mixture
    #ncc, hFiles, z, ierr, herr = rp.SETMIXdll(mixture_fluids, "HMX.BNC", "DEF")
    #if ierr != 0:
     #   print(f"Error in setting up the mixture: {herr}")

    # Set up splines for the mixture
    ierr, herr = rp.SATSPLNdll(mixture_composition)
    if ierr != 0:
        print(f"Error in setting up splines for the mixture: {herr}")

    # Get the critical properties of the mixture
    z = mixture_composition + [0.0] * (20 - len(mixture_composition))
    output, _, _, ierr, herr = rp.ALLPROPSdll('TC;PC;DC', units, 0, 0, 1, 1, z) #Doesn't work
    #output, _, _, ierr, herr = rp.ALLPROPS20dll('TC;PC;DC', units, 0, 0, 1, 1, z)

    if ierr != 0:
        print(f"Error in getting critical properties: {herr}")

    T_critical, P_critical, rho_critical = output[:3]

    return T_critical, P_critical

if __name__ == "__main__":
    rp, units = setup_refprop()
    get_mixture_critical_properties('R32 * R125', [0.5,0.5], units, rp)

[ianhbell]

Because you need to upgrade to 10.0.0.02. Shoot me an email (ian.bell@nist.gov) for that.

[ianhbell]

This is a memory issue in REFPROP that has been fixed in the development version