UDUNITS-2 does not run on non-native Windows drives

[DPeterK]

On Windows, UDUNITS-2 fails if you try and use it on a drive that is not the one it is installed to. Below is a trace from the Windows command-line that demonstrates this. In this trace I have UDUNITS-2 installed to C: but it does not work if I try to use it from U::

[cubebrowser] U:\>udunits2 --help
Usage:
    udunits2 -h
    udunits2 [-A|-L|-U] [-r] [-H have] [-W want] [XML_file]

where:
    -A         Use ASCII encoding (default).
    -L         Use ISO-8859-1 (ISO Latin-1) encoding.
    -U         Use UTF-8 encoding.
    -h         Help.  Print this message.
    -r         Reveal any problems in the database.
    -H have    Use "have" unit for conversion. Default is reply to prompt.
    -W want    Use "want" unit for conversion. Empty string requests
               definition of "have" unit. Default is reply to prompt.
    XML_file   XML database file. Default is "C:\Users\peter.killick\App\Miniconda\envs\cubebrowser\Library\share\udunits\udunits2.xml".

[cubebrowser] U:\>udunits2
udunits2: Character encoding not specified and not settable from environment variables LC_ALL, LC_CTYPE, or LANG. Assuming ASCII encoding.
udunits2: Couldn't initialize unit-system from database "C:\Users\peter.killick\App\Miniconda\envs\cubebrowser\Library\share\udunits\udunits2.xml": No such file or directory

[cubebrowser] U:\>c:

[cubebrowser] C:\Users\peter.killick>udunits2
udunits2: Character encoding not specified and not settable from environment variables LC_ALL, LC_CTYPE, or LANG. Assuming ASCII encoding.
You have: K
You want:

This happens whether using UDUNITS-2 from the command-line (as above) or calling UDUNITS-2 within another application. The latter of these is tripping us up in the cf_units conda recipe for Windows.

[semmerson]

@dkillick Does the file C:\Users\peter.killick\App\Miniconda\envs\cubebrowser\Library\share\udunits\udunits2.xml exist?

[DPeterK]

@semmerson thanks for your interest in this issue - the file you asked about above does indeed exist. I guess this is what allows me to use Udunits so long as I'm on C: (which is quite badly shown in the trace in my initial post - perhaps I will edit that to make it a little clearer).

[QuLogic]

Some messages seem to be ignored, so can you run again with -r?

[semmerson]

@dkillick Is the environment variable "UDUNITS2_XML_PATH" set?

[DPeterK]

@QuLogic thanks for the suggestion, it's raised something quite interesting. On C: (the drive that UDUNITS2 is installed to):

udunits2: Character encoding not specified and not settable from environment variables LC_ALL, LC_CTYPE, or LANG. Assuming ASCII encoding.
Definition of "kt" in "\Temp\App\Miniconda\envs\iris192\Library\share\udunits\/udunits2-common.xml", line 105, overrides prefixed-unit "1000000 kilogram"
Definition of "microns" in "\Temp\App\Miniconda\envs\iris192\Library\share\udunits\/udunits2-common.xml", line 397, overrides prefixed-unit "1e-015 second"
Definition of "ft" in "\Temp\App\Miniconda\envs\iris192\Library\share\udunits\/udunits2-common.xml", line 508, overrides prefixed-unit "1e-012 kilogram"
Definition of "yd" in "\Temp\App\Miniconda\envs\iris192\Library\share\udunits\/udunits2-common.xml", line 517, overrides prefixed-unit "8.64e-020 second"
Definition of "pt" in "\Temp\App\Miniconda\envs\iris192\Library\share\udunits\/udunits2-common.xml", line 771, overrides prefixed-unit "1e-009 kilogram"
Definition of "at" in "\Temp\App\Miniconda\envs\iris192\Library\share\udunits\/udunits2-common.xml", line 1236, overrides prefixed-unit "1e-015 kilogram"
Definition of "ph" in "\Temp\App\Miniconda\envs\iris192\Library\share\udunits\/udunits2-common.xml", line 1866, overrides prefixed-unit "3.6e-009 second"
Definition of "nt" in "\Temp\App\Miniconda\envs\iris192\Library\share\udunits\/udunits2-common.xml", line 1875, overrides prefixed-unit "1e-006 kilogram"
You have: K
You want:

Meanwhile on U: (a different drive that UDUNITS2 is not installed to):

[iris192] U:\ > udunits2 -r
udunits2: Character encoding not specified and not settable from environment variables LC_ALL, LC_CTYPE, or LANG. Assuming ASCII encoding.
No such file or directory
Couldn't open file "\Temp\App\Miniconda\envs\iris192\Library\share\udunits\/udunits2-prefixes.xml"
parsing aborted
File "C:\Temp\App\Miniconda\envs\iris192\Library\share\udunits\udunits2.xml", line 11, column 42
udunits2: Couldn't initialize unit-system from database "C:\Temp\App\Miniconda\envs\iris192\Library\share\udunits\udunits2.xml": No such file or directory

Note the paths it's trying to load in each case (for example "\Temp\App\Miniconda\envs\iris192\Library\share\udunits\/udunits2-prefixes.xml"):

• they don't make sense for Windows machines as they're missing a drive letter,
• they're malformed - note the \/ between the path and the file.

My guess is that it implicitly works when run from the same drive as its installed to as Windows assumes the drive letter the path is relevant to is the current drive letter, which is only true in this particular case.

@semmerson does that give you something to go on? I checked for the UDUNITS2_XML_PATH and it does not seem to be set. It looks, however , like UDUNITS2 does know what its XML path is, as the XML path is set correctly in the help printout I included in the issue's description.

[semmerson]

@dkillick There appears to be some useful information there. I'm not sure when I'll get around to this, however, because of my work load and because the UDUNITS-2 library was designed to be used on a UNIX system (which seems to be the cause of this problem).

[semmerson]

@dkillick Set the environment variable UDUNITS2_XML_PATH to "C:\Temp\App\Miniconda\envs\iris192\Library\share\udunits\udunits2.xml" and retry.

[QuLogic]

I'm fairly certain the issue arises from here. _splitpath converts the original filename into drive/path/filename/extension, and only the path is applied as the "base" on this line.

Since udunits2.xml uses an <import> tag, that uses the base to get the full path to the imported file. Of course, since that full path doesn't have a drive letter, Windows tries to use the current working directory's drive to open it, which is incorrect.

I don't believe Windows cares about backslash vs. forward slash.

[QuLogic]

If that first link is changed to:

{
    char m_drive[_MAX_DRIVE];
    char m_dir[_MAX_DIR];
    _splitpath(base, m_drive, m_dir, NULL, NULL);
    (void)memmove(base, m_drive, _MAX_DRIVE);
    (void)strncat(base, m_dir, sizeof(base) - _MAX_DRIVE);
}

then this might work properly; I don't use Windows so I can't be certain.

[QuLogic]

BTW, this line does not look safe to me; there's no reason why the result from dirname should be large enough to contain sizeof(base) bytes and memmove does not look for NUL-termination, so it could be reading into invalid memory.

[semmerson]

@QuLogic I agree with your problem analysis. There's also this statement where the pathname of the file to be imported is constructed incorrectly under Windows because the directory pathname (from _splitpath()) contains a trailing backslash.

I also agree that the statement

(void)memmove(base, dirname(base), sizeof(base));

is not safe.

I've corrected these problems and the fix will be in the next release.

[QuLogic]

I don't believe the extra slashes matter, assuming stdlib's open goes through regular Windows channels; see the Canonicalizing Separators section here.

[semmerson]

@QuLogic You're right but I fixed it anyway.

[DPeterK]

@semmerson and @QuLogic - many thanks to you both. We will eagerly await the next release of udunits!

@semmerson - I tried setting the UDUNITS2_XML_PATH environment variable as you suggested. Unfortunately it did not solve the problem - apparently because it does not affect the path that udunits2-prefixes.xml is being searched for at:

> set | findstr UDUNITS
UDUNITS2_XML_PATH=C:\Temp\App\Miniconda\envs\iris192\Library\share\udunits\udunits2.xml

> udunits2 -r
udunits2: Character encoding not specified and not settable from environment variables LC_ALL, LC_CTYPE, or LANG. Assuming ASCII encoding.
No such file or directory
Couldn't open file "\Temp\App\Miniconda\envs\iris192\Library\share\udunits\/udunits2-prefixes.xml"
parsing aborted
File "C:\Temp\App\Miniconda\envs\iris192\Library\share\udunits\udunits2.xml", line 11, column 42
udunits2: Couldn't initialize unit-system from database "C:\Temp\App\Miniconda\envs\iris192\Library\share\udunits\udunits2.xml": No such file or directory

[semmerson]

@dkillick Try it now: ftp://ftp.unidata.ucar.edu/pub/udunits/udunits-2.2.21.tar.gz