Veeam won't back up VCH due to parsing error of guestinfo keys

[corrieb]

I've been running some tests using Veeam while putting together a backup story and it turns out that some of the characters we use in our guestinfo keys are confusing it. In particular it seems it doesn't like '/' or '|' and possibly '~' and '@'.

If we replace '/' with '_' and '|' with ':', it seems to be happy. However, the formatting used for these guestinfo keys is not special to VIC - it's derived from pkg/vsphere/extraconfig/keys.go and it appears that some of the characters have special meaning, such as "read-only".

Veeam is one of our customers most popular backup tools and customers are increasingly asking about how to back up a VCH.

It's looking like this is something we need to raise with Veeam, if the characters we're using are a standard format. We need to find a spec somewhere online that defines them.

If we were make a change, we hit big challenges with version compatibility. If we changed the key structure in one version, we need code that can continue to recognize the old key structure in previous versions. That would make such a change non-trivial.

I've tagged this as a security issue, since backup is a critical part of data security.

[anchal-agrawal]

@corrieb Should this be high priority? Marking as medium for now.

[emlin]

@corrieb couldn't actually found some where described the supported special characters for vm extra config key/value. Just the API document: https://www.vmware.com/support/developer/converter-sdk/conv60_apireference/vim.option.OptionValue.html. And '.' is mentioned to be used to represent hierarchy.

In reality, all those special characters worked well in VIC as the guestinfo key, and '/' is special, with '/' in the key name, in guest vmtools cannot change the value of that key.

@dougm if you know the document to describe the supported special character, please let us know. It's really hard to search vSphere document for this topic.

[dougm]

I don't know of such a document. If I edit a VM config parameters from the UI, add a new param with Key dougm.test and Value /usr/bin/cat ~/.bashrc | scp root@host

The UI doesn't complain and displays the saved value as-is above. The API also returns the same value. However, in the .vmx file itself: dougm.test = "/usr/bin/cat ~/.bashrc |7C scp root@host"

Note the 7C that follows the | - interesting, as %7C is the standard URL encoding for '|'.

The same happens if I add a config param with the same value using govc vm.change -e ...

However, none of those characters are escaped when used in the Key.

So if the UI permits these chars and they are not escaped within the .vmx files, certainly Veeam is doing something it shouldn't be.

[dougm]

Also note, if you use a special char such as = or " in the Key, results in the error: A specified parameter was not correct

If you use = in the Value that's fine.

If you use " in the Value, it is escaped as |22. Again, the URL encoding but | instead of % prefix.

[mdubya66]

making an enhancement vs a bug.

[zjs]

It's looking like this is something we need to raise with Veeam, if the characters we're using are a standard format. We need to find a spec somewhere online that defines them.

@corrieb: Do you happen to recall whether we ever investigated this and/or raised an issue with Veeam?

[zjs]

@lgayatri Do we have any capability to test Veeam?