search

autopkg / hansen-m-recipes

My recipes for Autopkg - https://github.com/autopkg
27 stars 50 forks source link

GoogleChrome-win BigFix Relevance unable to get version #47

Closed chemxboy closed 6 years ago

chemxboy commented 8 years ago

Looks like when the Google Chrome msi is extracted, the "[5] Summary Information" does not contain the string looked for. It might be an issue with the extraction or the file itself as when extracted with 7zip as the character set seems off when viewed in a text editor.

hansen-m commented 8 years ago

I noticed this issue the other day and it seems that QnA doesn't like a certain character in the only file in the msi that contains the version number. Any attempt at lines of file... causes the error below. Using the MSIVersionProvider processor might be an option but would take some additional work and isn't ideal for those who don't have wine available. Another option might be a custom Python processor. Will keep working on this...

CharacterSetResult: FileLineLoop::TranscodeHelper: UErrorCode=12 [U_ILLEGAL_CHAR_FOUND] in ucnv_toUChars ["\xD8\x00"].

cc: @jgstew

jgstew commented 8 years ago

Can you provide the part of the file that contains the version info that QnA is trying to read?

Do you know which version of QnA is being used? Does this happen with 9.5+? Does it happen with 9.2 or earlier?

These seem to be the lines with relevance that is the cause of the issue:

This is the relevance: (I'm pretty sure)

preceding text of first " Copyright 2011 Google Inc." of substrings separated by "%00%00%00" whose (it contains "Google Inc.") of lines whose (it contains "Installer") of file "%RECIPE_CACHE_DIR%/InstallerExtract/[5]SummaryInformation" as version

@bigfix

jgstew commented 8 years ago

What does this relevance return?

lines containing " Copyright 2011 Google Inc." of files "Desktop/Chrome/[5]SummaryInformation" of folders of folders "/Users"

What about this?

lines whose (exists it AND it contains " Copyright 2011 Google Inc.") of files "Desktop/Chrome/[5]SummaryInformation" of folders of folders "/Users"

chemxboy commented 8 years ago

I can't paste in the contents of the file in the web form. Probably due to the the character set. But here is a screenshot...

contents

I also tried changing the relevance from looking at the extracted to just looking at the raw msi and I got an "unknown error" from QnA on OS X. No error is returned from Windows debugger and the version number is returned as expected. Workaround might be complicated by a bug in QnA.

(Just checked with an older version of QnA on OS X: 9.1 .1117.0 works as expected. I'll try and find which version it stops working)

jgstew commented 8 years ago

That is interesting.

So if you have the same file on a Windows machine, then the relevance works correctly?

How strange. That definitely points to a bug in QnA or in the relevance processing on the Mac in general.

I wonder if you have the relevance evaluated through an analysis property on the Mac if it would also work, or if it would fail the same way. This would be a way to test if the issue is with QnA or relevance on the Mac in general.

In either case, do either of the relevances I provided above work on the Mac?

This may be something that Alan from IBM will have to look into since it seems like a deeper problem.

chemxboy commented 8 years ago

No, the same extracted file on windows does not work. I was talking about running the relevance on the raw msi instead of the extracted file.

chemxboy commented 8 years ago

I will test the relevance for looking at the msi in a property now.

Update: Looking for the string in the raw MSI has on OS X, and the expected result "51.0.2704.84" on Windows.

jgstew commented 8 years ago

That is interesting that it works when run against the raw MSI file itself.

I was referring to testing the relevance looking at the extracted file on the mac:

lines containing " Copyright 2011 Google Inc." of files "Desktop/Chrome/[5]SummaryInformation" of folders of folders "/Users"

It would be interesting to know if the 2 I proposed above work either through QnA or through a property, as well as the original relevance.

chemxboy commented 8 years ago

That has no result/no errors. Probably due to the fact that it cant find any lines that match.

hansen-m commented 8 years ago

As a simple workaround for now, I've updated the BESRelevanceProvider to not overwrite an existing variable if it fails. So the key can be provided as an input or override:

autopkg run -v --key version="51.0.2704.84" GoogleChrome-Win.bigfix

jgstew commented 8 years ago

In another channel, @hansen-m said "The last line of the file contains the version number for Chrome".

Is it always the very last line? That could actually make this work.

This relevance checks the last 2 lines only:

lines whose( line number of it > ((it - 3) of number of lines of files "Desktop/Chrome/[5]SummaryInformation" of folders of folders "/Users") AND it contains " Copyright 2011 Google Inc.") of files "Desktop/Chrome/[5]SummaryInformation" of folders of folders "/Users"

Does this relevance return anything?

If the relevance is failing on earlier lines in the file, then this might be a solution.

hansen-m commented 8 years ago

Depends on the editor or how the file is read maybe, but either last or second to last line.

jgstew commented 8 years ago

So, I was just testing this in @rustymyers ' QnA GUI on a Mac, and it didn't seem to have a problem at all when using the file downloaded from slack. I guess I'd have to test it using AutoPkg to really have a better test.

Just the version:

Q: preceding text of first " Copyright 2011 Google Inc." of substrings separated by "%00%00%00" whose (it contains "Google Inc.") of lines whose(line number of it = number of lines of files "/Users/username/Downloads/[5]SummaryInformation") of files "/Users/username/Downloads/[5]SummaryInformation"
A: 51.0.2704.84

All lines:

Q: lines of files "/Users/username/Downloads/[5]SummaryInformation"
A: %fe%ff%00%00%06%01%02%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%00%01%00%00%00%e0%85%9f%f2%f9Oh%10%ab%91%08%00+'%b3%d90%00%00%00%c4%01%00%00%0e%00%00%00%01%00%00%00x%00%00%00%02%00%00%00%80%00%00%00%03%00%00%00%a0%00%00%00%04%00%00%00%c0%00%00%00%05%00%00%00%d8%00%00%00%06%00%00%00%ec%00%00%00%07%00%00%00%1c%01%00%00%09%00%00%000%01%00%00%0c%00%00%00`%01%00%00
A: %00%00%00l%01%00%00%0e%00%00%00x%01%00%00%0f%00%00%00%80%01%00%00%12%00%00%00%88%01%00%00%13%00%00%00%bc%01%00%00%02%00%00%00%e4%04%00%00%1e%00%00%00%16%00%00%00Installation Database%00%00%00%1e%00%00%00%18%00%00%00Google Chrome Installer%00%1e%00%00%00
A: %00%00%00Google, Inc.%00%00%00%00%1e%00%00%00
A: %00%00%00Installer%00%00%00%1e%00%00%00(%00%00%0051.0.2704.84 Copyright 2011 Google Inc.%00%1e%00%00%00%0b%00%00%00Intel;1033%00%00%1e%00%00%00'%00%00%00{A843382C-B154-431D-8325-DC6B82AE6A9C}%00%00@%00%00%00%00%d0%a6e%05%be%d1%01@%00%00%00%00%d0%a6e%05%be%d1%01%03%00%00%00%96%00%00%00%03%00%00%00%02%00%00%00%1e%00%00%00+%00%00%00Windows Installer XML Toolset (3.8.1128.0)%00%00%03%00%00%00%02%00%00%00
hansen-m commented 8 years ago

@jgstew Which version of QnA are you using? I believe I noticed that it worked too on an older version of QnA but I didn't get a chance to test it.

jgstew commented 8 years ago

I can't really tell which version of QnA I'm using. Created: January 27, 2016 at 2:33 PM

It is this: https://www.virustotal.com/en/file/a84dfaec921abd7ee978a4d13eee43b6fefe7c40374d488f73826d38e8518e86/analysis/

I do get this result in QnA:

Q: version of client
A: 9.2.7.53

But I think that is the version of the BigFix agent, not QnA.

There needs to be a relevance introspector to get the version of whatever is doing the thing (QnA, Fixlet Debugger, BigFix agent, etc...)

This could actually be a partial proxy for the version, especially if the number always goes up and never goes down, though it is platform dependant:

Q: number of properties
A: 1693
hansen-m commented 8 years ago

I found an old copy of QnA and ran it against the same file and it worked. So I'm guessing it is an issue with the latest version of QnA that is included in the 9.5 agent. Unfortunately it seems to be difficult or impossible to find the old versions of QnA in the IBM archive.

So here it is... https://psu.box.com/s/iowbvgctb4cj4qumzyzj72d4l4lcbiwg

jgstew commented 8 years ago

Then I'd say this is definitely a bug in QnA that IBM will need to address.

version 9.5 of BigFix added Unicode capabilities and this seems like an error somewhere with the Unicode subsystem.

Having Unicode support shouldn't be a step backwards.

You should be able to get old versions of QnA by downloading the BigFix client for the platform in question of whatever version of QnA you desire.

hansen-m commented 8 years ago

Agreed. This is an unusually formatted file, so probably not a typical use case, but an option to disable those capabilities and just read the file as plain text would be helpful.

jgstew commented 8 years ago

It does seem like this was the version of QnA I was using:

Q: version of client
A: 9.2.7.53

In the Windows Fixlet Debugger, this returns the version of the debugger when run in the debugger context and it returns the installed client version when run in the client context, so it does appear to allow you to know which version of QnA you are using.

I wouldn't say with absolute certainty that it works the same on macOS without verification, because that seems to be the way of things.

jgstew commented 8 years ago

This seems to work to attach the file in question to the issue: (add .txt extention)

[5]SummaryInformation.txt

Nice to have a sample file to work with.

Reference: https://help.github.com/articles/file-attachments-on-issues-and-pull-requests/

jgstew commented 8 years ago

This seems to be what is being used by BigFix to handle the Unicode and where the error is coming from internally to BigFix itself:

http://userguide.icu-project.org/conversion/converters#TOC-Error-Codes

@hansen-m This is definitely something that should be brought to Alan's attention, and probably a PMR.

blinvisible commented 8 years ago

I can't add anything helpful to the BigFix/QnA issue, but as a temporary workaround I can confirm that MSIVersionProvider is currently successful at getting the version number for both the x86 and x64 versions of the Chrome.

jgstew commented 8 years ago

I thought that the MSIVersionProvider got the wrong version or something?

hansen-m commented 8 years ago 
$ wine RecipeRepos/com.github.autopkg.hansen-m-recipes/SharedProcessors/lessmsi/lessmsi.exe v Cache/com.github.hansen-m.download.googlechrome-Win/downloads/GoogleChrome.msi 
66.164.103
blinvisible commented 8 years ago

It gets the ChromeProductVersion string, I think from the !_StringData file (currently set to 66.164.103), it didn't occur to me that isn't considered the current product version. /selffacepalm

So, yeah, it's wrong, since the application itself reports its version as 51.0.2704.103.

jgstew commented 8 years ago

I think it is because Chrome installs using Google Updater or something like that.

rustymyers commented 7 years ago

FYI, looking at this issue again. Still having the same error on latest QnA, reverting to 9.2.7.53 QnA and updating substrings separated by "%00%00%00" to as single "%00" returns correct version number.

Have any PMRs been entered on this QnA issue?

jgstew commented 7 years ago

I don't think there are any PMRs on this. The 9.5 version changes how unicode is handled and may also change the ways long lines are handled.

I haven't looked into this in a long time so I don't remember what the issue was, but if it works in 9.2 but not 9.5 that is suspicious.

rustymyers commented 7 years ago

Yes, PMR was filed. It's a bug.

jgstew commented 6 years ago

I think there is now a solution to this issue with QnA. You can cause QnA to read files the "old way" using an encoding inspector.

The issue is, this new thing doesn't work with older versions of QnA that don't have the problem, it only works with newer versions of QnA that do have the problem, but you can use some relevance to get around this issue, which I wrote:

( if ( exists properties whose(it as string contains "folder <string> of <encoding>: folder") ) then folder "/" of encoding "ISO-8859-1" else folder "/" )

Not sure if this will work properly due to the use of %RECIPE_CACHE_DIR% (might need to take the leading / off of that value for this to work)

(it as version) of preceding text of first " Copyright 2011 Google Inc." of substrings separated by "%00%00%00" whose (it contains "Google Inc.") of lines whose (it contains "Installer") of file "%RECIPE_CACHE_DIR%/InstallerExtract/[5]SummaryInformation" of ( if ( exists properties whose(it as string contains "folder <string> of <encoding>: folder") ) then folder "/" of encoding "ISO-8859-1" else folder "/" )

rustymyers commented 6 years ago

New processor (GoogleChromeWinVersioner.py) to get Google Chrome version from MSI is now available. Removes the requirement for using QnA with Google Chrome downloads. Uses 7zip to extract and regex to pull the version from the [5]SummaryInformation file.