search

chocolatey / choco

Chocolatey - the package manager for Windows
https://chocolatey.org
Other
10.21k stars 896 forks source link

chocolatey package updated but msi app not #679

Closed bodyakov closed 8 years ago

bodyakov commented 8 years ago

On 1% of workstation package successfully update, but msi app not.

Installed app

We are using puppet provider to update packages from private nuget repo. On some workstation msi package not update.

Output Log

bad_log https://gist.github.com/bodyakov/e71cff6c330b6983b47e0b2ec4caf9f1 ok_log https://gist.github.com/bodyakov/d4e830f8f2273724d70a38d918852276

ferventcoder commented 8 years ago

@bodyakov Neither of these logs have all of the output. We need all of the output (the configuration information is super important) to help us understand what may be wrong. Without that information, there isn't much we can help with.

bodyakov commented 8 years ago

At one time puppet update many packages. Log will be very big.

bodyakov commented 8 years ago

ok https://gist.github.com/bodyakov/0896babe7ad213e078ba6cf80aee2fb3 bad https://gist.github.com/bodyakov/d22e229ddcc8153f06fb35666b68df2b

ferventcoder commented 8 years ago

I see the issue. On the systems it is failing on, it's when it is already installed.

ferventcoder commented 8 years ago

Maybe....still investigating.

bodyakov commented 8 years ago

it can't be installed because: DEBUG: Running 'Start-ChocolateyProcessAsAdmin' with exeToRun:'msiexec', 2016-04-01 07:51:01,805 [INFO ] - statements: '/x{F17FBC84-B1B8-4816-B0A2-24F86C20E8E6} /q REBOOT=ReallySuppress'

And problem that program actually was unistalled after chocolatey update. We have a lot tickets from users to HelpDesk.

bodyakov commented 8 years ago

This problem occur not only on our internal packages. There are many cases (about 3%) when other programs are not installed. Such as flashplayerplugin, adobereader, etc

ferventcoder commented 8 years ago

FYI - don't do this https://gist.github.com/bodyakov/0896babe7ad213e078ba6cf80aee2fb3#file-ok_long-L120. It's really bad what it does to your system - https://github.com/chocolatey/package-validator/wiki/DontUseGetWmiObjectForFindingInstalledPackages

ferventcoder commented 8 years ago

If you are using this as an organization, please ensure you are not using the Community Feed (https://chocolatey.org/packages). Please see https://puppetlabs.com/blog/chocolatey-hosting-your-own-server for the reasoning.

If you are downloading packages from the community feed to use internally, ensure you are recompiling the packages for internal use. See https://docs.puppetlabs.com/pe/latest/windows_modules.html#copy-an-existing-package-and-make-it-internal-repackaging-packages

bodyakov commented 8 years ago

Accepted. In some packages i rewrite this on regisrty, accepted. in many packages I rewrote it through the registry, but there is still left.

ferventcoder commented 8 years ago

From what I can tell, right now it appears it is a logic error in your automation scripts. I'm not sure Version is guaranteed coming from what you are calling. Instead if you must remove old versions as you are doing, I would do the search based on the product code.

ferventcoder commented 8 years ago

https://gist.github.com/bodyakov/0896babe7ad213e078ba6cf80aee2fb3#file-ok_long-L135-L145

Your package is installing two things. Not usually recommended.

So crazy question - why can't your MSI handle upgrading properly?

If it's _all internal_ and you can't get MSIs working with upgrades and downgrades, my recommendation may be to stop using MSIs and just package the binaries right into the package. Again, if it is all internal.

I know that most commercials MSIs I deal with can handle the upgrading just fine, so you don't need to attempt to remove older versions.

ferventcoder commented 8 years ago

In 0.9.9.x - if anything writes to the stderr, it will cause the package to fail. reg is known to do this when things are successful, regedit I am not sure about. I would suggest taking a look at the 0.9.10 beta in this case. It is fully featured and has fixed the behavior (with a switch/feature to bring it back).

Here's all the features in 0.9.10 that are completed - https://github.com/chocolatey/choco/blob/master/CHANGELOG.md

bodyakov commented 8 years ago

The proble not in reg, or regedit, there all is ok. Problem in downloading or checking msi file. in ok log - 2016-04-01 09:16:21,366 [INFO ] - �������: Running 'Get-WebHeaders' with 2016-04-01 09:16:21,368 [INFO ] - url:'http:///package/ctios/AgentDesktopSetup-7.5.610 2016-04-01 09:16:21,369 [INFO ] - 3.msi', userAgent: 'chocolatey command line' 2016-04-01 09:16:21,370 [INFO ] - �������: Setting the UserAgent to 'chocolatey command line' 2016-04-01 09:16:21,372 [INFO ] - �������: Request Headers: 2016-04-01 09:16:21,373 [INFO ] - �������: 'Accept':'/' 2016-04-01 09:16:21,375 [INFO ] - �������: 'User-Agent':'chocolatey command line' 2016-04-01 09:16:21,385 [INFO ] - �������: Response Headers: 2016-04-01 09:16:21,388 [INFO ] - �������: 'Accept-Ranges':'bytes' 2016-04-01 09:16:21,389 [INFO ] - �������: 'Content-Length':'51046912' 2016-04-01 09:16:21,390 [INFO ] - �������: 'Content-Type':'application/x-msi' 2016-04-01 09:16:21,392 [INFO ] - �������: 'Date':'Fri, 01 Apr 2016 06:16:45 GMT' 2016-04-01 09:16:21,393 [INFO ] - �������: 'ETag':'"30aea00-52f1cf2f0a440"' 2016-04-01 09:16:21,394 [INFO ] - �������: 'Last-Modified':'Mon, 28 Mar 2016 14:44:25 GMT' 2016-04-01 09:16:21,399 [INFO ] - �������: 'Server':'Apache/2.4.7 (Ubuntu)' 2016-04-01 09:16:21,403 [INFO ] - Downloading agent 32 bit 2016-04-01 09:16:21,404 [INFO ] - from 'http:///package/ctios/AgentDesktopSetup-7.5.6103.msi' 2016-04-01 09:16:21,405 [INFO ] - �������: Running 'Get-WebFile' for 2016-04-01 09:16:21,406 [INFO ] - C:\windows\TEMP\chocolatey\agent\7.5.103.1\agentInstall.msi with 2016-04-01 09:16:21,407 [INFO ] - url:'http:///package/ctios/AgentDesktopSetup-7.5.610 2016-04-01 09:16:21,407 [INFO ] - 3.msi', userAgent: 'chocolatey command line' 2016-04-01 09:16:21,408 [INFO ] - �������: Setting the UserAgent to 'chocolatey command line' 2016-04-01 09:16:33,294 [INFO ] - �������: Checking that 2016-04-01 09:16:33,296 [INFO ] - 'C:\windows\TEMP\chocolatey\agent\7.5.103.1\agentInstall.msi' is the size we 2016-04-01 09:16:33,297 [INFO ] - expect it to be. 2016-04-01 09:16:33,298 [INFO ] - �������: Verifying package provided checksum of '' for 2016-04-01 09:16:33,301 [INFO ] - 'C:\windows\TEMP\chocolatey\agent\7.5.103.1\agentInstall.msi'. 2016-04-01 09:16:33,305 [INFO ] - �������: Running 'Get-ChecksumValid' with 2016-04-01 09:16:33,306 [INFO ] - file:'C:\windows\TEMP\chocolatey\agent\7.5.103.1\agentInstall.msi', checksum: 2016-04-01 09:16:33,314 [INFO ] - '', checksumType: ''

in bad log - 2016-04-01 07:51:11,033 [DEBUG] - DEBUG: Running 'Get-WebHeaders' with 2016-04-01 07:51:11,034 [INFO ] - url:'http:///package/ctios/AgentDesktopSetup-7.5.610 2016-04-01 07:51:11,034 [INFO ] - 3.msi', userAgent: 'chocolatey command line' 2016-04-01 07:51:11,035 [DEBUG] - DEBUG: Setting the UserAgent to 'chocolatey command line' 2016-04-01 07:51:11,035 [DEBUG] - DEBUG: Request Headers: 2016-04-01 07:51:11,036 [DEBUG] - DEBUG: 'Accept':'/' 2016-04-01 07:51:11,036 [DEBUG] - DEBUG: 'User-Agent':'chocolatey command line' 2016-04-01 07:51:11,076 [DEBUG] - DEBUG: Response Headers: 2016-04-01 07:51:11,096 [DEBUG] - DEBUG: 'Accept-Ranges':'bytes' 2016-04-01 07:51:11,096 [DEBUG] - DEBUG: 'Content-Length':'51046912' 2016-04-01 07:51:11,097 [DEBUG] - DEBUG: 'Content-Type':'application/x-msi' 2016-04-01 07:51:11,097 [DEBUG] - DEBUG: 'Date':'Fri, 01 Apr 2016 04:51:36 GMT' 2016-04-01 07:51:11,098 [DEBUG] - DEBUG: 'ETag':'"30aea00-52f1cf2f0a440"' 2016-04-01 07:51:11,098 [DEBUG] - DEBUG: 'Last-Modified':'Mon, 28 Mar 2016 14:44:25 GMT' 2016-04-01 07:51:11,099 [DEBUG] - DEBUG: 'Server':'Apache/2.4.7 (Ubuntu)' 2016-04-01 07:51:11,099 [INFO ] - Downloading agent 32 bit 2016-04-01 07:51:11,100 [INFO ] - from 'http:///package/ctios/AgentDesktopSetup-7.5.6103.msi' 2016-04-01 07:51:11,100 [DEBUG] - DEBUG: Running 'Get-WebFile' for 2016-04-01 07:51:11,101 [INFO ] - C:\windows\TEMP\chocolatey\agent\7.5.103.1\agentInstall.msi with 2016-04-01 07:51:11,101 [INFO ] - url:'http:///package/ctios/AgentDesktopSetup-7.5.610 2016-04-01 07:51:11,102 [INFO ] - 3.msi', userAgent: 'chocolatey command line' 2016-04-01 07:51:11,102 [DEBUG] - DEBUG: Setting the UserAgent to 'chocolatey command line'

I think in bad case chocolatey can't complete Get-WebFile but not error occur.

ferventcoder commented 8 years ago

http://<url>/package/ctios/AgentDesktopSetup-7.5.6103.msi? What is this location? Are you trying to say something exists in the package you want to use?

Nevermind, missed that you removed the url.

bodyakov commented 8 years ago

I remove the domain from url before posting log here.

ferventcoder commented 8 years ago

No worries, github hides what look like html tags by default.

ferventcoder commented 8 years ago

I missed it until I looked at the actual words.

So you are saying there is no file downloaded when you go check?

ferventcoder commented 8 years ago

Because the way Chocolatey works, if it can't download the file, it throws an error.

bodyakov commented 8 years ago

After falling i go remotely to that pc and see that file size is about 22mb, but real file on web server have about 40mb. C:\windows\TEMP\chocolatey\agent\7.5.103.1\agentInstall.msi

bodyakov commented 8 years ago

i also think so. but didn't

bodyakov commented 8 years ago

Strange why not launched Get-ChecksumValid. I think that chocolatey can crashed on download file. And no error in log.

bodyakov commented 8 years ago

Also think will be better not to place package directory in /lib folder before install start. On start put package directory in /lib-bad or somwhere like temp. And only if chocolatey se that chocolateyinstall.ps1 exit with exit status 0 move to /lib. Apt works on that principle.

ferventcoder commented 8 years ago

So a few fundamental things about Chocolatey:

https://github.com/chocolatey/choco/wiki/HelpersReference

ferventcoder commented 8 years ago

Strange why not launched Get-ChecksumValid. I think that chocolatey can crashed on download file. And no error in log.

There is likely an error, but it would have been in a previous install attempt.

ferventcoder commented 8 years ago

Also think will be better not to place package directory in /lib folder before install start. On start put package directory in /lib-bad or somwhere like temp. And only if chocolatey se that chocolateyinstall.ps1 exit with exit status 0 move to /lib. Apt works on that principle.

Do you mean something like mark the package as pending until successful? It's likely better to talk about high level features versus implementation details. What you are requesting will break dependency resolution and upgrades as NuGet works off of the existing packages directory and not a temporary directory. We have https://github.com/chocolatey/choco/issues/383 for the pending behavior, which I believe achieves the goal of what you are requesting here.

bodyakov commented 8 years ago

If a file is already there, Get-ChocolateyWebFile will not redownload

I cannot reproduce this. Start install skype. cinst skype -y. at 50% download msi - disable network. chocolatey fail. Enable network, run cinst skype -y again, and chocolatey redownload msi, and sucess. What here wrong?

Do you mean something like mark the package as pending until successful?

I think this will bery good

ferventcoder commented 8 years ago

I cannot reproduce this. Start install skype. cinst skype -y. at 50% download msi - disable network. chocolatey fail. Enable network, run cinst skype -y again, and chocolatey redownload msi, and sucess. What here wrong?

I likely should have been a bit more clear. Get-ChocolateyWebFile does request headers to determine the size the file should be. If it receives the information it uses that to determine if the content-length is the same prior to moving forward.

https://github.com/chocolatey/choco/blob/0.9.9.11/src/chocolatey.resources/helpers/functions/Get-ChocolateyWebFile.ps1#L128-L135

and

https://github.com/chocolatey/choco/blob/0.9.9.11/src/chocolatey.resources/helpers/functions/Get-ChocolateyWebFile.ps1#L161-L162

ferventcoder commented 8 years ago

It does look from your logs like you are doing that. However it also looks like you have multiple choco processes running at the same time making it hard to read the bad log. I think there is more to it that needs to be added to the end to see the rest of the bad install.

I'm not sure I'd recommend multiple choco processes running at the same time. And I'm not sure what is causing it because I don't know your environment.

bodyakov commented 8 years ago

I'm not sure I'd recommend multiple choco processes running at the same time

You dont think that will be good idea to do some like dpkg lock. And exit if choco alredy running.

ferventcoder commented 8 years ago

It's probably a good idea to ensure that only one process runs at a time.

bodyakov commented 8 years ago

Now i see next problems in using chocolatey on workstations:

  1. Main problem - default behaviour during install packet: copying to /lib directory. If during installation pc will be rebooted or choco crashed or something else, choco will think that work done and package installed, but really don't.
  2. Block concurent executing choco.
  3. Bad mechanism to controll sources from puppet.
  4. Hard to read chocolatey log. There to many debug information.
ferventcoder commented 8 years ago

Now i see next problems in using chocolatey on workstations:

  1. Main problem - default behaviour during install packet: copying to /lib directory. If during installation pc will be rebooted or choco crashed or something else, choco will think that work done and package installed, but really don't.

383 - pretty sure we already covered this above in https://github.com/chocolatey/choco/issues/679#issuecomment-204432211. Same comments, it's about marking the package pending.

  1. Block concurent executing choco.

This one will become a github issue - definitely worth ensuring only one is running

  1. Bad mechanism to controll sources from puppet.

Please be more specific?

Package has built in support for controlling source - https://forge.puppetlabs.com/chocolatey/chocolatey#source

It's the same as any other package resource with Puppet. So not bad, maybe not what you intended.

Or perhaps you mean something more like https://tickets.puppetlabs.com/browse/MODULES-3037 ?

  1. Hard to read chocolatey log. There to many debug information.

Perhaps a secondary log for general output that has less debugging/verbose information in it?

bodyakov commented 8 years ago

it's about marking the package pending.

I think it will be resolve 95% my problems with chocolatey.

Please be more specific?

I know about source in puppet, but this is not apply in my case. We have two repositories (nuget feed). One stable, and other testing. We add to specific host only atribute to be a member of class testing, and then i nedd to push to host different source to testing nuget feed. If property testing was removed, the i also need to remove testing source.

Perhaps a secondary log for general output that has less debugging/verbose information in it?

It will be nice.

ferventcoder commented 8 years ago

Created https://github.com/chocolatey/choco/issues/682 for the secondary log.

bodyakov commented 8 years ago

Created #682 for the secondary log.

Also will be nice remove -dv options from puppet provider. And leave only chocolatey.log for debug information.

ferventcoder commented 8 years ago

@bodyakov wrong repo for that discussion - see https://github.com/chocolatey/puppet-chocolatey/issues/100

ferventcoder commented 8 years ago

I know about source in puppet, but this is not apply in my case. We have two repositories (nuget feed). One stable, and other testing. We add to specific host only atribute to be a member of class testing, and then i nedd to push to host different source to testing nuget feed. If property testing was removed, the i also need to remove testing source.

I'm still not sure what you are asking for here.

ferventcoder commented 8 years ago

https://github.com/chocolatey/choco/issues/683 for one running choco.exe

bodyakov commented 8 years ago

I'm still not sure what you are asking for here.

I think i found what i need - https://github.com/chocolatey/puppet-chocolatey/pull/79

ferventcoder commented 8 years ago

I believe I already mentioned that one, but maybe not. :+1:

ferventcoder commented 8 years ago

Think we created issues out of everything we were fixing here. If I'm wrong feel free to comment and we can reopen this issue.