search

Homebrew / homebrew-cask

🍻 A CLI workflow for the administration of macOS applications distributed as binaries
https://brew.sh
BSD 2-Clause "Simplified" License
20.56k stars 10.53k forks source link

finding outdated casks with and without appcast #55268

Closed core-code closed 5 years ago

core-code commented 5 years ago

hello HBC community,

one of the 'problems' of Homebrew Cask is that it requires constant maintainance to keep working. any Cask with a version number needs to be updated every time a new version of this app is released.

for apps that have an sparkle/rss or github/atom feed from the vendor, keeping them up-to-date is relatively straightforward. @brianmorton has a great system implemented here. (his system incidentally also works for non-sparkle appcasts, but currently most of the Casks that have no sparkle, also have no appcast-stanza)

therefore i've been focusing my efforts on automatically finding outdated Casks on those without "sparkle appcast". every day i let a script run that compares the version of the Casks without appcast, to the database of app-versions that our company maintains. every day the script finds around 5 - 10 outdated Casks that should be updated e.g. using 'cask-repair'.

the design and implementation of the system is discussed here: https://github.com/Homebrew/homebrew-cask/issues/52837 and there was also a bit discussion about this here: https://github.com/Homebrew/homebrew-cask/issues/54811

so, long story short, basically it would be great if the community could have a look at the "cr-processed-DATE.txt" files i place here: http://macupdater.net/cask-repair/ each day and look through them to see which of those entries are real, official new releases of the apps and submit cask-repairs accordingly. i think this would also be a good task to get started with contributing to HBC.

for the time being, it would be best to note false positives (i.e. entries that are not real official releases of this app) here so i'll add them to the different filter-lists in the postprocessing script so that they won't appear in the next days output anymore.

UPDATE 11.dec.2018: ok, the cask-repair system for apps without appcast has been working quite well over the past month, i guess around 80% success rate (i.e. percentage of entries in the txt files that are actually valid new releases). but, it seems like there are quite a lot of casks WITH appcast that are outdated despite the appcast. so i've implemented two new checks to try to improve the situation there:

crsparklefeed-DATE.txt => this checks each cask with an valid sparkle appcast each day and generates an output if the appcast has a new 'newest' entry since yesterday. since sparkle feeds should only contain valid releases, 100% of those should be valid updates to be done via 'cask-repair' (if someone else hasn't beaten you to it).

crgithub-DATE.txt => this checks each cask that has its releases on github (as denoted via the URL or APPCAST stanza) and checks github if a new release has been posted since yesterday. if available, it tries to query the 'Latest Release' as denoted on github by the developer, so that beta releases are not picked up. if no 'Latest Release' is marked, it falls back to just using the newest release and downgrades from error to warning.

core-code commented 5 years ago

2018-11-21 output up

ran-dall commented 5 years ago

@core-code Thanks! Got the PRs up as well. The output is getting better and better.

No problems today.

core-code commented 5 years ago

awesome, thanks ;)

2018-11-22 output up

ran-dall commented 5 years ago

@core-code Happy Thanksgiving! 🦃

Sorry I was a little late today. Here's the work for the day...

data-dog - https://github.com/Homebrew/homebrew-cask/pull/54739 flash-player-debugger - I don't see the release anywhere. parallels - https://github.com/Homebrew/homebrew-cask/pull/55270 prusa3d - Version is an RC unshaky - Version is an Pre-release

Everything else was updated.

core-code commented 5 years ago

thanks ;)

2018-11-23 output up

ran-dall commented 5 years ago

@core-code ...done.👍

ipsecuritas-This is a build number update. It's also not on their website. http://www.lobotomo.com/products/IPSecuritas/index.html

...ipsecuritas cask: 4.9.1: 4.9.1|(null) cgi: 4.9.1 (4554)|4554

core-code commented 5 years ago

...ipsecuritas cask: 4.9.1: 4.9.1|(null) cgi: 4.9.1 (4554)|4554

actually there was a problem 'mapping' the version number here, thanks for the heads-up, should be fixed now

core-code commented 5 years ago

2018-11-24 output up

ran-dall commented 5 years ago

@core-code Done. Some of them weren't merged but are now...👌

oni - Beta

core-code commented 5 years ago

thanks ;)

2018-11-25 output up

ran-dall commented 5 years ago

@core-code done.

datadog-agent - https://github.com/Homebrew/homebrew-cask/pull/54739

core-code commented 5 years ago

datadog-agent - #54739

this seems to be some sort of eternal quest ^^

2018-11-26 output up - really tiny this time

ran-dall commented 5 years ago

@core-code Done.

datadog-agent - https://github.com/Homebrew/homebrew-cask/pull/54739 (The quest continues...) bonitastudiocommunity - 7.7.1 is the latest release on their site

core-code commented 5 years ago

bonitastudiocommunity - 7.7.1 is the latest release on their site

it seems they changed their download URLs, and are now hosting here: https://release.ow2.org/bonita/

i've sent a PR

ran-dall commented 5 years ago

@core-code Good work! 👌

core-code commented 5 years ago

2018-11-27 output up

ran-dall commented 5 years ago

@core-code done - no problemos today 🎉

core-code commented 5 years ago

thanks ;) 2018-11-28 output up

ran-dall commented 5 years ago

@core-code Done.

appgate-sdp-client - No newer version available for download. flightgear - No newer version available for download. android-studio - No newer version available for download.

core-code commented 5 years ago

thanks, adjusted the postprocessing

2018-11-29 output up

ran-dall commented 5 years ago

@core-code done - no problemos today 🎉

core-code commented 5 years ago

2018-11-30 output up

ran-dall commented 5 years ago

@core-code 🙌 done.

alfred - No newer version available for download. gray - https://github.com/Homebrew/homebrew-cask/pull/55526

core-code commented 5 years ago

2018-12-01 output up

ran-dall commented 5 years ago

@core-code done - no problemos today 🎉

suschizu commented 5 years ago

2018-12-02 output is online 🙂

ran-dall commented 5 years ago

@suschizu Thank you for the heads up! 🙏 @core-code Done. ✊

datadog-agent - https://github.com/Homebrew/homebrew-cask/pull/54739 cracked - No newer version available for download. inkdrop - https://github.com/Homebrew/homebrew-cask/pull/55568 birdfont - I get the following error, twice...

curl: (56) Unexpected EOF                                                                                                                            
Error: Download failed on Cask 'birdfont' with message: Download failed: https://birdfont.org/download/birdfont-3.17.0-free.dmg

musescore - No newer version available for download. gray - https://github.com/Homebrew/homebrew-cask/pull/55526

core-code commented 5 years ago

2018-12-03 output up

probably a lot of things in there that already have open pull requests as things haven't been merged for a few days now. i want to include a check for open pull requests but didn't come around to it yet.

also there is something new today: failingcasks_withoutfeed_2018-12-03.txt lists all the casks that failed to 'brew cask fetch', which usually means either • there is a new version and it needs a cask-repair • there is no new version but the checksum changed which is problematic due to our checksum rules • host is (temporary?) offline in any case, i only checked the ~2300 casks that have no real sparkle appcasts - around 150GB. downloading the full cask set is around 400GB i think.

ran-dall commented 5 years ago

@core-code Done.

Progressive Downloader - No newer version available for download.

ran-dall commented 5 years ago

@core-code Casks with outdated checksums have been updated. 🎉

core-code commented 5 years ago

awesome, good job ;)

core-code commented 5 years ago

2018-12-04 output up

ran-dall commented 5 years ago

@core-code Done. 👍

frostwire - No newer version available for download.

core-code commented 5 years ago

there seems to be a frostwire 6.7.5 listed on their homepage, here: https://www.frostwire.com/downloads

however, clicking on download gives us an installer, that, surprise surprise, contains the same adware as FileZilla!

(btw, the same is true for the bitlord & bittorrent casks. in the case of 'bittorrent' the cask download leads directly to the adware version)

ran-dall commented 5 years ago

@core-code I say we give them all the FileZilla! treatment for being 💩heads.

ran-dall commented 5 years ago

@core-code Finally got other to those other ones. 👌

These are no longer available from what I was able to research.

curl: (60) SSL certificate problem: certificate has expired
More details here: https://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
of Certificate Authority (CA) public keys (CA certs). If the default
bundle file isn't adequate, you can specify an alternate file
using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
the bundle, the certificate verification probably failed due to a
problem with the certificate (it might be expired, or the name might
not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
the -k (or --insecure) option.
HTTPS-proxy has similar options --proxy-cacert and --proxy-insecure.
Error: Download failed on Cask 'pdfkey-pro' with message: Download failed: https://pdfkey.com/PDFKeyPro.dmg
There was an error fetching ./pdfkey-pro.rb. Please check your connection and try again.


- protopie
```bash
cask-repair protopie -v 3.9.1
Updating taps… Already up-to-date.                                                                                                                   
-----------------------------------------------------------------------------------------------------------------------------------------------------
cask 'protopie' do                                                                                                                                   
  version '3.9.1'                                                                                                                                    
  sha256 '4cda5fb8ad26b54f23e25a3dc5d8c18520c5391a61f479234b32999fad2d1b10'                                                                          

  url "https://release.protopie.io/ProtoPie-#{version}.dmg"                                                                                          
  appcast 'https://www.protopie.io/support/updates/'                                                                                                 
  name 'ProtoPie'                                                                                                                                    
  homepage 'https://www.protopie.io/'                                                                                                                

  app 'ProtoPie.app'                                                                                                                                 
end                                                                                                                                                  
-----------------------------------------------------------------------------------------------------------------------------------------------------
==> Downloading external files for Cask protopie                                                                                                     
==> Downloading https://release.protopie.io/ProtoPie-3.9.1.dmg                                                                                       

curl: (51) SSL: no alternative certificate subject name matches target host name 'release.protopie.io'                                               
Error: Download failed on Cask 'protopie' with message: Download failed: https://release.protopie.io/ProtoPie-3.9.1.dmg                              
There was an error fetching ./protopie.rb. Please check your connection and try again.

EVERYTHING ELSE WAS UPDATED! 👌

suschizu commented 5 years ago

2018-12-05 output is here 🎁

ran-dall commented 5 years ago

@suschizu Thank you! ✊ My favourite type of type of gift in the morning!

@core-code Done. ⚡️

alfred - No newer version available for download. musescore - Beta

core-code commented 5 years ago

thanks @ran-dall

"pdfkey-pro" certainly seems still available, @suschizu could you please have a look at the other ones?

ran-dall commented 5 years ago

@core-code They are all available but fail because they try to use https. It's why I sent you the output.

core-code commented 5 years ago

the SSL issue seems fixed in the pdfkey-pro case. i am unsure what the HBC policy is in case of SSL failures...downgrade to http?

ran-dall commented 5 years ago

@core-code Ok Cool. I'll take a look at pdfkey-pro

I don't know about the HBC position on SSL failure but I would go with a remove from what I've been reading over and over again in docs, but otherwise, this might fall into one of those 'case-by-case' things.

ran-dall commented 5 years ago

@core-code pdkey-pro looks good. It's up to date.

Here are some other outputs...

cask-repair rpvoip
Updating taps… Already up-to-date.                                                                                                                   
-----------------------------------------------------------------------------------------------------------------------------------------------------
cask 'rpvoip' do                                                                                                                                     
  version :latest                                                                                                                                    
  sha256 :no_check                                                                                                                                   

  url 'https://www.intercall.com/downloads/rpvoip_mac.pkg'                                                                                           
  name 'RPVoIP'                                                                                                                                      
  homepage 'https://www.intercall.com/'                                                                                                              

  pkg 'rpvoip_mac.pkg'                                                                                                                               

  uninstall pkgutil: 'com.intercall.webexVoipClient.RPVoIP.pkg'                                                                                      

  zap trash: [                                                                                                                                       
               '~/Library/Application Support/RPVoIP',                                                                                               
               '~/Library/Preferences/com.intercall.rpvoip.mac.plist',                                                                               
             ]                                                                                                                                       
end                                                                                                                                                  
-----------------------------------------------------------------------------------------------------------------------------------------------------
Type the new version (or leave blank to use current one, or use `s` to skip)                                                                         
>                                                                                                                                                    
Paste the new URL (or leave blank to use the current one)                                                                                            
>                                                                                                                                                    
-----------------------------------------------------------------------------------------------------------------------------------------------------
• url is probably incorrect, as a non-200 (OK) HTTP response code was returned (000)    
cask-repair tinyumbrella
Updating taps… Already up-to-date.                                                                                                                   
-----------------------------------------------------------------------------------------------------------------------------------------------------
cask 'tinyumbrella' do                                                                                                                               
  version '9.3.4'                                                                                                                                    
  sha256 '2d04d6503c6e130a43615d6f59be0092fb2d5300b4f414c5c36f0c60bffee3ff'                                                                          

  # tinydev.space was verified as official when first introduced to the cask                                                                         
  url "https://tinydev.space/downloads/#{version.dots_to_underscores}/tinyumbrella_macos_#{version.dots_to_underscores}_AppOnly_InstalledJRE.dmg"    
  name 'TinyUmbrella'                                                                                                                                
  homepage 'https://www.firmwareumbrella.com/'                                                                                                       

  app 'TinyUmbrella.app'                                                                                                                             

  caveats do                                                                                                                                         
    depends_on_java                                                                                                                                  
  end                                                                                                                                                
end                                                                                                                                                  
-----------------------------------------------------------------------------------------------------------------------------------------------------
Type the new version (or leave blank to use current one, or use `s` to skip)                                                                         
>                                                                                                                                                    
==> Caveats                                                                                                                                          
tinyumbrella requires Java. You can install the latest version with                                                                                  

  brew cask install java                                                                                                                             

==> Downloading external files for Cask tinyumbrella                                                                                                 
==> Downloading https://tinydev.space/downloads/9_3_4/tinyumbrella_macos_9_3_4_AppOnly_InstalledJRE.dmg                                              

curl: (60) SSL certificate problem: certificate has expired                                                                                          
More details here: https://curl.haxx.se/docs/sslcerts.html                                                                                           

curl performs SSL certificate verification by default, using a "bundle"                                                                              
 of Certificate Authority (CA) public keys (CA certs). If the default                                                                                
 bundle file isn't adequate, you can specify an alternate file                                                                                       
 using the --cacert option.                                                                                                                          
If this HTTPS server uses a certificate signed by a CA represented in                                                                                
 the bundle, the certificate verification probably failed due to a                                                                                   
 problem with the certificate (it might be expired, or the name might                                                                                
 not match the domain name in the URL).                                                                                                              
If you'd like to turn off curl's verification of the certificate, use                                                                                
 the -k (or --insecure) option.                                                                                                                      
HTTPS-proxy has similar options --proxy-cacert and --proxy-insecure.                                                                                 
Error: Download failed on Cask 'tinyumbrella' with message: Download failed: https://tinydev.space/downloads/9_3_4/tinyumbrella_macos_9_3_4_AppOnly_I
nstalledJRE.dmg                                                                                                                                      
There was an error fetching ./tinyumbrella.rb. Please check your connection and try again.
suschizu commented 5 years ago

2018-12-06 new cases to solve online 🔎

suschizu commented 5 years ago

@ran-dall I'm checking the apps you had problems with - perhaps I can find a solution :)

ran-dall commented 5 years ago

@suschizu Shouldn’t be hard to fix, however I don’t know of any ways that would still be adhering to regular HBC guidelines because https is required from my understanding.

ran-dall commented 5 years ago

@core-code Done.

rawtherapee - https://rawtherapee.com/blog/rawtherapee-5.5-release-candidate-1 sqlitebrowser - No newer version available for download. musescore - REPEAT Beta hma-pro-vpn - https://github.com/Homebrew/homebrew-cask/pull/55647

suschizu commented 5 years ago

Hey @ran-dall … so this is what I found out:

ran-dall commented 5 years ago

@suschizu Good work! 👍 I try to would avoid using HTTP:// as much as possible. It's an intelligent effort to contact the original developer; 👌 I like it. As for ionic-lab, I suggest we remove it. I think it's simply been super-seeded ⚡️ as suggested by Ionic their notice and blog posts.

suschizu commented 5 years ago

Hey @ran-dall thanks :) you're right but in this case it's a subdomain and the domain itself has a valid encryption - I think the developer missed to fix this :/ you're right with iconic-lab - they discontinued it 2 years ago