GithubRelease assets are not found via autobump

[holtjma]

Hello,

I've having issues getting autobump to correctly identify a version change for this recipe: https://github.com/bioconda/bioconda-recipes/blob/master/recipes/hiphase/meta.yaml. It appears to be an issue between the GithubRelease hoster and what GitHub returns via HTML now. Couple notes:

• I verified that autobump is correctly trying to use GithubRelease
• I verified that it's fetching the correct URL in this instance
• When I checked the downloaded HTML, I noticed that the links required (of this form https://github.com/PacificBiosciences/HiPhase/releases/download/v{{ version }}/hiphase-v{{ version }}-x86_64-unknown-linux-gnu.tar.gz) don't actually show up in the HTML.
• When I reviewed it with web, it appears to be lazily loaded via a second URL of this pattern: https://github.com/PacificBiosciences/HiPhase/releases/expanded_assets/v0.10.0
• That pattern does show up in the autobump download

Below are some logs with things I added while trying to debug:

$ bioconda-utils autobump recipes/ config.yml --packages hiphase --loglevel debug
...
08:09:03 BIOCONDA DEBUG Matching url 'https://github.com/PacificBiosciences/HiPhase/releases/download/v0.8.1/hiphase-v0.8.1-x86_64-unknown-linux-gnu.tar.gz'                                   
08:09:03 BIOCONDA DEBUG GithubRelease matched https://github.com/PacificBiosciences/HiPhase/releases/download/v0.8.1/hiphase-v0.8.1-x86_64-unknown-linux-gnu.tar.gz with {'account': 'PacificBiosciences', 'project': 'HiPhase', 'prefix': '', 'version': '0.8.1', 'fname': 'hiphase-v0.8.1-x86_64-unknown-linux-gnu.tar.gz', 'ext': ''}
08:09:03 BIOCONDA DEBUG Scanning with GithubRelease                                                                                                                                            
08:09:03 BIOCONDA DEBUG before: <bioconda_utils.aiopipe.AsyncRequests object at 0x7f9b61f85eb0>, 0.8.1                                                                                         
08:09:03 BIOCONDA DEBUG OHH start: <bioconda_utils.aiopipe.AsyncRequests object at 0x7f9b61f85eb0>, 0.8.1                                                                                      
08:09:03 BIOCONDA DEBUG HH exclude: {'version', 'fname'}                                                                                                                                       
08:09:03 BIOCONDA DEBUG HH vals: {'account': 'PacificBiosciences', 'project': 'HiPhase', 'prefix': '', 'ext': ''}                                                                              
08:09:03 BIOCONDA DEBUG HH link_pattern: /PacificBiosciences/HiPhase/releases/download/??(?:(?<=[/._-])[rv])?(?P<version>\d[\da-zA-Z\-+\.:\~_]{0,30})/(?P<fname>[^/]+)?                        
08:09:03 BIOCONDA DEBUG HH link_re: regex.Regex('/PacificBiosciences/HiPhase/releases/download/??(?:(?<=[/._-])[rv])?(?P<version>\\d[\\da-zA-Z\\-+\\.:\\~_]{0,30})/(?P<fname>[^/]+)?', flags=regex.V0)
08:09:03 BIOCONDA DEBUG HH url: https://github.com/PacificBiosciences/HiPhase/releases  

So my questions are basically how do we fix this? Unless I'm mistaken, this should be broken for every recipe that is pulling from the assets. I tried using the GitHub API via https://api.github.com/repos/PacificBiosciences/HiPhase/releases. However, I think the parser is expecting HTML so it fails to parse out the URLs.

Hopefully, I'm not missing something obvious here. Any suggestion?

[aliciaaevans]

A variation on this issue came up from @AndreaGuarracino in Gitter with https://github.com/pangenome/smoothxg/releases. The previous release descriptions had a link to the tarball, so that's getting found and the Assets section of the most recent release is never checked.

Would it make sense to to return all versions found with either HTML method? Or give the Assets section priority over the static HTML.

[holtjma]

This isn’t fully functional yet. We’re still waiting on #912 to make it into prod. I don’t know if that will resolve the issue, but it’s probably worth waiting to see if it does.

[holtjma]

As of #920, the original issue is fixed.