Add note about pinning extension-helpers

[astrofrog]

Possible implementation of https://github.com/astropy/extension-helpers/issues

Obviously given that there are already a lot of releases on PyPI without pinning we should wait a while before we ever do bump the major version, but might as well get things in place so we can one day do it.

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (f4a0859) 76.92% compared to head (b730db3) 74.67%.

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #72 +/- ## ========================================== - Coverage 76.92% 74.67% -2.25% ========================================== Files 4 4 Lines 312 312 ========================================== - Hits 240 233 -7 - Misses 72 79 +7 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[neutrinoceros]

@astrofrog I just issued a bunch of PRs to (almost) all affected packages within the astropy org (coordinated and affiliated). The one or two that I'm missing live on gitlab, and I'll deal with them now !

edit: actually one of them seems inactive and the other is privately hosted, so I won't issue patches there. For reference I'm talking about https://gitlab.com/mcfuman/SpectraPy/ and https://git.ligo.org/lscsoft/ligo.skymap

[pllim]

Thanks, @neutrinoceros ! Do you also plan to do https://github.com/astropy/astropy/blob/6e695b69f7f530e541e2c0c00ffa9c0e5630e87f/pyproject.toml#L124 ?

[pllim]

Also, how did you search for this usage? For instance, https://github.com/astropy/astropy-healpix/blob/09a25789dcf105827b4e380081f278edb6a107e9/pyproject.toml#L5 has it but you did not issue a PR.

[pllim]

For ligo.skymap , maybe can try ping @lpsinger .

[pllim]

p.s. Some packages appear to really not need extension-helpers as I do not see they have any C-extension. I guess that is not your problem but maybe can add a message to the maintainer to consider removing extension-helpers as a build dependency if they do not really need it. Some people copy paste from some package and they did not bother to clean up the infrastructure after.

[neutrinoceros]

@pllim Here's what I did:

• go to https://www.astropy.org/affiliated/index.html
• manually fork and clone all repositories (with the 3 exceptions I mentioned earlier). Locally I organised my clones into coordinated and affiliated subdirectories
• from the parent directory containing the aforementioned subduers, run the following script

  
  # need_extension_helpers.py
  import re
  from pprint import pprint
  import subprocess

subprocess.run(["grep", '"extension[-_]helpers"', '-r', '.', '>', 'need_extension_helpers.txt'], check=True)

with open("need_extension_helpers.txt") as fh: lines = fh.readlines()

REGEXP = re.compile(r"./(?Paffiliated|coordinated)/(?P\w+)") names = [] for line in lines: if (match:=REGEXP.search(line)) is not None: names.append(match.group("kind") + "/" + match.group("name"))

names = sorted(set(names)) for name in names: print(name)

- visit local copies of matching repos, edit `pyproject.toml` (or `setup.py`), open PR

Thanks for pointing out that I missed the core library in the process ! I'll issue a PR now !

[neutrinoceros]

p.s. Some packages appear to really not need extension-helpers as I do not see they have any C-extension. I guess that is not your problem but maybe can add a message to the maintainer to consider removing extension-helpers as a build dependency if they do not really need it. Some people copy paste from some package and they did not bother to clean up the infrastructure after.

good thinking ! I'll go another round to remove extension-helpers where it's not actually needed, thanks !

[pllim]

For future reference, it might be more efficient to use the GitHub search API (I am used to REST but some people also use the GraphQL). In cases like this, a general search will have too many matches, so I would start with limiting to astropy org. And your idea to use the Affiliated listing is very good, so those would be then manually added to the search results as you see fit.

curl -o searchresults_astropy_01.json -u user:token -H "Accept: application/vnd.github.v3+json" "https://api.github.com/search/code?q=\"extension-helpers\"+org%3Aastropy&page=1&per_page=100&sort=indexed&order=desc"

The query above would theoretically return you the first 100 matches in astropy org. Additional queries using page=N might be necessary if it has more than 100 matches. The JSON result would tell you if that is the case or not. It would also tell you which file the match is found in. Some JSON post-processing might be necessary to weed out things like forks, archived repos, false positives, etc.

Once you have the cleaned data, you can use https://github.com/astrofrog/batchpr to open PRs for all the affected repos.

Here is something I actually used before for a different update (before Dependabot took over), in case it helps:

https://gist.github.com/pllim/1fce67691c218aa2c3cec0ff81c724a1

[neutrinoceros]

Thanks a lot ! I knew about those APIs but felt like I would get around faster without learning them for this one-time thing, but I will definitely considered upgrading my game if I need it again in the future !

[pllim]

felt like I would get around faster without learning them for this one-time thing

Definitely a valid dilemma. Sometimes I decide to do this manually and then regret it and sometimes vice versa. 😅