Closed lazka closed 2 years ago
I don't know if it's appropriate here or not, but I've found on raspberry pi invoking repo-add
for each package seems quite slow. I believe it would be faster to invoke repo-add
for many packages at once. It would probably need some xargs-style logic to keep to a maximum command line length in a degenerate case with tons of packages though :wink:
sounds good. Since it only adds packages already in the queue and only if they are a real dependency I think the amount should be quite limited anyway.
staging download every 30 minutes created a lot of traffic
this is now fixed, it only downloads when things change now. -> https://github.com/msys2/msys2-main-server/commit/28363c4107aad5543e73f4c02f9ff2aeb0742a8a
I don't know if it's appropriate here or not, but I've found on raspberry pi invoking
repo-add
for each package seems quite slow. I believe it would be faster to invokerepo-add
for many packages at once. It would probably need some xargs-style logic to keep to a maximum command line length in a degenerate case with tons of packages though wink
Improved in 842072fe5530ea26588153e4e1e02df56d8df71b and 10fdc3ec57d6577a53f65261a4ff37c3c4a0c901
I've split up the upload release tag for each build type, which should help a bit with reducing API calls when there are many assets. (the code will create the releases automatically when needed btw)
PKGBUILD parsing takes too long for CI
while hacky, this should improve it by a factor of 3-4: https://github.com/msys2/MINGW-packages/commit/22ad01c8ac826951541e5a86f31e356dcbed1277
Wouldn't removing the lint_pkgbuild function allow wrong PKGBUILD file?
yes, but ideally that would be caught in CI first when building the package.
the regex filename based selection for packages to rebuild wasn't very exact. Basing it on actual file content would make it better.
This is now solved by looking for DLL uses like here: https://github.com/lazka/package-grokker/runs/3578861455?check_suite_focus=true in combination with grepping for *.pyc
and python3.9/site-packages
I've added a braindump for the next rebuild here: https://www.msys2.org/dev/python/
I think with all those improvements we can close this, and re-evaluate on the next rebuild.
the regex filename based selection for packages to rebuild wasn't very exact. Basing it on actual file content would make it better.
This is now solved by looking for DLL uses like here: https://github.com/lazka/package-grokker/runs/3578861455?check_suite_focus=true in combination with grepping for
*.pyc
andpython3.9/site-packages
This made me decide to improve that script for the use case of finding DLL dependents of a scripting language. (jeremyd2019/package-grokker#7). For perl, this reduced the runtime of the script from 17 minutes to 3, with also probably a good reduction of packages downloaded to check.