search

OCA / maintainer-tools

Odoo Maintainers Tools & conventions for OCA members which evaluate and maintain repositories.
GNU Affero General Public License v3.0
271 stars 455 forks source link

copy_maintainers script fails syncing the big teams #615

Open hbrunn opened 4 months ago

hbrunn commented 4 months ago

We call copy_maintainers --team "OCA Contributors" and copy_maintainers --team "OCA Members" daily to update the appropriate teams in github.

Since a while I receive nearly daily mails about one of those failing with a bad gateway error (503). Can just reading this char field for a few hundred users really take 10min? (that's the proxy_read_timeout value of the oca nginx server). Or does nginx return 503 for some other reason? Buffer size possibly? I can't check myself because I can't see nginx' error log on the OCA server.

Chunking should help I guess? And if I touch this anyways, are there reservations against using oca's own odoorpc instead of the unmaintained erppeek?

pedrobaeza commented 4 months ago

Is this still in use? @etobella what you did is complementary to this?

etobella commented 4 months ago

Well, this is used for Oca contributors team, it is not maintaned there :thinking:

hbrunn commented 4 months ago

@pedrobaeza what is what @etobella did that might be complementary? I've never been involved in the housekeeping so much, but it seems if this fails new contributors aren't added to the good github group?

pedrobaeza commented 4 months ago

I'm talking about https://github.com/OCA/repo-maintainer-conf

sbidoul commented 4 months ago

I just tried to reproduce the team sync error, and... it succeeded, except for a bunch of problems related to invalid GitHub handles in our database. I'll look if the error comes up again.

hbrunn commented 4 months ago

yesterday night it failed because of the rate limit, I assume that's because you used the gitbot user's token for your tests?

You receive the error messages per mail too, right?

Not sure how to continue here. Should we just accept this to fail sometimes? Is this a timing issue and we possibly just need to spread out the heavy scripts in time more?

sbidoul commented 3 months ago

We probably need to add a retry mechanism on GitHub operations in copy_maintainers.py. This is low priority for me, but if I was to do it I'd try with tenacity or stamina.