search

go-gitea / gitea

Git with a cup of tea! Painless self-hosted all-in-one software development service, including Git hosting, code review, team collaboration, package registry and CI/CD
https://gitea.com
MIT License
44.6k stars 5.45k forks source link

drone.io integration broken after migration from gogs #11970

Closed moepman closed 4 years ago

moepman commented 4 years ago

Note: the instance was migrated from Gogs 0.11.91 to Gitea 1.12.0. All old webhooks were deleted.

I setup a new drone server using the current version, created it as an oauth application as per its documentation and then allowed it to access gitea as my user. After the inital login it stays in "syncing" forever producing about 1000 log entries as seen above on my gitea server without ever finishing. The requests only stop once I stop the drone server. Drone itself does not seem to log anything useful, even with DRONE_LOGGING_TRACE=true.

I suspect this is an incompatibility with the paging in the repo API. Is there anything I can do to debug and help fix this problem?

lunny commented 4 years ago

Any logs?

moepman commented 4 years ago

On the drone.io side I have this:

drone-server[6999]: DEBU[0040] begin synchronization                         login=moepman
drone-server[6999]: DEBU[0040] syncer: begin repository sync                 login=moepman
drone-server[6999]: DEBU[0040]                                               fields.time="2020-06-19T08:26:33+02:00" latency="924.594µs" method=GET remote="127.0.0.1:50522" request=/ request-id=1dWjxxtquwp0KVvVOXLs0YBean1
drone-server[6999]: DEBU[0040] events: stream cancelled                      request-id=1dWjxqBT7v1p6leyTs6n13NTPJk
drone-server[6999]: DEBU[0040] events: stream closed                         request-id=1dWjxqBT7v1p6leyTs6n13NTPJk
drone-server[6999]: DEBU[0040] api: guest access                             request-id=1dWjxqBT7v1p6leyTs6n13NTPJk
drone-server[6999]: DEBU[0040]                                               fields.time="2020-06-19T08:26:33+02:00" latency=306.966922ms method=GET remote="[::1]:33174" request=/api/stream request-id=1dWjxqBT7v1p6leyTs6n13NTPJk
drone-server[6999]: DEBU[0040]                                               fields.time="2020-06-19T08:26:33+02:00" latency=1.413933ms method=GET remote="[::1]:33186" request=/api/user request-id=1dWjy1t1FtyWDp47SPBKJ3Au8y1
drone-server[6999]: DEBU[0040]                                               fields.time="2020-06-19T08:26:33+02:00" latency=9.526562ms method=GET remote="127.0.0.1:50526" request="/api/user/repos?latest=true" request-id=1dWjy2pmhPi23wDfdTLou2ct6EH
drone-server[6999]: DEBU[0040]                                               fields.time="2020-06-19T08:26:33+02:00" latency=3.516751ms method=GET remote="[::1]:33190" request=/api/user/builds/recent request-id=1dWjy3KEAGkXwa3Mdls4Yf8lF7O
drone-server[6999]: DEBU[0040]                                               fields.time="2020-06-19T08:26:33+02:00" latency=13.200574ms method=GET remote="127.0.0.1:50530" request="/api/user/repos?latest=true" request-id=1dWjy1XnkSjmn4UIK4kH1i50o7P
drone-server[6999]: DEBU[0040] events: stream opened                         request-id=1dWjxxWsALH4Qzd7ywkzv4xDEAL user.admin=true user.login=moepman
drone-server[6999]: DEBU[0060]                                               fields.time="2020-06-19T08:26:53+02:00" latency=2.180165ms method=GET remote="127.0.0.1:50542" request=/api/user request-id=1dWk0THrJnHS7PLvYl1pp4qf1jk
drone-server[6999]: DEBU[0070]                                               fields.time="2020-06-19T08:27:03+02:00" latency=1.845105ms method=GET remote="[::1]:33210" request=/api/user request-id=1dWk1iX75jgkOFZUNKNWDI272GL
drone-server[6999]: DEBU[0080]                                               fields.time="2020-06-19T08:27:13+02:00" latency=2.060026ms method=GET remote="127.0.0.1:50554" request=/api/user request-id=1dWk2yL5aVuPtO5ga5SJ9i2Tpjm
drone-server[6999]: DEBU[0090]                                               fields.time="2020-06-19T08:27:23+02:00" latency=2.069861ms method=GET remote="[::1]:33222" request=/api/user request-id=1dWk4Hpaj87f5yIwIsilaWQgiiu
drone-server[6999]: DEBU[0100]                                               fields.time="2020-06-19T08:27:33+02:00" latency=1.010055ms method=GET remote="127.0.0.1:50566" request=/api/user request-id=1dWk5W6IcoKJrOmkvzaXthaOygu
drone-server[6999]: DEBU[0110]                                               fields.time="2020-06-19T08:27:43+02:00" latency=2.088476ms method=GET remote="[::1]:33234" request=/api/user request-id=1dWk6quCYGeHpWnjvdfk0qqvyz5
drone-server[6999]: DEBU[0120]                                               fields.time="2020-06-19T08:27:53+02:00" latency=2.089175ms method=GET remote="127.0.0.1:50578" request=/api/user request-id=1dWk82BvmboTTbTL9WTp7zv02ok

On the gitea side I can enable SQL logging which gives me a ton of the same queries with each of the /api/v1/user/repos requests. Should I get them for a few of those and attach?

moepman commented 4 years ago

I have run tcpdump to see what gitea returns to drone: it will always send a json object containing 30 repositories of 33 in total (X-Total-Count: 33). I still don't know why drone just won't show only those 30 but rather fetches the list in an infinite loop.

CirnoT commented 4 years ago

Does drone attempt to request another page, or the request URI query stays the same all the time?

moepman commented 4 years ago

As far as I can tell drone always requests the same URL, even after minutes.

@CirnoT your PR for drone seems to suggest that this is indeed purely a problem with drone?

CirnoT commented 4 years ago

Most precisely with go-scm, and us adding proper pagination in v1.12 caused the issue to manifest between Drone and go-scm, because pagination handling is done in Drone (via checking Link header) but then request for page 2 is passed to go-scm which in turn would return page 1

moepman commented 4 years ago

I can confirm that after upgrading to the current version of drone (which includes https://github.com/drone/go-scm/pull/66) everything works fine.