search

permitio / opal

Policy and data administration, distribution, and real-time updates on top of Policy Agents (OPA, Cedar, ...)
https://opal.ac
Apache License 2.0
5.15k stars 180 forks source link

opal-server stops pulling new updates if the policy repo master branch has a force push #99

Open hongbo-miao opened 3 years ago

hongbo-miao commented 3 years ago

opal-server stops pulling new updates if the policy repo master branch has a force push (replace the last one commit with a new one).

Here is the log, it just stops further pulling without any error. And current pulling will be pending.

Logs ```shell [2021-07-12 19:21:33 +0000] [1] [INFO] Starting gunicorn 20.1.0 [2021-07-12 19:21:33 +0000] [1] [INFO] Listening at: http://0.0.0.0:7002 (1) [2021-07-12 19:21:33 +0000] [1] [INFO] Using worker: uvicorn.workers.UvicornWorker [2021-07-12 19:21:33 +0000] [10] [INFO] Booting worker with pid: 10 [2021-07-12 19:21:33 +0000] [11] [INFO] Booting worker with pid: 11 [2021-07-12 19:21:33 +0000] [12] [INFO] Booting worker with pid: 12 [2021-07-12 19:21:33 +0000] [13] [INFO] Booting worker with pid: 13 2021-07-12T19:21:34.654948+0000 |opal_common.authentication.signer | INFO | OPAL was not provided with JWT encryption keys, cannot verify api requests! 2021-07-12T19:21:34.655596+0000 |opal_common.authentication.signer | INFO | OPAL was not provided with JWT encryption keys, cannot verify api requests! 2021-07-12T19:21:34.661812+0000 |opal_common.authentication.signer | INFO | OPAL was not provided with JWT encryption keys, cannot verify api requests! 2021-07-12T19:21:34.668544+0000 |opal_common.authentication.signer | INFO | OPAL was not provided with JWT encryption keys, cannot verify api requests! 2021-07-12T19:21:34.726044+0000 |opal_server.server | INFO | triggered startup event 2021-07-12T19:21:34.726328+0000 |opal_common.topics.publisher | INFO | started topic publisher 2021-07-12T19:21:34.727299+0000 |opal_server.server | INFO | leadership lock acquired, leader pid: 11 2021-07-12T19:21:34.727466+0000 |opal_server.server | INFO | listening on webhook topic: 'webhook' 2021-07-12T19:21:34.728867+0000 |fastapi_websocket_pubsub.event_notifier | INFO | New subscription {'id': '5d84fef30f3f4f0a8f712adee48a1bfd', 'subscriber_id': '8e5cdf788008461e9a0d900294df08ce', 'topic': 'webhook', 'callback': functools.partial(, ), 'notifier_id': None} 2021-07-12T19:21:34.729054+0000 |opal_server.policy.watcher.task | INFO | Launching repo watcher 2021-07-12T19:21:34.729679+0000 |opal_common.git.repo_cloner | INFO | Cloning repo from 'https://github.com/Hongbo-Miao/opal-example-policy-repo' to '/regoclone' 2021-07-12T19:21:34.746472+0000 |opal_server.server | INFO | triggered startup event 2021-07-12T19:21:34.752982+0000 |opal_common.topics.publisher | INFO | started topic publisher 2021-07-12T19:21:34.764244+0000 |opal_server.server | INFO | triggered startup event 2021-07-12T19:21:34.764488+0000 |opal_common.topics.publisher | INFO | started topic publisher 2021-07-12T19:21:34.765975+0000 |opal_server.server | INFO | triggered startup event 2021-07-12T19:21:34.766253+0000 |opal_common.topics.publisher | INFO | started topic publisher 2021-07-12T19:21:37.484660+0000 |opal_common.git.repo_cloner | INFO | Clone succeeded 2021-07-12T19:21:37.491395+0000 |opal_common.git.repo_watcher | INFO | Launching polling task, interval: 30 seconds 2021-07-12T19:21:59.811384+0000 |fastapi_websocket_pubsub.event_broadc...| INFO | Listening for incoming events from broadcast channel (first listener started) 2021-07-12T19:21:59.812194+0000 |fastapi_websocket_pubsub.event_broadc...| INFO | Spawning broadcast listen task 2021-07-12T19:21:59.877097+0000 |fastapi_websocket_pubsub.event_broadc...| INFO | Subscribing to ALL TOPICS, and sharing messages with broadcast channel 2021-07-12T19:21:59.878474+0000 |fastapi_websocket_pubsub.event_notifier | INFO | New subscription {'id': '450b2e2b53934279991697fee7738013', 'subscriber_id': '16eb38578f7446728d09c53f9adbf579', 'topic': '__EventNotifier_ALL_TOPICS__', 'callback': >, 'notifier_id': None} 2021-07-12T19:21:59.880866+0000 |fastapi_websocket_rpc.websocket_rpc_e...| INFO | Client connected 2021-07-12T19:21:59.881216+0000 |fastapi_websocket_pubsub.event_broadc...| INFO | Starting broadcaster listener 2021-07-12T19:21:59.949840+0000 |fastapi_websocket_pubsub.event_notifier | INFO | New subscription {'id': 'e08d36109cda402c8860d1cdf755e057', 'subscriber_id': '25b6419659f54c1696d9781a8cbaa0de', 'topic': 'policy_data', 'callback': .callback at 0x7fd96c47f310>, 'notifier_id': None} 2021-07-12T19:22:07.458130+0000 |opal_common.git.repo_watcher | INFO | Pulling changes from remote: 'origin' 2021-07-12T19:22:09.299170+0000 |opal_common.git.repo_watcher | INFO | No new commits: HEAD is at 'b7480e393d5a21086a07d8da6c99ff2516f270f4' 2021-07-12T19:22:39.266851+0000 |opal_common.git.repo_watcher | INFO | Pulling changes from remote: 'origin' 2021-07-12T19:22:41.104471+0000 |opal_common.git.repo_watcher | INFO | No new commits: HEAD is at 'b7480e393d5a21086a07d8da6c99ff2516f270f4' 2021-07-12T19:23:11.071210+0000 |opal_common.git.repo_watcher | INFO | Pulling changes from remote: 'origin' 2021-07-12T19:23:12.905227+0000 |opal_common.git.repo_watcher | INFO | No new commits: HEAD is at 'b7480e393d5a21086a07d8da6c99ff2516f270f4' 2021-07-12T19:23:42.872009+0000 |opal_common.git.repo_watcher | INFO | Pulling changes from remote: 'origin' 2021-07-12T19:23:44.737825+0000 |opal_common.git.repo_watcher | INFO | No new commits: HEAD is at 'b7480e393d5a21086a07d8da6c99ff2516f270f4' 2021-07-12T19:24:14.702912+0000 |opal_common.git.repo_watcher | INFO | Pulling changes from remote: 'origin' 2021-07-12T19:24:16.494131+0000 |opal_common.git.repo_watcher | INFO | No new commits: HEAD is at 'b7480e393d5a21086a07d8da6c99ff2516f270f4' 2021-07-12T19:24:46.461508+0000 |opal_common.git.repo_watcher | INFO | Pulling changes from remote: 'origin' 2021-07-12T19:24:48.373691+0000 |opal_common.git.repo_watcher | INFO | No new commits: HEAD is at 'b7480e393d5a21086a07d8da6c99ff2516f270f4' 2021-07-12T19:25:18.342024+0000 |opal_common.git.repo_watcher | INFO | Pulling changes from remote: 'origin' 2021-07-12T19:25:20.176742+0000 |opal_common.git.repo_watcher | INFO | No new commits: HEAD is at 'b7480e393d5a21086a07d8da6c99ff2516f270f4' 2021-07-12T19:25:50.142963+0000 |opal_common.git.repo_watcher | INFO | Pulling changes from remote: 'origin' 2021-07-12T19:25:51.972437+0000 |opal_common.git.repo_watcher | INFO | No new commits: HEAD is at 'b7480e393d5a21086a07d8da6c99ff2516f270f4' 2021-07-12T19:26:21.938838+0000 |opal_common.git.repo_watcher | INFO | Pulling changes from remote: 'origin' 2021-07-12T19:26:23.796473+0000 |opal_common.git.repo_watcher | INFO | No new commits: HEAD is at 'b7480e393d5a21086a07d8da6c99ff2516f270f4' 2021-07-12T19:26:53.764178+0000 |opal_common.git.repo_watcher | INFO | Pulling changes from remote: 'origin' 2021-07-12T19:26:55.618927+0000 |opal_common.git.repo_watcher | INFO | No new commits: HEAD is at 'b7480e393d5a21086a07d8da6c99ff2516f270f4' 2021-07-12T19:27:25.584704+0000 |opal_common.git.repo_watcher | INFO | Pulling changes from remote: 'origin' 2021-07-12T19:27:27.492180+0000 |opal_common.git.repo_watcher | INFO | No new commits: HEAD is at 'b7480e393d5a21086a07d8da6c99ff2516f270f4' 2021-07-12T19:27:57.458852+0000 |opal_common.git.repo_watcher | INFO | Pulling changes from remote: 'origin' ```

This is a very rare case as usually we don't force push master branch so definitely a low priority bug.

However, just leave this here as I was curious what gonna happen if I do a force push. 😅

asafc commented 3 years ago

Mmm.. i can definitely see why a force push would interfere with updates. The branch tracker does a git pull to see if new commits are available - but if you forced push to the remote - a git pull will not work.

i'll need to fix the branch tracker to: 1) fetch instead of pull 2) identify if the new commits are rewriting history 3) either rebase or git reset if force push is detected (probably should be configurable)

i.e: that would work (source):

$ git fetch origin
$ git reset --hard origin/master # Destroys your work
gemanor commented 2 months ago

Acceptance criteria for the issue:

@asafc please confirm