Gravity update failure - Githubissues

threeonparfive commented 1 year ago

What happened?

Most of the time, the Gravity update fails at the end of the sync process. When this happens, the DNS resolves ends up in a down state and the only way to get it working again seems to be to manually run the Gravity update on the target Pi-hole. I tried flipped the update gravity option to "false" which does of course prevent the error but this too seems to cause the DNS resolver to fail to start after the sync has completed.

Version

Latest as of today

Log output

2/1/2023, 10:08:39 PM: ➡️ Signing in to http://192.168.1.5:8080/admin...
stdout
17:08:39
2/1/2023, 10:08:39 PM: ✔️ Successfully signed in to http://192.168.1.5:8080/admin!
stdout
17:08:39
2/1/2023, 10:08:39 PM: ➡️ Downloading backup from http://192.168.1.5:8080/admin...
stdout
17:08:41
2/1/2023, 10:08:41 PM: ✔️ Backup from http://192.168.1.5:8080/admin completed!
stdout
17:08:41
2/1/2023, 10:08:41 PM: ➡️ Signing in to http://192.168.1.10/admin...
stdout
17:08:41
2/1/2023, 10:08:41 PM: ✔️ Successfully signed in to http://192.168.1.10/admin!
stdout
17:08:41
2/1/2023, 10:08:41 PM: ➡️ Uploading backup to http://192.168.1.10/admin...
stdout
17:08:44
2/1/2023, 10:08:44 PM: ✔️ Backup uploaded to http://192.168.1.10/admin!
stdout
17:08:44
2/1/2023, 10:08:44 PM: ➡️ Updating gravity on http://192.168.1.10/admin...
stdout
17:08:48
2/1/2023, 10:08:48 PM: ⚠ Error: Failed updating gravity on "http://192.168.1.10/admin".
stderr
17:08:48
2/1/2023, 10:08:48 PM: ⚠ Failure: 0/1 hosts synced.
stderr
17:08:48
2/1/2023, 10:08:48 PM: Waiting 30 minutes...

ch33f commented 1 year ago

Try set SYNC_LOCALDNSRECORDS and SYNC_LOCALCNAMERECORDS to false. It seems that teleporter has Problems with importing Local DNS Records

threeonparfive commented 1 year ago

Try set SYNC_LOCALDNSRECORDS and SYNC_LOCALCNAMERECORDS to false. It seems that teleporter has Problems with importing Local DNS Records

I just tried only setting CNAME records to false as I don't use them anyway and this didn't resolve the issue. Unfortunately for me, local DNS records are the primary thing I'm trying to sync in the first place.

ch33f commented 1 year ago

Me also :) Work-Around: i set up an second Container to only Sync Local DNS with INTERVAL_MINUTES 1440 (24 hours). Cronjob in Pihole with restarting the FTL Service. The error is in teleporter not in orbital-sync Try backup and import via Web-Ui, after importing with Local DNS the piohole-ftl will crash

threeonparfive commented 1 year ago

Me also :) Work-Around: i set up an second Container to only Sync Local DNS with INTERVAL_MINUTES 1440 (24 hours). Cronjob in Pihole with restarting the FTL Service. The error is in teleporter not in orbital-sync Try backup and import via Web-Ui, after importing with Local DNS the piohole-ftl will crash

I figured it was a Pihole bug. I'm glad you found a workaround but if I'm going to go to that level of trouble, I'll either stick with an occasional manual run (and required baby sitting) or try Gravity Sync again. I use to use Gravity Sync with 2 Piholes both on Pis and it worked very well. However now my primary Pihole is running in Docker on a Synology which makes setting up Gravity Sync a bit more tedious.

I've also found another bug, probably with teleporter, where if group syncing is enabled, it syncs the groups but doesn't maintain the configured state properly. I have a couple of groups disabled on the primary for clients I specifically don't want Pihole to block anything for but on the restore to the secondary, it enables those groups.

I was thrilled when I stumbled upon Orbital Sync because it's SO much simpler! Sucks that it's having to rely on an unreliable Pihole feature.

I do appreciate your help though. Thanks!

ch33f commented 1 year ago

How often do you add/delete local DNS records?

threeonparfive commented 1 year ago

How often do you add/delete local DNS records?

Not very. And I really am only doing it because my router, like most it seems, doesn't support reverse lookup resolution for DHCP client names (support for conditional forwarding). So for clients I care most about seeing in Pihole, I've resorted to DHCP reservations and local DNS entries. I've toyed with the idea of making my Pihole the DHCP server but then you have to do a bit of "hacking" to make Pihole's DHCP service give out the address of a secondary Pihole as it doesn't natively support this. Plus, I use the TP-Link Omada system where I can give actual friendly names to all clients rather than the often worthless hostnames built in to the device (like IoT devices). So even if the router supported reverse lookup OR I used Pihole DHCP, I still wouldn't have very useful names for many clients anyway due to poor hostnaming.

mattwebbio commented 1 year ago

Yeah, unfortunately as @threeonparfive says this is a Pi-hole issue. I'm going to close this issue but please let me know if for some reason this turns out to be an issue with Orbital.

Sitehand commented 1 year ago

I have just setup Orbital-Sync and have run into the same issue, with it reporting failed updating gravity.

Log Output

8/8/2023, 11:57:06 AM: ➡️ Uploading backup to http://10.1.1.1:8001/admin...
8/8/2023, 11:57:09 AM: ✔️ Backup uploaded to http://10.1.1.1:8001/admin!
8/8/2023, 11:57:09 AM: ➡️ Updating gravity on http://10.1.1.1:8001/admin...
8/8/2023, 11:57:13 AM: ⚠ Error: Failed updating gravity on "http://10.1.1.1:8001/admin".
8/8/2023, 11:57:13 AM: ⚠ Failure: 0/1 hosts synced.
8/8/2023, 11:57:13 AM: Waiting 30 minutes...

Had a quick look at the code. And the client.ts in the following section, is look for "Pi-hole blocking is enabled" to be returned at the end of the gravity update. If you have ad blocking turned off, then "Pi-hole blocking is disabled" line is returned instead. So the update gravity completed successfully under both instances, it just orbital-sync thinks it has failed. Perhaps it could check if either response is returned.

    if (Config.updateGravity) {
      Log.info(chalk.yellow(`➡️ Updating gravity on ${this.host.fullUrl}...`));
      const gravityUpdateResponse = await this.fetch(
        `${this.host.fullUrl}/scripts/pi-hole/php/gravity.sh.php`,
        { method: 'GET' }
      );

      const updateText = (await gravityUpdateResponse.text())
        .replaceAll('\ndata:', '')
        .trim();
      if (
        gravityUpdateResponse.status !== 200 ||
        !updateText.endsWith('Pi-hole blocking is enabled')
      )
        throw new ErrorNotification({
          message: `Failed updating gravity on "${this.host.fullUrl}".`,
          verbose: {
            host: this.host.baseUrl,
            path: this.host.path,
            status: gravityUpdateResponse.status,
            eventStream: updateText
          }
        });

      Log.info(chalk.green(`✔️ Gravity updated on ${this.host.fullUrl}!`));
      Log.verbose(`Result:\n${chalk.blue(updateText)}`);
    }

apainter2 commented 4 days ago

This is exact same issue for me, the one pi-hole is configured that its only a DNS resolver, so I have the blocking feature disabled. When I enable blocking on this instance, it completes with out errors.

Did you find a fix?

I have just setup Orbital-Sync and have run into the same issue, with it reporting failed updating gravity.

Log Output

8/8/2023, 11:57:06 AM: ➡️ Uploading backup to http://10.1.1.1:8001/admin...
8/8/2023, 11:57:09 AM: ✔️ Backup uploaded to http://10.1.1.1:8001/admin!
8/8/2023, 11:57:09 AM: ➡️ Updating gravity on http://10.1.1.1:8001/admin...
8/8/2023, 11:57:13 AM: ⚠ Error: Failed updating gravity on "http://10.1.1.1:8001/admin".
8/8/2023, 11:57:13 AM: ⚠ Failure: 0/1 hosts synced.
8/8/2023, 11:57:13 AM: Waiting 30 minutes...

Had a quick look at the code. And the client.ts in the following section, is look for "Pi-hole blocking is enabled" to be returned at the end of the gravity update. If you have ad blocking turned off, then "Pi-hole blocking is disabled" line is returned instead. So the update gravity completed successfully under both instances, it just orbital-sync thinks it has failed. Perhaps it could check if either response is returned.

    if (Config.updateGravity) {
      Log.info(chalk.yellow(`➡️ Updating gravity on ${this.host.fullUrl}...`));
      const gravityUpdateResponse = await this.fetch(
        `${this.host.fullUrl}/scripts/pi-hole/php/gravity.sh.php`,
        { method: 'GET' }
      );

      const updateText = (await gravityUpdateResponse.text())
        .replaceAll('\ndata:', '')
        .trim();
      if (
        gravityUpdateResponse.status !== 200 ||
        !updateText.endsWith('Pi-hole blocking is enabled')
      )
        throw new ErrorNotification({
          message: `Failed updating gravity on "${this.host.fullUrl}".`,
          verbose: {
            host: this.host.baseUrl,
            path: this.host.path,
            status: gravityUpdateResponse.status,
            eventStream: updateText
          }
        });

      Log.info(chalk.green(`✔️ Gravity updated on ${this.host.fullUrl}!`));
      Log.verbose(`Result:\n${chalk.blue(updateText)}`);
    }

mattwebbio / orbital-sync

Gravity update failure #136

What happened?

Version

Log output