Allowlist constraint violation

[pcfreak30]

Current Behavior

couldn't update allow-list entries: transaction failed (attempt 1): failed to join host allowlist entry: FOREIGN KEY constraint failed

Expected Behavior

No errors

Steps to Reproduce

Try to add a hostkey to the allow-list twice

Version

v1.0.8

What operating system did the problem occur on (e.g. Ubuntu 22.04, macOS 12.0, Windows 11)?

Ubuntu

Autopilot Config

n/a

Bus Config

n/a

Contract Set Contracts

n/a

Anything else?

No response

[peterjan]

Can't reproduce it on both SQLite and MySQL. Can you reproduce it consistently?

[pcfreak30]

Yes, I just modified my host scanner code to test and this is crashing consistently. Im using the bus client at v1.0.8 as well go.sia.tech/renterd v1.0.8 for my renter service (which calls the bus for AddHostsToAllowlist, https://github.com/LumeWeb/portal/blob/51a9c1747da066e4b51ca04f931f064a517fd240/service/renter.go#L390) .

2024-11-04T08:54:48.218-0500    ERROR   renter/host_scanner.go:609      Failed to update host allowlist {"hostkey": "ed25519:d2d65a5cfd1d23cd826d394e42ef055350b0b0faf0ab9580ae6e4806fd5d2571", "error": "couldn't update allowlist entries: transaction failed (attempt 1): failed to join host allowlist entry: FOREIGN KEY constraint failed\n"}
2024-11-04T08:54:48.264-0500    ERROR   renter/host_scanner.go:609      Failed to update host allowlist {"hostkey": "ed25519:a90d3c26a22d66903c06a1bf869e14e829e95cfa25b6bf08189c98713fc92449", "error": "couldn't update allowlist entries: transaction failed (attempt 1): failed to join host allowlist entry: FOREIGN KEY constraint failed\n"}
2024-11-04T08:54:48.303-0500    ERROR   renter/host_scanner.go:609      Failed to update host allowlist {"hostkey": "ed25519:4298a298dd84bc48028f7bcf267e8b779a41fa671df3924cf47199fc7d344e2c", "error": "couldn't update allowlist entries: transaction failed (attempt 1): failed to join host allowlist entry: FOREIGN KEY constraint failed\n"}
2024-11-04T08:54:48.344-0500    ERROR   renter/host_scanner.go:609      Failed to update host allowlist {"hostkey": "ed25519:e89e13affe9d2ab4dc6f1e157376c60cdcadddf061ea78a52db68b63e6070ee4", "error": "couldn't update allowlist entries: transaction failed (attempt 1): failed to join host allowlist entry: FOREIGN KEY constraint failed\n"}
2024-11-04T08:54:48.389-0500    ERROR   renter/host_scanner.go:609      Failed to update host allowlist {"hostkey": "ed25519:85740f1238a363e39583ede79abf2026baeb5b83f7b5b1e0f5045a1fe421784a", "error": "couldn't update allowlist entries: transaction failed (attempt 1): failed to join host allowlist entry: FOREIGN KEY constraint failed\n"}
2024-11-04T08:54:48.430-0500    ERROR   renter/host_scanner.go:609      Failed to update host allowlist {"hostkey": "ed25519:45699ce33ed8161894e315d40fa9c8da85e815a2b8be1f1169a03bd098fe7a3c", "error": "couldn't update allowlist entries: transaction failed (attempt 1): failed to join host allowlist entry: FOREIGN KEY constraint failed\n"}

func (s *HostScanner) addNewHostsToAllowlist(ctx context.Context, newHosts []HostscoreHost, existingHosts map[types.PublicKey]struct{}) error {
    // Extract just the public keys
    newHostKeys := lo.Map(newHosts, func(h HostscoreHost, _ int) types.PublicKey {
        return h.PublicKey
    })

    var err error

    for _, pk := range newHostKeys {
        if err = s.renter.AddHostsToAllowlist(ctx, []types.PublicKey{pk}); err != nil {
            s.logger.Error("Failed to update host allowlist", zap.String("hostkey", pk.String()), zap.Error(err))
        }
    }

    if err != nil {
        return fmt.Errorf("failed to add host to allowlist: %w", err)
    }

    /*  // Add to renterd's allowlist
        if err := s.renter.AddHostsToAllowlist(ctx, newHostKeys); err != nil {
            return fmt.Errorf("failed to update host allowlist: %w", err)
        }*/

    // Update our tracking map
    for _, pk := range newHostKeys {
        existingHosts[pk] = struct{}{}
    }

    s.logger.Info("Added new hosts to allowlist",
        zap.Int("newHostsAdded", len(newHostKeys)))

    return nil
}

[ChrisSchinnerl]

@pcfreak30 what database are you using? If you are up for it you can run the queries manually with the keys that you know are failing to help us get to the bottom of this. The method is called UpdateHostAllowlistEntries and depending on whether you run MySQL or SQLite the query is slightly different.

[pcfreak30]

Running this on SQLite, and sure, I have no issues debugging it for you. I did give this to AI though and will have to test, but it suggested it might be due to adding a host to the whitelist that doesn't exist in the hosts table. While that would be a bug in my code, if that's the case, it should be handled in renterd too.