TT-12347 remove unknown keys from edge

sredxny commented 1 month ago

User description

Description

Related Issue

TT-12347

Motivation and Context

How This Has Been Tested

Screenshots (if appropriate)

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Refactoring or add test (improvements in base code or adds test coverage to functionality)

Checklist

[ ] I ensured that the documentation is up to date
[ ] I explained why this PR updates go.mod in detail with reasoning why it's required
[ ] I would like a code coverage CI quality gate exception and have explained why

PR Type

Enhancement, Bug fix

Description

Added a new method CleanInvalidEdgeKeys to the Gateway struct to handle the removal of unknown keys from the edge.
Integrated the CleanInvalidEdgeKeys method into the Start method to ensure it runs during the server start process.
Implemented logic within CleanInvalidEdgeKeys to scan for keys with the prefix apikey-* and delete them if they are not found in the RPC listener.
Added error handling and logging for key scanning and deletion processes.

Changes walkthrough 📝

Relevant files

Enhancement

server.go

Add and integrate method to clean invalid edge keys

gateway/server.go

Added CleanInvalidEdgeKeys method to Gateway struct.

Integrated CleanInvalidEdgeKeys call in Start method.

Implemented logic to scan and delete invalid edge keys.

+30/-0

💡 PR-Agent usage: Comment /help on the PR to get a list of all available PR-Agent tools and their descriptions

github-actions[bot] commented 1 month ago

PR Reviewer Guide 🔍

⏱️ Estimated effort to review [1-5]	3
🧪 Relevant tests	No
🔒 Security concerns	No
⚡ Key issues to review	Error Handling: The method `CleanInvalidEdgeKeys` uses `log.Fatalf` which will cause the program to exit if an error occurs during the scanning of keys. Consider handling this error more gracefully to avoid crashing the entire application.
	Error Checking: The error checking in the deletion of keys seems incorrect. The statement `if err := localStorage.DeleteRawKey(key); err != true` is checking if the error is equal to `true`, which is not a typical pattern in Go for error handling. It should check for `nil` to ensure the operation was successful.
	Resource Cleanup: There is no handling of potential resource leaks or failures in the Redis connections used in `CleanInvalidEdgeKeys`. Consider adding proper cleanup or retry mechanisms.

github-actions[bot] commented 1 month ago

API Changes

--- prev.txt    2024-07-01 20:23:44.376999767 +0000
+++ current.txt 2024-07-01 20:23:41.176988563 +0000
@@ -8351,6 +8351,8 @@

 func (gw *Gateway) BuildAndLoadAPI(apiGens ...func(spec *APISpec)) (specs []*APISpec)

+func (gw *Gateway) CleanInvalidEdgeKeys()
+
 func (gw *Gateway) CoProcessInit()
     CoProcessInit creates a new CoProcessDispatcher, it will be called when Tyk
     starts.

github-actions[bot] commented 1 month ago

PR Code Suggestions ✨

Category	Suggestion	Score
Possible bug	Improve error handling in key deletion logic ___ Consider handling the error from `localStorage.DeleteRawKey(key)` properly instead of comparing the error to `true`. Use a nil check to determine if the deletion was successful. [gateway/server.go [1956]](https://github.com/TykTechnologies/tyk/pull/6381/files#diff-4652d1bf175a0be8f5e61ef7177c9666f23e077d8626b73ac9d13358fa8b525bR1956-R1956) ```diff -if err := localStorage.DeleteRawKey(key); err != true { +if err := localStorage.DeleteRawKey(key); err != nil { ``` Suggestion importance[1-10]: 9 Why: The suggestion correctly identifies a potential bug where the error handling is improperly done by comparing the error to `true` instead of checking if it is `nil`. This change is crucial for proper error handling.	9
Possible issue	Prevent server crash on recoverable errors ___ Replace the `log.Fatalf` with `log.Printf` followed by a return statement to prevent the server from exiting unexpectedly due to a recoverable error. [gateway/server.go [1950]](https://github.com/TykTechnologies/tyk/pull/6381/files#diff-4652d1bf175a0be8f5e61ef7177c9666f23e077d8626b73ac9d13358fa8b525bR1950-R1950) ```diff -log.Fatalf("Error scanning keys: %v", err) +log.Printf("Error scanning keys: %v", err) +return ``` Suggestion importance[1-10]: 8 Why: Changing `log.Fatalf` to `log.Printf` followed by a return statement prevents the server from crashing on recoverable errors, which improves the robustness of the application.	8
Maintainability	Replace magic strings with constants for better code maintainability ___ Use a constant for the key prefix "apikey-" and "quota" to avoid magic strings and facilitate maintenance. [gateway/server.go [1948-1960]](https://github.com/TykTechnologies/tyk/pull/6381/files#diff-4652d1bf175a0be8f5e61ef7177c9666f23e077d8626b73ac9d13358fa8b525bR1948-R1960) ```diff -keys, err := localStorage.ScanKeys("apikey-") -quotaKey := strings.Replace(key, "apikey", "quota", 1) +const apikeyPrefix = "apikey-" +const quotaPrefix = "quota" +keys, err := localStorage.ScanKeys(apikeyPrefix + "") +quotaKey := strings.Replace(key, apikeyPrefix, quotaPrefix, 1) ``` Suggestion importance[1-10]: 7 Why: Using constants for the key prefixes improves code readability and maintainability by avoiding magic strings. This is a good practice but not critical.	7
Enhancement	Add nil check for keys slice to handle cases where no keys are found ___ Add error handling for the `localStorage.ScanKeys` method to handle possible nil `keys` slice when an error occurs. [gateway/server.go [1948-1950]](https://github.com/TykTechnologies/tyk/pull/6381/files#diff-4652d1bf175a0be8f5e61ef7177c9666f23e077d8626b73ac9d13358fa8b525bR1948-R1950) ```diff keys, err := localStorage.ScanKeys("apikey-*") if err != nil { - log.Fatalf("Error scanning keys: %v", err) + log.Printf("Error scanning keys: %v", err) + return +} +if keys == nil { + log.Printf("No keys found to process.") + return } ``` Suggestion importance[1-10]: 7 Why: Adding a nil check for the keys slice ensures that the code handles cases where no keys are found, which enhances the robustness of the function. This is a useful enhancement but not critical.	7