CactuseSecurity / firewall-orchestrator

Network Security data repository for automation, reporting and compliance of firewall rules
https://fwo.cactus.de
Apache License 2.0
29 stars 10 forks source link

workflow - logs on test systems are flooded because of missing config entry #2450

Open tpurschke opened 2 weeks ago

tpurschke commented 2 weeks ago

Jun 13 09:26:47 srv-name fworch-ui[1660376]: 2024-06-13T09:26:47+02:00 Debug - Load Global Config Items (Config.cs in line 99), Config item with key "reqReducedView" could not be found. Using default value.

Could you have a look and maybe fix this in your current PR?

Would be good to know who/why this entry is permanently read.

tpurschke commented 2 weeks ago

not reproducible - errors seems to go away when restarting the UI the UI is the process that tries to reads the config

The following error is not the same issue:

this is triggered when changing settings - or even when visiting settings pages?

During app import when clicking into various settings pages, we get the error: Lesen der Konfiguration - Konfiguration konnte nicht gelesen oder verarbeitet werden. . Für Details in den Log-Dateien nachsehen!

- API call (GraphQlApiConnection.cs in line 121), Sending API call  in role : query getStates {  state_list: request_state (order_by: { id: asc })...
2024-06-13T11:29:27.858915+02:00 srv1 fworch-ui:  2024-06-13T11:29:27+02:00 Debug - API call (GraphQlApiConnection.cs in line 121), Sending API call  in role : query getNwGroupObjects ($grpType: Int!){  modelling_nwgroup (where: ... with variables: {"grpType":23}...
2024-06-13T11:29:34.874194+02:00 srv1 fworch-ui:  2024-06-13T11:29:34+02:00 Debug - API call (GraphQlApiConnection.cs in line 121), Sending API call  in role : query getNwGroupObjects ($grpType: Int!){  modelling_nwgroup (where: ... with variables: {"grpType":23}...
2024-06-13T11:29:40.217384+02:00 srv1 fworch-ui:  2024-06-13T11:29:40+02:00 Debug - Jwt Validation (JwtReader.cs in line 82), Jwt was successfully validated.
2024-06-13T11:29:40.217510+02:00 srv1 fworch-ui:  2024-06-13T11:29:40+02:00 Error - Lesen der Konfiguration (MainLayout.razor in line 311), User: dv884qb, Role: admin, Konfiguration konnte nicht gelesen oder verarbeitet werden.
2024-06-13T11:29:40.217554+02:00 srv1 fworch-ui:   --- 
2024-06-13T11:29:40.217602+02:00 srv1 fworch-ui:  Exception thrown:
2024-06-13T11:29:40.217639+02:00 srv1 fworch-ui:   JsonException
2024-06-13T11:29:40.217674+02:00 srv1 fworch-ui:  Message:
2024-06-13T11:29:40.217709+02:00 srv1 fworch-ui:   '0x0A' is invalid within a JSON string. The string should be correctly escaped. Path: $[0].TicketTemplate | LineNumber: 0 | BytePositionInLine: 30. 
2024-06-13T11:29:40.217744+02:00 srv1 fworch-ui:  Stack Trace:
2024-06-13T11:29:40.217779+02:00 srv1 fworch-ui:   at System.Text.Json.ThrowHelper.ReThrowWithPath(ReadStack& state, JsonReaderException ex) 
2024-06-13T11:29:40.217814+02:00 srv1 fworch-ui:     at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
2024-06-13T11:29:40.217870+02:00 srv1 fworch-ui:     at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 utf8Json, JsonTypeInfo`1 jsonTypeInfo, Nullable`1 actualByteCount)
2024-06-13T11:29:40.217898+02:00 srv1 fworch-ui:     at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 json, JsonTypeInfo`1 jsonTypeInfo)
2024-06-13T11:29:40.217926+02:00 srv1 fworch-ui:     at System.Text.Json.JsonSerializer.Deserialize[TValue](String json, JsonSerializerOptions options)
2024-06-13T11:29:40.217957+02:00 srv1 fworch-ui:     at FWO.Ui.Pages.Settings.SettingsExternalWorkflow.OnInitializedAsync() in /usr/local/fworch/ui/files/FWO.UI/Pages/Settings/SettingsExternalWorkflow.razor:line 95
2024-06-13T11:29:40.217994+02:00 srv1 fworch-ui:  FWORCHAlert - source: "ui", userId: "18", title: "Lesen der Konfiguration", description: "Konfiguration konnte nicht gelesen oder verarbeitet werden. . Für Details in den Log-Dateien nachsehen!", alertCode: "UiError"
2024-06-13T11:29:40.218029+02:00 srv1 fworch-ui:  2024-06-13T11:29:40+02:00 Debug - API call (GraphQlApiConnection.cs in line 121), Sending API call  in role : query getOpenAlerts {  alert (where: {ack_by: {_is_null: true}} order...
tpurschke commented 1 week ago
tpurschke commented 1 week ago

debugging this revealed that the config is permanently read but there seems to be no activity in-between reads. the only angle we found is to make sure that userConfig and globalConfig are correctly used to avoid circle reads between the two. Plan: include a check in unserconfig to prevent userconfigs with userid 0

Could also provoke a situation (during continuous config reading) where we could not save the modelling settings because in WriteToDatabase the semaphoreSlim.WaitAsync was not available (even though read locking was turned off at the time) . After the semaphore became available after a few seconds and was released again after the write operation, CreateSubscription was invoked with userId 0

subscription getConfigItemsByUser($userId: Int) {
  config(where: {config_user: {_eq: $userId}}) {
    config_key
    config_value
  }
}

But the save never went through (in fact no settings save was possible). Only an UI restart could fix this.