search

qwc-services / qwc-config-generator

QWC Config Generator
2 stars 12 forks source link

Improve logging for schema validation, so that location of errors & warnings in data service config are given in terms of field/layer names #72

Open w0pr opened 8 months ago

w0pr commented 8 months ago

running from docker image: sourcepole/qwc-config-generator:v2024.03.05

Errors in warnings such as below are difficult to debug, because Location is only given in internal terms with index numbers like

Location: .resources.datasets[9].fields[34].data_type

and no names of the underlying columns tables/layers are output.

Code location is here: https://github.com/qwc-services/qwc-config-generator/blob/c847713e0d5cc652b1e874bc16b5a2210e613d86/src/config_generator/config_generator.py#L640

qwc-config-service-1           | 2024-03-21T10:16:43.077071162Z [2024-03-21 10:16:43,076] ERROR in config_generator: Validation error: None is not of type 'string'
qwc-config-service-1           | 2024-03-21T10:16:43.077181108Z [2024-03-21 10:16:43,077] WARNING in config_generator: Location: .resources.datasets[9].fields[34].data_type
qwc-config-service-1           | 2024-03-21T10:16:43.077210245Z [2024-03-21 10:16:43,077] WARNING in config_generator: Value: null
qwc-config-service-1           | 2024-03-21T10:16:43.077346083Z [2024-03-21 10:16:43,077] ERROR in config_generator: Validation error: None is not one of ['USER-DEFINED', 'bigint', 'boolean', 'character', 'character varying', 'date', 'double precision', 'file', 'integer', 'json', 'jsonb', 'numeric', 'real', 'smallint', 'text', 'time', 'timestamp with time zone', 'timestamp without time zone', 'uuid']
qwc-config-service-1           | 2024-03-21T10:16:43.077372045Z [2024-03-21 10:16:43,077] WARNING in config_generator: Location: .resources.datasets[9].fields[34].data_type
qwc-config-service-1           | 2024-03-21T10:16:43.077526059Z [2024-03-21 10:16:43,077] WARNING in config_generator: Value: null
qwc-config-service-1           | 2024-03-21T10:16:43.077653144Z [2024-03-21 10:16:43,077] ERROR in config_generator: Validation error: None is not of type 'string'
qwc-config-service-1           | 2024-03-21T10:16:43.077679526Z [2024-03-21 10:16:43,077] WARNING in config_generator: Location: .resources.datasets[9].fields[35].data_type
qwc-config-service-1           | 2024-03-21T10:16:43.077760100Z [2024-03-21 10:16:43,077] WARNING in config_generator: Value: null
qwc-config-service-1           | 2024-03-21T10:16:43.077914280Z [2024-03-21 10:16:43,077] ERROR in config_generator: Validation error: None is not one of ['USER-DEFINED', 'bigint', 'boolean', 'character', 'character varying', 'date', 'double precision', 'file', 'integer', 'json', 'jsonb', 'numeric', 'real', 'smallint', 'text', 'time', 'timestamp with time zone', 'timestamp without time zone', 'uuid']
qwc-config-service-1           | 2024-03-21T10:16:43.077978757Z [2024-03-21 10:16:43,077] WARNING in config_generator: Location: .resources.datasets[9].fields[35].data_type
qwc-config-service-1           | 2024-03-21T10:16:43.078356226Z [2024-03-21 10:16:43,078] WARNING in config_generator: Value: null
qwc-config-service-1           | 2024-03-21T10:16:43.078521141Z [2024-03-21 10:16:43,078] ERROR in config_generator: Validation error: None is not of type 'string'
qwc-config-service-1           | 2024-03-21T10:16:43.078619132Z [2024-03-21 10:16:43,078] WARNING in config_generator: Location: .resources.datasets[9].fields[36].data_type
qwc-config-service-1           | 2024-03-21T10:16:43.078703416Z [2024-03-21 10:16:43,078] WARNING in config_generator: Value: null
qwc-config-service-1           | 2024-03-21T10:16:43.078773329Z [2024-03-21 10:16:43,078] ERROR in config_generator: Validation error: None is not one of ['USER-DEFINED', 'bigint', 'boolean', 'character', 'character varying', 'date', 'double precision', 'file', 'integer', 'json', 'jsonb', 'numeric', 'real', 'smallint', 'text', 'time', 'timestamp with time zone', 'timestamp without time zone', 'uuid']
qwc-config-service-1           | 2024-03-21T10:16:43.078855555Z [2024-03-21 10:16:43,078] WARNING in config_generator: Location: .resources.datasets[9].fields[36].data_type
qwc-config-service-1           | 2024-03-21T10:16:43.078922571Z [2024-03-21 10:16:43,078] WARNING in config_generator: Value: null
qwc-config-service-1           | 2024-03-21T10:16:43.086756520Z [2024-03-21 10:16:43,086] ERROR in config_generator: 'data' service config failed schema validation
qwc-config-service-1           | 2024-03-21T10:16:43.114184166Z [2024-03-21 10:16:43,114] CRITICAL in config_generator: A critical error occurred while processing the configuration.
qwc-config-service-1           | 2024-03-21T10:16:43.114205933Z [2024-03-21 10:16:43,114] CRITICAL in config_generator: The configuration files were not updated!
qwc-config-service-1           | 2024-03-21T10:16:43.326807024Z [2024-03-21 10:16:43,326] CRITICAL in config_generator: A critical error occurred while processing the configuration.
qwc-config-service-1           | 2024-03-21T10:16:43.326836718Z [2024-03-21 10:16:43,326] CRITICAL in config_generator: The permission files were not updated!