search

Baltic-RCC / EMF

Repository for Open Source EMF implementation
Mozilla Public License 2.0
8 stars 3 forks source link

Store validation error log data to ELK #56

Closed mr0321 closed 1 month ago

mr0321 commented 3 months ago

Regarding to #55 current implementation is to add custom logger that gathers pypowsybl logs (setLevel(1)) and in case of a pypowsybl log.error saves it to file and sends it to elastic. From the elastic the idea is to use the index "emfos-logs" (currently stores other log events regarding to running emf) TODO: fill the columns/fields, figure out in which field the log file (byte array) will be stored

mr0321 commented 3 months ago

Entry that triggers the log saving can be saved to elastic (uses same connection parameters as elk logger, index must be determined by setting the ELASTIC_INDEX_FOR_PYPOWSYBL_LOGS in custom_logger.properties). Due to the fact that entire log (if logger is set to ALL_ENTRIES, ENTRIES_IF_LEVEL_REACHED or ENTRIES_COLLECTED_TO_LEVEL) can be rather large, then currently they are stored to Minio (uses parameters from minio.properties, bucket and subfolders can be set by using appropriate parameters in custom_logger.properties)