search

fli-iam / shanoir-ng

Shanoir (SHAring iN vivO Imaging Resources)
GNU General Public License v3.0
19 stars 22 forks source link

[Log management] Unify logs over the application in order to load them in an ELK stack #2141

Open jcomedouteau opened 4 months ago

jcomedouteau commented 4 months ago

Qualif = Toutes les semaines Prod = Conserver 1 mois ? 3 mois ? (Conserver les logs plus anciens compressés et ré-importables dans un dossier ?) Loguer les métriques spécifiquement ? (téléchargement, import, etc..)

%d{yyyy-MM-dd HH:mm:ss,SSS} %-5p [%c] (%t) %s%e%n Date LEVEL [Category] (Thread_name) message_exception_newline

DCM4CHEE server.log 2024-03-11 10:48:40,969 INFO [org.jboss.as] (MSC service thread 1-2) WFLYSRV0049: WildFly Full 26.1.1.Final (WildFly Core 18.1.1.Final) starting 5Mo 110Mo (6 Mois = trop)

KEYCLOAK keycloak.log 2024-03-11 10:47:48,486 INFO [org.keycloak.services] (main) KC-SERVICES0032: Import finished successfully 20Ko 100Ko

NGINX access.log 172.17.42.1 shanoir-ofsep.irisa.fr - [21/Jan/2018:06:00:00 +0100] "GET / HTTP/1.1" 301 178 "-" "check_http/v2.1.1 (monitoring-plugins 2.1.1)" "-" 100 Mo (Sur 7 jours) Pas de séparation entre les différents serveurs, à configurer

SHANOIR

shanoir-dimse.log 2023-10-10_14:30:07.293 - Start: C-STORE sending 80 dicom files to PACS from folder: /tmp/85/553739773059183051/SERIES/1.2.840.113619.2.322.4120.14227240.28341.1613978745.529/dataset0 1.6Mo 6Mo -> ANCIEN, to remove ?

shanoir-equipments.log -> Vide, création auto des équipements par Michael, à retirer ?

shanoir-events.log %-4relative [%thread] %-5level %logger{35} - %msg%n 2024-03-11_07:42:10.710 [AsyncThread-1] INFO o.s.n.s.event.ShanoirEventService - Event:[id=4035169842941084914;user_id=197;event_type=importDataset.event;object_id=62319;message=Converting to NIfTI for serie [3D T1 NEURONAV SAG] (10/10)...;report=null;status=2;progress=0.50000006] 500Ko 2.5Mo (6 mois 600Mo)

shanoir-imports.log %d{yyyy-MM-dd_HH:mm:ss.SSS} - %msg%n 2024-03-11_11:26:41.784 - user=mmoyon,size=231,1MB,userId=133,studyName=Cohorte Principale - CHU Nantes,studyCardId=null,type=SHUP,workFolder=/tmp/133/8414402644722892675,pseudoProfile=null,modality=MR,e nhanced=false,subjectName=1910177,examId=62341,converterId=6,numberOfSeries=20,seriesNames=[fl3d1_ns,fl3d1_ns,tir2d1_11,fl3d1_ns,tir2d1_11,fl3d1_ns,null,null,fl3d1_ns,tir2d1_11,fl3d1_ns,fl3d1_n s,fl3d1_ns,fl3d1_ns,fl3d1_ns,fl3d1_ns,fl3d1_ns,fl3d1_ns,fl3d1_ns,fl3d1_ns] 5Ko 50Ko

shanoir-ng-datasets.log 2024-03-11T01:47:21.173+01:00 WARN 1 --- [async-1] .s.n.v.m.s.ExecutionStatusMonitorService : Stopping thread... 100Ko 500Ko (6 mois = 100Mo)

shanoir-ng-import.log 2024-03-11T16:21:11.647+01:00 ERROR 1 --- [AsyncThread-1] .DatasetsCreatorAndNIfTIConverterService : removeUnusedFiles : error while deleting /tmp/54/3924882054380907799/SERIES/1.3.46.670589.11.21769.5.0 .6484.2022042014544835059/xDWIogSENSEs1501a015.bvec 100Ko 200Ko

shanoir-ng-preclinical.log 2024-03-11T04:13:05.174+01:00 INFO 1 --- [main] o.s.ng.ShanoirPreclinicalApplication : Started ShanoirPreclinicalApplication in 52.192 seconds (process running for 61.324) 2Ko 5Ko

shanoir-ng-studies.log 2024-03-11T07:35:56.255+01:00 INFO 1 --- [http-nio-9902-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring DispatcherServlet 'dispatcherServlet' 10Ko 80Ko

shanoir-ng-users.log 2024-03-11T04:13:01.273+01:00 INFO 1 --- [main] o.s.s.web.DefaultSecurityFilterChain : Will secure any request with [org.springframework.security.web.session.DisableEncodeUrlFilter@61d8a98b, org.spr 150Ko 500Ko

youennmerel commented 4 months ago

https://stackoverflow.com/questions/74054123/how-to-add-current-user-details-in-log-spring-boot

youennmerel commented 4 months ago

As discussed with you, also unify the way object id are referenced in the logs, e.g. ... dataset [123] ..., ... center [123] ..., ... study [123] ..., ... subject [123] ..., ... examination [123] ..., etc so that we can trace the path of an object through the microservices