cnr-ibf-pa / hbp-bsp-issues

Ticketing system for developers/testers and power users of the Brain Simulation Platform of the Human Brain Project
4 stars 0 forks source link

Optimization results analysis loops on some optimization results #421

Closed lbologna closed 4 years ago

lbologna commented 5 years ago

Expected behavior

After the optimization process is finished on HPC and the output is downloaded, the analysis process is triggered and positively ends. The .zip folder is successively created to be uploaded to BlueNaaS for the model to be simulated

Actual Behavior

On some optimization, the steps which involve compiling the results with neuron and run the analysis script loops indefinitely (this is visible on the backend log). The process in the backend stops when the uwsgi server interrupts, but the frontend hangs on the spinning wheel.

Steps to reproduce the problem

I am investigating this. It is not clear at the moment whether the bug depends on a server configuration which relaunches the process after a given timeout or on some other reasons (analysis scripts, django, python, ...) and the bug does not always shows up.

Optional infrastructural data (user, platform, browser, environment, ...)

Bug is present in both dev and prod environment

lbologna commented 4 years ago

After investigation on several platforms with both Chrome and Firefox browsers, I found that the problem was due to my avast antivirus, which implements a web-shield that repeats a frontend GET request if it does not receive any reply in a certain amount of time (in my case it was 2 min). This causes the nginx-uwsgi communication to be interrupted, so the processes repeat and at the same time get lost (the frontend does not receive any reply ever). Surfing the web I found that this problem may occur also in case a physical firewall is present with very short latency request allowance (we have a firewall at our institute but everything works well, so I expect this to be a rare case). In order not to have a hanged request, I implemented a timeout for the call which triggers a message with useful information for the user.

Closing the ticket.