Add opv-status support

[zyioump]

Opv-status is bind on port 80 with nginx. If you want to test don't forget to install a firefox extension like CORS Everywhere to use opv-status-api

[Benvii]

@Valdimus what do you think ? (see my global review comment)

[Benvii]

Tested in new containers, import didn't work got the following message from the status api :

Exception : 'ascii' codec can't decode byte 0xc2 in position 97: ordinal not in range(128)

Nothing in the log file /home/opv/logs/status-api.log there is only access logs. Need further investigation. I won't accept something that's not working even if the problem comes from an other repository/module we need to find it and fix it before adding it to the Ansible stable version. Stable means it need to be working ;)

I will try to investigate it this afternoon but first of all I need to know how I can had some logs. I will try to install the status API in development mode and reconfigure the logs to have more details and see if I can reproduce it or not.

[Benvii]

Ok, found it after 30min of investigation.

The bug comes from this line in opv_import meta CSV parser. According to the documentation of open (see Unicode Basics bullet 9), for texte file (here it's RedebroCSV) python uses by default the local prefered encoding. I've checked when it's started when systemctl locale.getpreferredencoding() returns ANSI_X3.4-1968 whereas when we start it from a shell it returns UTF-8.

He diged a bit further and compared the return of the command locale, here is what it returns in a shell :

(opv) opv@statusmaster:~$ locale
LANG=en_US.UTF-8
LANGUAGE=
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

Here is what it returns behind systemctl :

LANG=
LANGUAGE=
LC_CTYPE="POSIX"
LC_NUMERIC="POSIX"
LC_TIME="POSIX"
LC_COLLATE="POSIX"
LC_MONETARY="POSIX"
LC_MESSAGES="POSIX"
LC_PAPER="POSIX"
LC_NAME="POSIX"
LC_ADDRESS="POSIX"
LC_TELEPHONE="POSIX"
LC_MEASUREMENT="POSIX"
LC_IDENTIFICATION="POSIX"
LC_ALL=

Clearly the locale isn't set behind systemctl, so I assume that pythons locale.getpreferredencoding() default to ASCII because it's the old behavior of python.

To set the local we could simply add the following line to launch_status-api.sh :

#!/bin/bash

source /home/opv/venvs/opv/bin/activate

export LC_ALL=en_US.UTF-8
opv-status-api &>> /home/opv/logs/status-api.log

A better way would be to set the system default local to do so simply edit /etc/locale.conf and add this line :

$ cat /etc/locale.conf 
LC_ALL=en_US.UTF-8

[Benvii]

So now import is working, spark launch is working (when campaign name doesn't contains spaces). I opened an issue about it. https://github.com/OpenPathView/opv-status-api/issues/4

[Benvii]

Got a wrong hostname in spark interface (so Rest API doesn't work as the URL is wrong) :

I'm not sure it's related to this PR. @Valdimus where is this configured ?