search

Orange-OpenSource / its-client

This Intelligent Transportation Systems (ITS) MQTT client based on the JSon ETSI specification transcription provides a ready to connect project for the mobility (connected and autonomous vehicles, road side units, vulnerable road users,...). Let's connect your device or application to our Intelligent Transport Systems (ITS) platform!
MIT License
7 stars 8 forks source link

python/iqm: fix #103, #107; misc fixes and cleanups #108

Closed ymorin-orange closed 7 months ago

ymorin-orange commented 9 months ago

Changes:

How to test:

Note: you'll need an MQTT broker that allows anonymous connections, listening on port 1883 on local host. To display the traffic on the broker, start a mosquitto client (%t displays the topic, %p the payload): mosquitto_sub -t '#' -F '%t %p'

  1. Build and install its-iqm: 
    $ pip3 install .

    Note the installation path (e.g. /home/login/.local/bin).

  2. Prepare configuration files:
    1. the iqm.cfg configuration file: 
      [general]
instance-id = iqm-1
prefix = 5G-root
suffix = v2x
[local]
host = 127.0.0.1
port = 1883
[authority]
type = file
reload = 5
path = neighbours.cfg
[neighbours]
client_id = neighbour-1
    2. the neighbours.cfg.tmp configuration file (note the extension .tmp): 
      [neighbour-1]
type = mqtt
host = 127.0.0.1
port = 1883
prefix = other-pfx
suffix = other-sfx
queue = other-queue
  3. Start the IQM with the above configuration, in debug mode: 
    $ /home/login/.local/bin/its-iqm --debug -c iqm.cfg
  4. Send a message on the local inQueue, and on the neighbour's interQueue: 
    $ mosquitto_pub -t '5G-root/inQueue/v2x/cam/0/1/2/3' -m '{"type": "cam", "source": "local"}'
$ mosquitto_pub -t 'other-pfx/other-queue/other-sfx/cam/0/1/2/3' -m '{"type": "cam", "source": "remote"}'
  5. Tell the IQM about its neighbour: 
    $ cp neighbours.cfg.tmp neighbours.cfg
  6. Send a message on the local inQueue, and on the neighbour's interQueue: 
    $ mosquitto_pub -t '5G-root/inQueue/v2x/cam/0/1/2/3' -m '{"type": "cam", "source": "local"}'
$ mosquitto_pub -t 'other-pfx/other-queue/other-sfx/cam/0/1/2/3' -m '{"type": "cam", "source": "remote"}'
  7. Change the neighbour configuration (e.g. change its suffix): 
    $ sed -r -i -e 's/suffix =.*/suffix = yadayada/; s/queue = .*/queue = couic/' neighbours.cfg
  8. Send a message on the local inQueue, and on the neighbour's new interQueue: 
    $ mosquitto_pub -t '5G-root/inQueue/v2x/cam/0/1/2/3' -m '{"type": "cam", "source": "local"}'
$ mosquitto_pub -t 'other-pfx/couic/yadayada/cam/0/1/2/3' -m '{"type": "cam", "source": "remote"}'
  9. remove the neighbour: 
    $ rm neighbours.cfg
  10. Send a message on the local inQueue, and on the neighbour's new interQueue: 
    $ mosquitto_pub -t '5G-root/inQueue/v2x/cam/0/1/2/3' -m '{"type": "cam", "source": "local"}'
$ mosquitto_pub -t 'other-pfx/couic/yadayada/cam/0/1/2/3' -m '{"type": "cam", "source": "remote"}'
  11. Stop the IQM, by typing Ctrl-C.
  12. Configure the IQM to use an MQTT central authority:
    1. the iqm.cfg configuration file: 
      [general]
instance-id = iqm-1
prefix = 5G-root
suffix = v2x
[local]
host = 127.0.0.1
port = 1883
[authority]
type = mqtt
host = 127.0.0.1
port = 1883
client_id = iqm-1-central
topic = 5G-root/neighbours/iqm-1-central
[neighbours]
client_id = neighbour-1
    2. the neighbours.json configuration file: 
      {
  "neighbour-1": {
    "type": "mqtt",
    "host": "127.0.0.1",
    "port": 1883,
    "prefix": "other-pfx",
    "suffix": "other-sfx",
    "queue": "other-queue"
  }
}
  13. Start the IQM with the above configuration, in debug mode: 
    $ /home/login/.local/bin/its-iqm --debug -c iqm.cfg
  14. Send the neighbours configuration: 
    $ mosquitto_pub -t 5G-root/neighbours/iqm-1-central -f neighbours.json
  15. Send an empty neighbours configuration: 
    $ mosquitto_pub -t 5G-root/neighbours/iqm-1-central -m '{}'
  16. Stop the IQM, by typing Ctrl-C.

Expected results:

  1. its-iqm has been installed.
  2. The configuration files are ready and available in the current directory.
  3. The IQM is running, and reports loading 0 neighbour(s).
  4. The local CAM is forwarded to the local outQueue and interQueue, the remote CAM is not forwarded: 
    5G-root/inQueue/v2x/cam/0/1/2/3 {"type": "cam", "source": "local"}
5G-root/outQueue/v2x/cam/0/1/2/3 {"type": "cam", "source": "local"}
5G-root/interQueue/v2x/cam/0/1/2/3 {"type": "cam", "source": "local"}
other-pfx/other-queue/other-sfx/cam/0/1/2/3 {"type": "cam", "source": "remote"}
  5. The IQM reports loading 1 neighbour(s), and report subscribing to topic other-pfx/interQueue/other-sfx/#.
  6. The local CAM is forwarded to the local outQueue and interQueue, the remote CAM is forwarded to the local outQueue: 
    5G-root/inQueue/v2x/cam/0/1/2/3 {"type": "cam", "source": "local"}
5G-root/outQueue/v2x/cam/0/1/2/3 {"type": "cam", "source": "local"}
5G-root/interQueue/v2x/cam/0/1/2/3 {"type": "cam", "source": "local"}
other-pfx/other-queue/other-sfx/cam/0/1/2/3 {"type": "cam", "source": "remote"}
5G-root/outQueue/v2x/cam/0/1/2/3 {"type": "cam", "source": "remote"}
  7. The IQM reports loading 1 neighbour(s), and stops and starts the neighbour on the new topic other-pfx/couic/yadayada/#.
  8. The local CAM is forwarded to the local outQueue and interQueue, the remote CAM is forwarded to the local outQueue: 
    5G-root/inQueue/v2x/cam/0/1/2/3 {"type": "cam", "source": "local"}
5G-root/outQueue/v2x/cam/0/1/2/3 {"type": "cam", "source": "local"}
5G-root/interQueue/v2x/cam/0/1/2/3 {"type": "cam", "source": "local"}
other-pfx/couic/yadayada/cam/0/1/2/3 {"type": "cam", "source": "remote"}
5G-root/outQueue/v2x/cam/0/1/2/3 {"type": "cam", "source": "remote"}
  9. The IQM reports loading 0 neighbour(s), and stops the previous neighbour.
  10. The local CAM is forwarded to the local outQueue and interQueue, the remote CAM is not forwarded: 
    5G-root/inQueue/v2x/cam/0/1/2/3 {"type": "cam", "source": "local"}
5G-root/outQueue/v2x/cam/0/1/2/3 {"type": "cam", "source": "local"}
5G-root/interQueue/v2x/cam/0/1/2/3 {"type": "cam", "source": "local"}
other-pfx/couic/yadayada/cam/0/1/2/3 {"type": "cam", "source": "remote"}
  11. The IQM stops.
  12. The configuration files are ready and available in the current directory.
  13. The IQM is running, and does not report anything about loading any neighbours.
  14. The IQM reports receiving and loading 1 neighbour(s), and starts subscribing to the expected topic: 
    2024-02-21 09:38:56,361 mqtt: received neighbours
2024-02-21 09:38:56,362 mqtt: loaded 1 neighbour(s)
2024-02-21 09:38:56,362 iqm: stopping old neighbours (if any)...
2024-02-21 09:38:56,362 iqm: starting new neighbours (if any)...
2024-02-21 09:38:56,362 iqm: creating qm for neighbour-1
2024-02-21 09:38:56,362 mqtt_client: [neighbour-1]: create to 127.0.0.1:1883, listening on other-pfx/other-queue/other-sfx
2024-02-21 09:38:56,362 mqtt_client: [neighbour-1]: starting for 127.0.0.1:1883
2024-02-21 09:38:56,363 mqtt_client: [neighbour-1]: connected to 127.0.0.1:1883
2024-02-21 09:38:56,364 mqtt_client: [neighbour-1]: subscribed to other-pfx/other-queue/other-sfx/#
  15. The IQM reports receiving and loading 0 neighbour(s), and stops the previous neighbour.
  16. The IQM stops.
tigroo commented 8 months ago

tests 1. to 4. ok:

kvmk8371@yd-89lzhs3:~/Workspace/its-client/python/its-interqueuemanager$ /home/kvmk8371/.local/bin/its-iqm --debug -c iqm.cfg
2024-03-14 10:20:11,615 main: loading config file iqm.cfg...
2024-03-14 10:20:11,616 main: create IQM...
2024-03-14 10:20:11,616 iqm: create
2024-03-14 10:20:11,616 iqm: create local qm
2024-03-14 10:20:11,616 mqtt_client: [local]: create to 127.0.0.1:1883, listening on 5G-root/inQueue/v2x
2024-03-14 10:20:11,616 main: run IQM...
2024-03-14 10:20:11,616 mqtt_client: [local]: starting for 127.0.0.1:1883
2024-03-14 10:20:11,616 file: starting authority file client to /etc/its/neighbours.cfg@60
2024-03-14 10:20:11,616 file: loading neighbours
2024-03-14 10:20:11,617 file: loaded 0 neighbour(s)
2024-03-14 10:20:11,617 iqm: stopping old neighbours (if any)...
2024-03-14 10:20:11,617 iqm: starting new neighbours (if any)...
2024-03-14 10:20:11,617 mqtt_client: [local]: connected to 127.0.0.1:1883
2024-03-14 10:20:11,617 mqtt_client: [local]: subscribed to 5G-root/inQueue/v2x/#
2024-03-14 10:20:28,911 mqtt_client: [local]: received message on 5G-root/inQueue/v2x/cam/0/1/2/3: b'{"type": "cam", '[...]
2024-03-14 10:20:28,911 mqtt_client: [local]:  -> forwarding to 5G-root/outQueue/v2x/cam/0/1/2/3
2024-03-14 10:20:28,911 mqtt_client: [local]: publishing b'{"type": "cam", '[...] on 5G-root/outQueue/v2x/cam/0/1/2/3
2024-03-14 10:20:28,911 mqtt_client: [local]:  -> forwarding to 5G-root/interQueue/v2x/cam/0/1/2/3
2024-03-14 10:20:28,911 mqtt_client: [local]: publishing b'{"type": "cam", '[...] on 5G-root/interQueue/v2x/cam/0/1/2/3
2024-03-14 10:21:11,677 file: loading neighbours
2024-03-14 10:21:11,677 file: loaded 0 neighbour(s)
2024-03-14 10:21:11,677 iqm: stopping old neighbours (if any)...
2024-03-14 10:21:11,677 iqm: starting new neighbours (if any)...
2024-03-14 10:22:11,737 file: loading neighbours
2024-03-14 10:22:11,737 file: loaded 0 neighbour(s)
2024-03-14 10:22:11,737 iqm: stopping old neighbours (if any)...
2024-03-14 10:22:11,737 iqm: starting new neighbours (if any)...

Test 5. ko : no reload of the config file:

2024-03-14 10:45:12,719 file: loaded 0 neighbour(s)
2024-03-14 10:45:12,719 iqm: stopping old neighbours (if any)...
2024-03-14 10:45:12,719 iqm: starting new neighbours (if any)...
2024-03-14 10:46:12,779 file: loading neighbours
2024-03-14 10:46:12,779 file: loaded 0 neighbour(s)
2024-03-14 10:46:12,779 iqm: stopping old neighbours (if any)...
2024-03-14 10:46:12,779 iqm: starting new neighbours (if any)...
2024-03-14 10:47:12,839 file: loading neighbours
2024-03-14 10:47:12,839 file: loaded 0 neighbour(s)
2024-03-14 10:47:12,839 iqm: stopping old neighbours (if any)...
2024-03-14 10:47:12,839 iqm: starting new neighbours (if any)...

I note a curious log starting authority file client to /etc/its/neighbours.cfg@60 onstead of something like starting authority file client to neighbours.cfg

tigroo commented 8 months ago

test 5. and 6. ok if I put the file into the (root) directory /etc/its:

2024-03-14 11:17:08,887 file: loading neighbours
2024-03-14 11:17:08,887 file: loaded 1 neighbour(s)
2024-03-14 11:17:08,887 iqm: stopping old neighbours (if any)...
2024-03-14 11:17:08,887 iqm: starting new neighbours (if any)...
2024-03-14 11:17:14,183 mqtt_client: [neighbour-1]: received message on other-pfx/other-queue/other-sfx/cam/0/1/2/3: b'{"type": "cam", '[...]
2024-03-14 11:17:14,183 mqtt_client: [neighbour-1]:  -> forwarding to 5G-root/outQueue/v2x/cam/0/1/2/3
2024-03-14 11:17:14,183 mqtt_client: [local]: publishing b'{"type": "cam", '[...] on 5G-root/outQueue/v2x/cam/0/1/2/3
tigroo commented 8 months ago

tests 7. and 8. ok:

2024-03-14 11:24:09,193 file: loading neighbours
2024-03-14 11:24:09,193 file: loaded 1 neighbour(s)
2024-03-14 11:24:09,193 iqm: stopping old neighbours (if any)...
2024-03-14 11:24:09,193 iqm: stopping neighbour-1...
2024-03-14 11:24:09,193 mqtt_client: [neighbour-1]: stopping for 127.0.0.1:1883
2024-03-14 11:24:09,194 mqtt_client: [neighbour-1]: disconnected from 127.0.0.1:1883
2024-03-14 11:24:09,194 mqtt_client: [neighbour-1]: disconnected (socket) from 127.0.0.1:1883
2024-03-14 11:24:09,194 iqm: starting new neighbours (if any)...
2024-03-14 11:24:09,194 iqm: creating qm for neighbour-1
2024-03-14 11:24:09,194 mqtt_client: [neighbour-1]: create to 127.0.0.1:1883, listening on other-pfx/couic/yadayada
2024-03-14 11:24:09,194 mqtt_client: [neighbour-1]: starting for 127.0.0.1:1883
2024-03-14 11:24:09,195 mqtt_client: [neighbour-1]: connected to 127.0.0.1:1883
2024-03-14 11:24:09,195 mqtt_client: [neighbour-1]: subscribed to other-pfx/couic/yadayada/#
2024-03-14 11:24:38,655 mqtt_client: [local]: received message on 5G-root/inQueue/v2x/cam/0/1/2/3: b'{"type": "cam", '[...]
2024-03-14 11:24:38,655 mqtt_client: [local]:  -> forwarding to 5G-root/outQueue/v2x/cam/0/1/2/3
2024-03-14 11:24:38,655 mqtt_client: [local]: publishing b'{"type": "cam", '[...] on 5G-root/outQueue/v2x/cam/0/1/2/3
2024-03-14 11:24:38,655 mqtt_client: [local]:  -> forwarding to 5G-root/interQueue/v2x/cam/0/1/2/3
2024-03-14 11:24:38,655 mqtt_client: [local]: publishing b'{"type": "cam", '[...] on 5G-root/interQueue/v2x/cam/0/1/2/3
2024-03-14 11:25:07,768 mqtt_client: [neighbour-1]: received message on other-pfx/couic/yadayada/cam/0/1/2/3: b'{"type": "cam", '[...]
2024-03-14 11:25:07,768 mqtt_client: [neighbour-1]:  -> forwarding to 5G-root/outQueue/v2x/cam/0/1/2/3
2024-03-14 11:25:07,768 mqtt_client: [local]: publishing b'{"type": "cam", '[...] on 5G-root/outQueue/v2x/cam/0/1/2/3
2024-03-14 11:25:09,251 file: loading neighbours
2024-03-14 11:25:09,251 file: loaded 1 neighbour(s)
2024-03-14 11:25:09,251 iqm: stopping old neighbours (if any)...
2024-03-14 11:25:09,251 iqm: starting new neighbours (if any)...
tigroo commented 8 months ago

tests 9. to 11. ok:

2024-03-14 11:28:09,388 file: loading neighbours
2024-03-14 11:28:09,388 file: loaded 0 neighbour(s)
2024-03-14 11:28:09,388 iqm: stopping old neighbours (if any)...
2024-03-14 11:28:09,388 iqm: stopping neighbour-1...
2024-03-14 11:28:09,388 mqtt_client: [neighbour-1]: stopping for 127.0.0.1:1883
2024-03-14 11:28:09,388 mqtt_client: [neighbour-1]: disconnected from 127.0.0.1:1883
2024-03-14 11:28:09,389 mqtt_client: [neighbour-1]: disconnected (socket) from 127.0.0.1:1883
2024-03-14 11:28:09,389 iqm: starting new neighbours (if any)...
2024-03-14 11:29:09,449 file: loading neighbours
2024-03-14 11:29:09,449 file: loaded 0 neighbour(s)
2024-03-14 11:29:09,449 iqm: stopping old neighbours (if any)...
2024-03-14 11:29:09,449 iqm: starting new neighbours (if any)...
2024-03-14 11:29:35,772 mqtt_client: [local]: received message on 5G-root/inQueue/v2x/cam/0/1/2/3: b'{"type": "cam", '[...]
2024-03-14 11:29:35,772 mqtt_client: [local]:  -> forwarding to 5G-root/outQueue/v2x/cam/0/1/2/3
2024-03-14 11:29:35,772 mqtt_client: [local]: publishing b'{"type": "cam", '[...] on 5G-root/outQueue/v2x/cam/0/1/2/3
2024-03-14 11:29:35,772 mqtt_client: [local]:  -> forwarding to 5G-root/interQueue/v2x/cam/0/1/2/3
2024-03-14 11:29:35,772 mqtt_client: [local]: publishing b'{"type": "cam", '[...] on 5G-root/interQueue/v2x/cam/0/1/2/3

^C2024-03-14 11:30:38,387 file: stopping authority file client to /etc/its/neighbours.cfg
2024-03-14 11:30:38,387 mqtt_client: [local]: stopping for 127.0.0.1:1883
2024-03-14 11:30:38,387 mqtt_client: [local]: disconnected from 127.0.0.1:1883
2024-03-14 11:30:38,388 mqtt_client: [local]: disconnected (socket) from 127.0.0.1:1883
tigroo commented 8 months ago

test 13. ko:

kvmk8371@yd-89lzhs3:~/Workspace/its-client/python/its-interqueuemanager$ cat iqm.cfg
[general]
instance-id = iqm-1
prefix = 5G-root
suffix = v2x
[local]
host = 127.0.0.1
port = 1883
[central]
type = mqtt
host = 127.0.0.1
port = 1883
client_id = iqm-1-central
topic = 5G-root/neighbours/iqm-1-central
[neighbours]
client_id = neighbour-1kvmk8371@yd-89lzhs3:~/Workspace/its-client/python/its-interqueuemanager$ /home/kvmk8371/.local/bin/its-iqm --debug -c iqm.cfg
2024-03-14 11:37:26,452 main: loading config file iqm.cfg...
2024-03-14 11:37:26,452 main: create IQM...
2024-03-14 11:37:26,452 iqm: create
2024-03-14 11:37:26,452 iqm: create local qm
2024-03-14 11:37:26,452 mqtt_client: [local]: create to 127.0.0.1:1883, listening on 5G-root/inQueue/v2x
2024-03-14 11:37:26,452 main: run IQM...
2024-03-14 11:37:26,452 mqtt_client: [local]: starting for 127.0.0.1:1883
2024-03-14 11:37:26,453 file: starting authority file client to /etc/its/neighbours.cfg@60
2024-03-14 11:37:26,453 file: loading neighbours
2024-03-14 11:37:26,453 file: loaded 0 neighbour(s)
2024-03-14 11:37:26,453 iqm: stopping old neighbours (if any)...
2024-03-14 11:37:26,453 iqm: starting new neighbours (if any)...
2024-03-14 11:37:26,453 mqtt_client: [local]: connected to 127.0.0.1:1883
2024-03-14 11:37:26,454 mqtt_client: [local]: subscribed to 5G-root/inQueue/v2x/#

The iqm stays in file mode.

ymorin-orange commented 8 months ago

Test 5. ko : no reload of the config file: [..] I note a curious log starting authority file client to /etc/its/neighbours.cfg@60 onstead of something like starting authority file client to neighbours.cfg

Indeed, I forgot to amend the iqm.cfg: the configuration section is not central, but authority:

[general]
instance-id = iqm-1
prefix = 5G-root
suffix = v2x
[local]
host = 127.0.0.1
port = 1883
[authority]   <<=== Here
type = file
reload = 5
path = /home/nyma7486/dev/work/5GCroCo/its-client/iqm-neighbour-config/neighbours.cfg
[neighbours]
client_id = neighbour-1

I've now updated the PR description accordingly...

I suspect that's also the reason why test 13 failed...