tests: CI/CD pipeline (tmt) - agent-flood test is failing due the change of the default Network in the qm.container

[dougsland]

Related to: https://github.com/containers/qm/pull/409

[Yarboa]

@dougsland need to change qm quadlet to change network=host before running And then return it to previous state

[dougsland]

@dougsland need to change qm quadlet to change network=host before running And then return it to previous state

Hi @Yarboa, after speaking with you today I was wondering here.... In the end I think this will solve partially the problem. It will get back to the green state for the CI/CD for now BUT we need to cover tests against network=private, see #417.

Maybe write tests plans which we can set a variable for network type? Later we use this variable for the tests?

A mock example:

Test plan:

environment:
    NETWORK: private

or

environment:
   NETWORK: host

Later we could use in our tests:

(Another mock example, something like:)

test plan file:

environment:
   NETWORK: host

test file using the test plan above which NETWORK is set to host:

tests = []
    ...

    # 
    tests.append(test_qm_issue_12345)
    tests.append(test_qm_issue_67785)
    tests.append(test_qm_issue_56675)

    if NETWORK == host:
        tests.append(skip_agent_flooder_as_cannot_reach_bluechi_host_in_another_network) 

    if NETWORK == private:
        tests.append(specific_test_for_private_network)
    ...

tests.run()

How about this?

[pengshanyu]

Hi @dougsland , I think I may not fully get your idea, so have a few questions:

if NETWORK == host:
        tests.append(skip_agent_flooder_as_cannot_reach_bluechi_host_in_another_network) 

1) Should agent-flood be executed against NETWORK == host ?

2) May I ask which test cases need to run under NETWORK: host and which test cases need to run under NETWORK: private? 3) For "we need to cover tests against network=private", do you mean we need to add some new test cases to test when NETWORK == private?

[dougsland]

Hi @dougsland , I think I may not fully get your idea, so have a few questions:

if NETWORK == host:
        tests.append(skip_agent_flooder_as_cannot_reach_bluechi_host_in_another_network) 

1. Should agent-flood be executed against NETWORK == host ?

You are correct. The agent-flood doesn't work well when enabled in the private mode.

2. May I ask which test cases need to run under NETWORK: host and which test cases need to run under NETWORK: private?

At this moment, the only one that needed to be skipped in host private is the agent-flood. All the rest must be equal for both.

3. For "we need to cover tests against network=private", do you mean we need to add some new test cases to test when NETWORK == private?

At this moment no, but the code must be modular enough if we need to set private or host be easy as an if network == private or if network == host

make sense? Please let me know if you have any additional questions.

[pengshanyu]

Is the function we expect as follows (if I understand correctly): 1) Make a common function to modify the value of NETWORK in qm to host or private 2) Then, in the test plan, set the environment variable, when NETWORK==private agent-flood (at this moment) runs, when NETWORK==host, other test cases run. 3) In this way, it seems need to maintain an array tests = [] of test cases, and add a environment variable NETWORK in the test plan as you mentioned in https://github.com/containers/qm/issues/416#issuecomment-2089040206

One thing that confuses me is that in this way, the test plan is executed once will not execute all test cases, and different NETWORK values ​​are needed to cover all test cases. Is this what we expect? Thanks.

[dougsland]

Is the function we expect as follows (if I understand correctly):

1. Make a common function to modify the value of NETWORK in qm to host or private
2. Then, in the test plan, set the environment variable, when NETWORK==private agent-flood (at this moment) runs, when NETWORK==host, other test cases run.

Please run all tests in both (of course, agent-flood is only possible in NETWORK==private)

3. In this way, it seems need to maintain an array tests = [] of test cases, and add a environment variable NETWORK in the test plan as you mentioned in tests: CI/CD pipeline (tmt) - agent-flood test is failing due the change of the default Network in the qm.container #416 (comment)

One thing that confuses me is that in this way, the test plan is executed once will not execute all test cases, and different NETWORK values ​​are needed to cover all test cases. Is this what we expect? Thanks.

No, we need to execute all tests in both scenarios (network = hosts and network = private) but as github actions + network private + agent_flood are not working well we need to skip it only in private. Make sense?

[pengshanyu]

I seem to get your meaning a little bit. In this case, do we need to get the value of the current QM network before executing the test case, and then decide what value to assign to the NETWORK variable? Or, here we are just focusing on CI/CD. And in CI/CD, the QM network is private by default, so we just need to exclude agent-flood directly.

[dougsland]

@pengshanyu something like below, (never really tested in CI/CD the below code):

summary: "Run tests with different network settings"
execute:
  how: tmt
  script: ./run_tests.sh
environment:
  NETWORK: "host"
  TESTS: ["test1", "test2", "test3", "agent-flood"]
  SKIP_TESTS: ["agent-flood"]

#!/bin/bash

# Function to modify the value of NETWORK
set_network() {
    export NETWORK=$1
    echo "NETWORK is set to $NETWORK"
}

# Array of all tests
tests=("test1" "test2" "test3" "agent-flood")

# Function to run tests
run_tests() {
    for test in "${tests[@]}"; do
        if [[ "$NETWORK" == "private" && "$test" == "agent-flood" ]]; then
            echo "Skipping $test in private network"
            continue
        fi
        echo "Running $test in $NETWORK network"
        # Add the command to execute the test here
        # For example: ./run_single_test.sh "$test"
    done
}

# Run tests with NETWORK=host
set_network "host"
run_tests

# Run tests with NETWORK=private
set_network "private"
run_tests

[pengshanyu]

@dougsland Thank you so much for the details. What confuses me is that should we add the above functions as a new test plan to replace the existing plan (such as plans/e2e/ffi) ? I feel that it should be used to replace the discover strategy in the existing plan.

[dougsland]

@dougsland Thank you so much for the details. What confuses me is that should we add the above functions as a new test plan to replace the existing plan (such as plans/e2e/ffi) ? I feel that it should be used to replace the discover strategy in the existing plan.

That make sense but would like to see the patch to make sure we are in the same place. Also having @Yarboa eyes agreeing it would be nice.

[pengshanyu]

Hi @dougsland @Yarboa, I would like to confirm with you what kind of function expected. Do we want to use this newly added test plan to replace the current ffi test plan? I feel that we should only change the strategy of the discover test case in the current ffi plan, something like if NETWORK is private, then exclude agent-flood. I have read the manual of tmt, but there is no solution yet.

Another question is what kind of operation mode we expect (that is, where does the original value of NETWORK come from):

1) Before executing tmt, know the status of the network in the current QM environment, and then assign a value to the NETWORK environment variable in the tmt command line to execute the corresponding test case.

2) Do not know the network status of QM before the test is executed. Read the status of the QM network before the test case is executed, then assign a value to NETWORK in the code, and then select the corresponding test case. Thanks.

[pengshanyu]

I tried modifying “plans/e2e/ffi.fmf” and run it with tmt -c scenario=ffi -c qm_network=private run ... , then "agent-flood" did not run.

adjust+:
  - discover+:
        - name: qm network host
          how: fmf
          filter: tag:ffi
    when: qm_network == host

  - discover+:
        - name: qm network private
          how: fmf
          filter: tag:ffi
          exclude:
            - /tests/ffi/agent-flood
    when: qm_network == private

Does this make sense?

[dougsland]

I tried modifying “plans/e2e/ffi.fmf” and run it with tmt -c scenario=ffi -c qm_network=private run ... , then "agent-flood" did not run.

adjust+:
  - discover+:
        - name: qm network host
          how: fmf
          filter: tag:ffi
    when: qm_network == host

  - discover+:
        - name: qm network private
          how: fmf
          filter: tag:ffi
          exclude:
            - /tests/ffi/agent-flood
    when: qm_network == private

Does this make sense?

My guess this is because we force the ENV to SKIP it: https://github.com/containers/qm/blob/main/tests/ffi/agent-flood/main.fmf#L2

[pengshanyu]

no, I remember SKIP_TEST. When I tested the code above, I changed https://github.com/containers/qm/blob/main/tests/ffi/agent-flood/main.fmf#L2 to test: /bin/bash ./test.sh

[dougsland]

@Yarboa do you remember something like this ? Yes it was removed due to the quadlet changes based on RA interview

I think the test could detect the quadlet Network content,

In all other network use cases other then private it should not run

[Yarboa]

@dougsland @alexlarsson let me suggest this, for BluchiController ->qm-bluchi-agent we need one port open I think that QM quadlet can use ExposeHostPort=842 or either as templated file. with the relevant port change

So that is not working, since there is a need to port forward from --network-private of the qm to bluechi-controller host port,

IS there recommended way, to do the port forward from podman private network to controller host-port?

To open the connection for qm.host agent

root@localhost ~]# bluechictl status
NODE                          | STATE     | IP             | LAST SEEN                   
==========================================================================================
host                          | online    | 0.0.0.0        | now                         
qm.host                       | offline   |                | never                       

IIRC, Bluechi should deal with that noise for malicious activities. Isnt it?

[Yarboa]

Many thanks to @alexlarsson

Last update here. There is connectivity from private network outside, so the following should be updated ControllerHost=$(hostname -I | awk '{print $1}')

cat /etc/qm/bluechi/agent.conf [bluechi-agent] NodeName=qm.host ControllerHost=${ControllerHost}

Explanation: Using this ip 10.0.2.15 instead of podman network 10.88.0.X or localhost for ControllerHost

[pengshanyu]

@Yarboa thanks for the info. Do you mean that we need to change the configuration of bluechi agent for ControllerHost in qm so that the agent-flood test can be run in the private network. Is my understanding correct?

[Yarboa]

Yes, and the same goes for IP address of tests, the agent flood should connect to that host

[pengshanyu]

Hi @Yarboa , I modified the configuration mentioned above, but “bluechi-testers" still cannot start, could you help to take a look, the error is as follows:

bash-5.1# systemctl status bluechi-tester-1
× bluechi-tester-1.service - bluechi-tester-1
     Loaded: loaded (/etc/containers/systemd/bluechi-tester-1.container; generated)
     Active: failed (Result: exit-code) since Tue 2024-07-02 06:53:40 UTC; 41s ago
   Duration: 258ms
    Process: 290 ExecStart=/usr/bin/podman run --name=systemd-bluechi-tester-1 --cidfile=/run/bluechi-tester-1.cid --replace --rm --cgroups=split --network=host --sdnotify=conmon -d dir:/var/lib/containers/registry/tools-ffi:latest /root/tests/FFI/bin/bluechi-tester --url=tcp:host=10.0.2.15,port=842 --nodename=bluechi-tester-1 --numbersignals=11111111 --signal=JobDone (code=exited, status=1/FAILURE)
    Process: 308 ExecStopPost=/usr/bin/podman rm -v -f -i --cidfile=/run/bluechi-tester-1.cid (code=exited, status=0/SUCCESS)
   Main PID: 290 (code=exited, status=1/FAILURE)
        CPU: 694ms

Jul 02 06:53:39 36e4867a4c67 systemd-bluechi-tester-1[290]:     from dasbus.client.handler import ClientObjectHandler
Jul 02 06:53:39 36e4867a4c67 systemd-bluechi-tester-1[290]:   File "/usr/local/lib/python3.9/site-packages/dasbus/client/handler.py", line 29, in <module>
Jul 02 06:53:39 36e4867a4c67 systemd-bluechi-tester-1[290]:     from dasbus.typing import get_variant, get_variant_type, unwrap_variant
Jul 02 06:53:39 36e4867a4c67 systemd-bluechi-tester-1[290]:   File "/usr/local/lib/python3.9/site-packages/dasbus/typing.py", line 27, in <module>
Jul 02 06:53:39 36e4867a4c67 systemd-bluechi-tester-1[290]:     import gi
Jul 02 06:53:39 36e4867a4c67 systemd-bluechi-tester-1[290]: ModuleNotFoundError: No module named 'gi'
Jul 02 06:53:39 36e4867a4c67 podman[294]: 2024-07-02 06:53:39.932070225 +0000 UTC m=+0.073649292 container died f045fd627389e6c6b07ccca9bc5b0b98596cac9f701af7058a345af1cf2ce8a2 (image=0d6cdb63df5247cf445055f0feb4aa8ca3e8c56eebe0fc1a56b2fa504e74a160, name=systemd-bluechi-tester-1, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, PODMAN_SYSTEMD_UNIT=bluechi-tester-1.service, io.buildah.version=1.34.0, org.label-schema.build-date=20240625, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image)
Jul 02 06:53:39 36e4867a4c67 podman[294]: 2024-07-02 06:53:39.956008022 +0000 UTC m=+0.097587029 container remove f045fd627389e6c6b07ccca9bc5b0b98596cac9f701af7058a345af1cf2ce8a2 (image=0d6cdb63df5247cf445055f0feb4aa8ca3e8c56eebe0fc1a56b2fa504e74a160, name=systemd-bluechi-tester-1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, PODMAN_SYSTEMD_UNIT=bluechi-tester-1.service, io.buildah.version=1.34.0, org.label-schema.build-date=20240625, org.label-schema.license=GPLv2)
Jul 02 06:53:39 36e4867a4c67 systemd[1]: bluechi-tester-1.service: Main process exited, code=exited, status=1/FAILURE
Jul 02 06:53:40 36e4867a4c67 systemd[1]: bluechi-tester-1.service: Failed with result 'exit-code'.

[root@ibm-p8-kvm-03-guest-02 agent.conf.d]# pwd
/etc/qm/bluechi/agent.conf.d

[root@ibm-p8-kvm-03-guest-02 agent.conf.d]# cat agent.conf 
[bluechi-agent]
NodeName=qm.localrootfs
ControllerHost=10.0.2.15

[dougsland]

Jul 02 06:53:39 36e4867a4c67 systemd-bluechi-tester-1[290]: ModuleNotFoundError: No module named 'gi'

Could you please try:

sudo yum install -y python3-gobject

[pengshanyu]

@dougsland thanks. I tried to install python3-gobject on the system, and the above error still occurs. Should I install it inside qm?

[dougsland]

@dougsland thanks. I tried to install python3-gobject on the system, and the above error still occurs. Should I install it inside qm?

Looks like it's inside the "bluechi-tester" system. Might try to install qm: https://github.com/containers/qm/tree/main/docs/devel#installing-software-inside-qm-partition

[pengshanyu]

still got this error.

[root@ibm-p8-kvm-03-guest-02 ~]# podman exec -it qm bash
bash-5.1# 
bash-5.1# rpm -qa | grep python3-gobject
python3-gobject-base-noarch-3.40.1-6.el9.noarch
python3-gobject-base-3.40.1-6.el9.x86_64
python3-gobject-3.40.1-6.el9.x86_64
bash-5.1# 
bash-5.1# systemctl status bluechi-tester-1
× bluechi-tester-1.service - bluechi-tester-1
     Loaded: loaded (/etc/containers/systemd/bluechi-tester-1.container; generated)
     Active: failed (Result: exit-code) since Tue 2024-07-02 08:09:04 UTC; 2min 32s ago
   Duration: 443ms
    Process: 113 ExecStart=/usr/bin/podman run --name=systemd-bluechi-tester-1 --cidfile=/run/bluechi-tester-1.cid --replace --rm --cgroups=split --network=host --sdnotify=conmon -d dir:/var/lib/containers/registry/tools-ffi:latest /root/tests/FFI/bin/bluechi-tester --url=tcp:host=10.0.2.15,port=842 --nodename=bluechi-tester-1 --numbersignals=11111111 --signal=JobDone (code=exited, status=1/FAILURE)
    Process: 149 ExecStopPost=/usr/bin/podman rm -v -f -i --cidfile=/run/bluechi-tester-1.cid (code=exited, status=0/SUCCESS)
   Main PID: 113 (code=exited, status=1/FAILURE)
        CPU: 47.999s

Jul 02 08:09:03 2e8c77c43899 systemd-bluechi-tester-1[113]:   File "/usr/local/lib/python3.9/site-packages/dasbus/client/handler.py", line 29, in <module>
Jul 02 08:09:03 2e8c77c43899 systemd-bluechi-tester-1[113]:     from dasbus.typing import get_variant, get_variant_type, unwrap_variant
Jul 02 08:09:03 2e8c77c43899 systemd-bluechi-tester-1[113]:   File "/usr/local/lib/python3.9/site-packages/dasbus/typing.py", line 27, in <module>
Jul 02 08:09:03 2e8c77c43899 systemd-bluechi-tester-1[113]:     import gi
Jul 02 08:09:03 2e8c77c43899 systemd-bluechi-tester-1[113]: ModuleNotFoundError: No module named 'gi'
Jul 02 08:09:03 2e8c77c43899 podman[117]: 2024-07-02 08:09:03.89708643 +0000 UTC m=+0.074575414 container died 2129b5d4af57d81a8d863f340bb126a39ec0e7ef5651f67dcff5dc425e7b3d40 (image=0d6cdb63df5247cf445055f0feb4aa8ca3e8c56eebe0fc1a56b2fa504e74a160, name=systemd-bluechi-tester-1, org.label-schema.vendor=CentOS, PODMAN_SYSTEMD_UNIT=bluechi-tester-1.service, io.buildah.version=1.34.0, org.label-schema.build-date=20240625, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0)
Jul 02 08:09:03 2e8c77c43899 podman[117]: 2024-07-02 08:09:03.92766683 +0000 UTC m=+0.105155725 container remove 2129b5d4af57d81a8d863f340bb126a39ec0e7ef5651f67dcff5dc425e7b3d40 (image=0d6cdb63df5247cf445055f0feb4aa8ca3e8c56eebe0fc1a56b2fa504e74a160, name=systemd-bluechi-tester-1, io.buildah.version=1.34.0, org.label-schema.build-date=20240625, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, PODMAN_SYSTEMD_UNIT=bluechi-tester-1.service)
Jul 02 08:09:04 2e8c77c43899 systemd[1]: bluechi-tester-1.service: Main process exited, code=exited, status=1/FAILURE
Jul 02 08:09:04 2e8c77c43899 systemd[1]: bluechi-tester-1.service: Failed with result 'exit-code'.
Jul 02 08:09:04 2e8c77c43899 systemd[1]: bluechi-tester-1.service: Consumed 47.999s CPU time.

[dougsland]

Okay, let's understand what's going on:

First, let's reproduce the issue with a raw CentOS9:

(main) $ podman run -it --name centos9-python-test quay.io/centos/centos:stream9 /bin/bash
Trying to pull quay.io/centos/centos:stream9...
Getting image source signatures
Copying blob 0d9665cbb1e4 done   |
Copying config e0618e7a2b done   |
Writing manifest to image destination

[root@c60fbf3ca03e /]# python3 -c "from gi.repository import GLib; print(GLib)"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'gi'

As shown above we have reproduced exactly the issue. Now how to fix it?

[root@c60fbf3ca03e /]# dnf install python3-gobject -y
CentOS Stream 9 - BaseOS                                                                                4.4 MB/s | 8.1 MB     00:01
CentOS Stream 9 - AppStream                                                                             5.3 MB/s |  20 MB     00:03
CentOS Stream 9 - Extras packages                                                                        20 kB/s |  17 kB     00:00
Last metadata expiration check: 0:00:01 ago on Wed Jul  3 06:52:33 2024.
Dependencies resolved.
========================================================================================================================================
 Package                                      Architecture            Version                          Repository                  Size
========================================================================================================================================
Installing:
 python3-gobject                              x86_64                  3.40.1-6.el9                     appstream                   17 k
Installing dependencies:
 cairo                                        x86_64                  1.17.4-7.el9                     appstream                  660 k
 cairo-gobject                                x86_64                  1.17.4-7.el9                     appstream                   19 k
 fontconfig                                   x86_64                  2.14.0-2.el9                     appstream                  297 k
 freetype                                     x86_64                  2.10.4-9.el9                     baseos                     388 k
 gobject-introspection                        x86_64                  1.68.0-11.el9                    baseos                     250 k
 graphite2                                    x86_64                  1.3.14-9.el9                     baseos                      95 k
 harfbuzz                                     x86_64                  2.7.4-10.el9                     baseos                     624 k
 libX11                                       x86_64                  1.7.0-9.el9                      appstream                  646 k
 libX11-common                                noarch                  1.7.0-9.el9                      appstream                  205 k
 libXau                                       x86_64                  1.0.9-8.el9                      appstream                   31 k
 libXext                                      x86_64                  1.3.4-8.el9                      appstream                   40 k
 libXrender                                   x86_64                  0.9.10-16.el9                    appstream                   28 k
 libbrotli                                    x86_64                  1.0.9-6.el9                      baseos                     314 k
 libpng                                       x86_64                  2:1.6.37-12.el9                  baseos                     117 k
 libxcb                                       x86_64                  1.13.1-9.el9                     appstream                  243 k
 pixman                                       x86_64                  0.40.0-6.el9                     appstream                  269 k
 python3-cairo                                x86_64                  1.20.1-1.el9                     appstream                   93 k
 python3-gobject-base                         x86_64                  3.40.1-6.el9                     baseos                     183 k
 python3-gobject-base-noarch                  noarch                  3.40.1-6.el9                     baseos                     161 k
 xml-common                                   noarch                  0.6.3-58.el9                     appstream                   32 k

Transaction Summary
========================================================================================================================================
Install  21 Packages

Total download size: 4.6 M
Installed size: 13 M
Downloading Packages:
CentOS Stream 9 - AppStream                     192% [==================================================================================(1/21): graphite2-1.3.14-9.el9.x86_64.rpm                                                               274 kB/s |  95 kB     00:00
(2/21): gobject-introspection-1.68.0-11.el9.x86_64.rpm                                                  620 kB/s | 250 kB     00:00
(3/21): freetype-2.10.4-9.el9.x86_64.rpm                                                                846 kB/s | 388 kB     00:00
(4/21): libpng-1.6.37-12.el9.x86_64.rpm                                                                 920 kB/s | 117 kB     00:00
(5/21): libbrotli-1.0.9-6.el9.x86_64.rpm                                                                1.6 MB/s | 314 kB     00:00
(6/21): harfbuzz-2.7.4-10.el9.x86_64.rpm                                                                2.0 MB/s | 624 kB     00:00
(7/21): python3-gobject-base-noarch-3.40.1-6.el9.noarch.rpm                                             2.1 MB/s | 161 kB     00:00
(8/21): python3-gobject-base-3.40.1-6.el9.x86_64.rpm                                                    1.8 MB/s | 183 kB     00:00
(9/21): cairo-gobject-1.17.4-7.el9.x86_64.rpm                                                            37 kB/s |  19 kB     00:00
(10/21): fontconfig-2.14.0-2.el9.x86_64.rpm                                                             388 kB/s | 297 kB     00:00
(11/21): cairo-1.17.4-7.el9.x86_64.rpm                                                                  758 kB/s | 660 kB     00:00
(12/21): libX11-common-1.7.0-9.el9.noarch.rpm                                                           2.3 MB/s | 205 kB     00:00
(13/21): libXau-1.0.9-8.el9.x86_64.rpm                                                                  433 kB/s |  31 kB     00:00
(14/21): libXext-1.3.4-8.el9.x86_64.rpm                                                                 509 kB/s |  40 kB     00:00
(15/21): libX11-1.7.0-9.el9.x86_64.rpm                                                                  1.4 MB/s | 646 kB     00:00
(16/21): libXrender-0.9.10-16.el9.x86_64.rpm                                                            408 kB/s |  28 kB     00:00
(17/21): libxcb-1.13.1-9.el9.x86_64.rpm                                                                 2.7 MB/s | 243 kB     00:00
(18/21): pixman-0.40.0-6.el9.x86_64.rpm                                                                 3.0 MB/s | 269 kB     00:00
(19/21): python3-cairo-1.20.1-1.el9.x86_64.rpm                                                          1.0 MB/s |  93 kB     00:00
(20/21): python3-gobject-3.40.1-6.el9.x86_64.rpm                                                        192 kB/s |  17 kB     00:00
(21/21): xml-common-0.6.3-58.el9.noarch.rpm                                                             428 kB/s |  32 kB     00:00
----------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                   1.7 MB/s | 4.6 MB     00:02
CentOS Stream 9 - BaseOS                                                                                1.6 MB/s | 1.6 kB     00:00
Importing GPG key 0x8483C65D:
 Userid     : "CentOS (CentOS Official Signing Key) <security@centos.org>"
 Fingerprint: 99DB 70FA E1D7 CE22 7FB6 4882 05B5 55B3 8483 C65D
 From       : /etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
Key imported successfully
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                                                1/1
  Installing       : libpng-2:1.6.37-12.el9.x86_64                                                                                 1/21
  Installing       : pixman-0.40.0-6.el9.x86_64                                                                                    2/21
  Running scriptlet: xml-common-0.6.3-58.el9.noarch                                                                                3/21
  Installing       : xml-common-0.6.3-58.el9.noarch                                                                                3/21
  Installing       : libXau-1.0.9-8.el9.x86_64                                                                                     4/21
  Installing       : libxcb-1.13.1-9.el9.x86_64                                                                                    5/21
  Installing       : libX11-common-1.7.0-9.el9.noarch                                                                              6/21
  Installing       : libX11-1.7.0-9.el9.x86_64                                                                                     7/21
  Installing       : libXext-1.3.4-8.el9.x86_64                                                                                    8/21
  Installing       : libXrender-0.9.10-16.el9.x86_64                                                                               9/21
  Installing       : libbrotli-1.0.9-6.el9.x86_64                                                                                 10/21
  Installing       : graphite2-1.3.14-9.el9.x86_64                                                                                11/21
  Installing       : harfbuzz-2.7.4-10.el9.x86_64                                                                                 12/21
  Installing       : freetype-2.10.4-9.el9.x86_64                                                                                 13/21
  Installing       : fontconfig-2.14.0-2.el9.x86_64                                                                               14/21
  Running scriptlet: fontconfig-2.14.0-2.el9.x86_64                                                                               14/21
  Installing       : cairo-1.17.4-7.el9.x86_64                                                                                    15/21
  Installing       : cairo-gobject-1.17.4-7.el9.x86_64                                                                            16/21
  Installing       : python3-cairo-1.20.1-1.el9.x86_64                                                                            17/21
  Installing       : gobject-introspection-1.68.0-11.el9.x86_64                                                                   18/21
  Installing       : python3-gobject-base-noarch-3.40.1-6.el9.noarch                                                              19/21
  Installing       : python3-gobject-base-3.40.1-6.el9.x86_64                                                                     20/21
  Installing       : python3-gobject-3.40.1-6.el9.x86_64                                                                          21/21
  Running scriptlet: fontconfig-2.14.0-2.el9.x86_64                                                                               21/21
  Running scriptlet: python3-gobject-3.40.1-6.el9.x86_64                                                                          21/21
  Verifying        : freetype-2.10.4-9.el9.x86_64                                                                                  1/21
  Verifying        : gobject-introspection-1.68.0-11.el9.x86_64                                                                    2/21
  Verifying        : graphite2-1.3.14-9.el9.x86_64                                                                                 3/21
  Verifying        : harfbuzz-2.7.4-10.el9.x86_64                                                                                  4/21
  Verifying        : libbrotli-1.0.9-6.el9.x86_64                                                                                  5/21
  Verifying        : libpng-2:1.6.37-12.el9.x86_64                                                                                 6/21
  Verifying        : python3-gobject-base-3.40.1-6.el9.x86_64                                                                      7/21
  Verifying        : python3-gobject-base-noarch-3.40.1-6.el9.noarch                                                               8/21
  Verifying        : cairo-1.17.4-7.el9.x86_64                                                                                     9/21
  Verifying        : cairo-gobject-1.17.4-7.el9.x86_64                                                                            10/21
  Verifying        : fontconfig-2.14.0-2.el9.x86_64                                                                               11/21
  Verifying        : libX11-1.7.0-9.el9.x86_64                                                                                    12/21
  Verifying        : libX11-common-1.7.0-9.el9.noarch                                                                             13/21
  Verifying        : libXau-1.0.9-8.el9.x86_64                                                                                    14/21
  Verifying        : libXext-1.3.4-8.el9.x86_64                                                                                   15/21
  Verifying        : libXrender-0.9.10-16.el9.x86_64                                                                              16/21
  Verifying        : libxcb-1.13.1-9.el9.x86_64                                                                                   17/21
  Verifying        : pixman-0.40.0-6.el9.x86_64                                                                                   18/21
  Verifying        : python3-cairo-1.20.1-1.el9.x86_64                                                                            19/21
  Verifying        : python3-gobject-3.40.1-6.el9.x86_64                                                                          20/21
  Verifying        : xml-common-0.6.3-58.el9.noarch                                                                               21/21

Installed:
  cairo-1.17.4-7.el9.x86_64                   cairo-gobject-1.17.4-7.el9.x86_64                  fontconfig-2.14.0-2.el9.x86_64
  freetype-2.10.4-9.el9.x86_64                gobject-introspection-1.68.0-11.el9.x86_64         graphite2-1.3.14-9.el9.x86_64
  harfbuzz-2.7.4-10.el9.x86_64                libX11-1.7.0-9.el9.x86_64                          libX11-common-1.7.0-9.el9.noarch
  libXau-1.0.9-8.el9.x86_64                   libXext-1.3.4-8.el9.x86_64                         libXrender-0.9.10-16.el9.x86_64
  libbrotli-1.0.9-6.el9.x86_64                libpng-2:1.6.37-12.el9.x86_64                      libxcb-1.13.1-9.el9.x86_64
  pixman-0.40.0-6.el9.x86_64                  python3-cairo-1.20.1-1.el9.x86_64                  python3-gobject-3.40.1-6.el9.x86_64
  python3-gobject-base-3.40.1-6.el9.x86_64    python3-gobject-base-noarch-3.40.1-6.el9.noarch    xml-common-0.6.3-58.el9.noarch

Complete!
[root@c60fbf3ca03e /]# python3 -c "from gi.repository import GLib; print(GLib)"
<GLibProxyModule <IntrospectionModule 'GLib' from '/usr/lib64/girepository-1.0/GLib-2.0.typelib'>>
[root@c60fbf3ca03e /]#
# cat /etc/redhat-release
CentOS Stream release 9

So in summary, dnf install python3-gobject -y seems enough. Maybe restart qm service? @Yarboa ideas? It's getting late for me here.

@pengshanyu do you mind to send a patch to BlueChi adding into README.md that we need to install python3-gobject ? Please NOTE this README ----> https://github.com/eclipse-bluechi/bluechi/tree/main/tests/tools/FFI

cc @engelmi @mwperina @pbrilla-rh

thank you

[pengshanyu]

Hi @dougsland, sure, let me sort out my thoughts and then prepare the patch. I restarted qm service after installed python3-gobject , but still have the same problem. I tried to manually start a bluechi-agent using the /etc/bluechi/agent.conf.d/agent.conf configuration file to connect to the bluechi-controller in ASIL partion, and the network communication in this solution is OK. For the ModuleNotFoundError: No module named 'gi' problem, I saw someone say that it may be a problem with the symbolic link between python, python3, and python3.9. Is it possible that this might be the issue? Thanks.

[engelmi]

Adding my 5cents to the gi module missing issue:

The gi module is required by dasbus - and since this is used in the bluechi-tester, you need to install these modules on the machine/container you run it. The package installed via dnf resolves this dependency properly apparently (it requires the python3-gobject in its spec file). So doing a

$ dnf install python3-dasbus

should also take care of installing python3-gobject.

So I am wondering: Which container image you use? @pengshanyu @dougsland If I understand it correctly, its quay.io/repository/centos-sig-automotive/ffi-tools - is that correct? If yes, then I assume this image is managed, build and pushed in https://gitlab.com/CentOS/automotive/container-images/ffi-tools? Looking there at the Containerfile in Line 82:

...
RUN pip install dasbus
...

dasbus gets installed via pip and not dnf, so it doesn't install the required python3-gobject package (for centos, at least).

So I'd propose to replace this pip with a dnf install. This should then fix this issue at its root.
I can provide a small MR if you want @dougsland @pengshanyu

[pengshanyu]

@engelmi Thank you so much for the reply. Yes, you are right, the container image we using is quay.io/repository/centos-sig-automotive/ffi-tools Sure, please feel free to file MR, thanks.

[pengshanyu]

The MR that installed python3-gobject has been merged. This solution works, agent-flood is passed, I will prepare a PR accordingly.

Jul 03 21:38:40 ibm-p8-kvm-03-guest-02.virt.pnr.lab.eng.rdu2.redhat.com bluechi-controller[9979]: Registered managed node from fd 9 as 'localrootfs'
Jul 03 21:39:47 ibm-p8-kvm-03-guest-02.virt.pnr.lab.eng.rdu2.redhat.com bluechi-controller[9979]: Registered managed node from fd 10 as 'bluechi-tester-1'
Jul 03 21:39:48 ibm-p8-kvm-03-guest-02.virt.pnr.lab.eng.rdu2.redhat.com bluechi-controller[9979]: Registered managed node from fd 11 as 'bluechi-tester-2'
Jul 03 21:59:01 ibm-p8-kvm-03-guest-02.virt.pnr.lab.eng.rdu2.redhat.com bluechi-controller[9979]: Node 'bluechi-tester-1' disconnected
Jul 03 21:59:16 ibm-p8-kvm-03-guest-02.virt.pnr.lab.eng.rdu2.redhat.com bluechi-controller[9979]: Node 'bluechi-tester-2' disconnected

[dougsland]

The MR that installed python3-gobject has been merged. This solution works, agent-flood is passed, I will prepare a PR accordingly.

Jul 03 21:38:40 ibm-p8-kvm-03-guest-02.virt.pnr.lab.eng.rdu2.redhat.com bluechi-controller[9979]: Registered managed node from fd 9 as 'localrootfs'
Jul 03 21:39:47 ibm-p8-kvm-03-guest-02.virt.pnr.lab.eng.rdu2.redhat.com bluechi-controller[9979]: Registered managed node from fd 10 as 'bluechi-tester-1'
Jul 03 21:39:48 ibm-p8-kvm-03-guest-02.virt.pnr.lab.eng.rdu2.redhat.com bluechi-controller[9979]: Registered managed node from fd 11 as 'bluechi-tester-2'
Jul 03 21:59:01 ibm-p8-kvm-03-guest-02.virt.pnr.lab.eng.rdu2.redhat.com bluechi-controller[9979]: Node 'bluechi-tester-1' disconnected
Jul 03 21:59:16 ibm-p8-kvm-03-guest-02.virt.pnr.lab.eng.rdu2.redhat.com bluechi-controller[9979]: Node 'bluechi-tester-2' disconnected

good job, thanks so much.