Rahix / tbot

Automation/Testing tool for Embedded Linux Development
https://tbot.tools
GNU General Public License v3.0
84 stars 21 forks source link

Recommended way to handle multiple boards #118

Open jacus opened 2 months ago

jacus commented 2 months ago

My Device Under Test is defined as below:

    MyBoard(connector.ConsoleConnector, board.PowerControler, board.Board)
        - power control + establish picocom channel
    MyLinux(board.Connector, board.LinuxBootLogin, Bash)
        - linux commands via picocom channel
    MyLinuxSsh(connector.SSHConnector, Bash)
        - linux commands via SSH

    def register_machines(ctx):
        ctx.register(custom.mach.MyBoard, tbot.role.Board)
        ctx.register(custom.mach.MyLinux, tbot.role.BoardLinux)
        ctx.register(custom.mach.MyLinuxSsh, custom.role.BoardLinuxSSH)

I'd like to expand my framwork to be able to handle multiple Devices Under Test in one test case. Each instance of a device will have own set of parameters (IP address, /dev/ttyS, etc). I have a question on how to implement such scenario.

I'd like to use it like that:

    with tbot.ctx.request(tbot.role.BoardLinux) as linux_1, \
          tbot.ctx.request(custom.role.BoardLinux2) as linux_2:
        linux_2.exec('ping', '-c3', linux_1.ip_address)

Tbot documentation says that it's possible to have multiple roles for same machine, but in such case how do I know from MyLinux machine which role requested it? I need to know that to determine individual parameters for an instance (IP address, /dev/ttyS, etc). Could you please provide recommended way to handle such scenario?

Rahix commented 2 months ago

Hi,

this is very well supported but unfortunately not documented well. It works like this:

First, you define custom roles for any machines you'd like to request from testcases. These roles subclass the roles from tbot. I usually do this in a config/roles.py file:

# config/roles.py
import tbot
from tbot.machine import linux

class Board1(tbot.role.Board):
    pass

class Board2(tbot.role.Board):
    pass

class Board1Linux(tbot.role.BoardLinux):
    pass

class Board2Linux(tbot.role.BoardLinux):
    pass

class Board1LinuxSsh(linux.LinuxShell, tbot.role.Role):
    pass

Then, each hardware setup config defines the hardware specific roles:

# config/rahix_lab.py
import config.roles

class RahixBoard1(...):
    ...

class RahixBoard2(...):
    ...

class RahixLabhost(...):
    ...

def register_machines(ctx):
    ctx.register(RahixLabhost, tbot.role.LabHost)
    ctx.register(RahixBoard1, config.roles.Board1)
    ctx.register(RahixBoard2, config.roles.Board2)

Continuing on, the hardware independent roles get defined in a "common" config module:

# config/common.py
import config.roles

class Board1Linux(...):
    ...

class Board2Linux(...):
    ...

class Board1LinuxSsh(...):
    ...

def register_machines(ctx):
    ctx.register(Board1Linux, config.roles.Board1Linux)
    ctx.register(Board2Linux, config.roles.Board2Linux)
    ctx.register(Board1LinuxSsh, config.roles.Board1LinuxSsh)

At this point, you can write testcases referencing any of the custom roles to interact with both boards. The tbot default roles cannot be used, because the do not get defined anywhere. tbot is then invoked using

newbot -c config.rahix_lab -c config.common

Now this may be the solution for certain setups, but in a lot of cases you then also want some testcases to work with either board. This can be achieved with an additional trick:

# config/board1.py
import tbot
import config.common
import config.roles

def register_machines(ctx):
    config.common.register_machines(ctx)

    board1 = ctx.get_machine_class(config.roles.Board1)
    ctx.register(board1, tbot.role.Board)

    board1_linux = ctx.get_machine_class(config.roles.Board1Linux)
    ctx.register(board1_linux, tbot.role.BoardLinux)

Now you can write testcases that target the tbot default roles and use the board-specific config (-c config.rahix_lab -c config.board1) to select which board it runs on.

Hope this helps... If you have questions, let me know.