Implement Unit Test For Actual Faraday Radio

kb1lqc commented 6 years ago

We need a test which is normally skipped, but could be turned on by command line or similar means. This will help work with the radio as we develop the firmware.

Pytest "skipping" might work well: https://pytest.readthedocs.io/en/reorganize-docs/new-docs/user/skipping.html

Needs

During a unit test, connect to an actual serial port (let's use /dev/ttyUSB0 for now)
Send a packet such as a ping or just some string value
Consider having the test send a long duration of repetative packets (i.e. one ever seconds for 60 seconds)

lqdev commented 6 years ago

Any specific modules in mind for connecting to the serial port? If not, this one looks pretty good

pyserial

kb1lqc commented 6 years ago

Yes we already use pyserial for the current unit tests. To negate the need for hardware the pyserial tests use the loopback URL. Check out the serialtestclass in the tests folder!

What this request is doing is marking a spot for a use of pyserial unit test that can be enabled only when we know there is hardware connected to a specific port.

---- On Mon, 05 Feb 2018 03:54:26 -0800 notifications@github.com wrote ----

Any specific modules in mind for connecting to the serial port? If not, this one looks pretty good

pyserial

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

lqdev commented 6 years ago

Gotcha

lqdev commented 6 years ago

So here's a start. This is all in one file. Close to intended results or completely off mark?

from serial.tools.list_ports import grep
import pytest

def increment(x):
    '''
    Function to be tested. Increments x by 1
    x: int - any integer
    '''
    return x + 1

def is_port_available(port='/dev/ttyUSB0'):
  '''
  Checks whether specified port is available.
  port:string -  i.e. 'COM1'. Default is /dev/ttyUSB0
  '''
  is_port_available = grep(port)

  try:
    next(is_port_available)
    available = True
  except:
    available = False

  return available 

@pytest.mark.skipif(is_port_available,reason="hardware connected")
def test_skip_inc():
    '''
    Function to be skipped.
    '''
    assert increment(3) == 5

def test_inc():
    '''
    Evaluated function
    '''
    assert increment(3) == 4

The output is the following.

C:\Temp\LocalDev\Python\farada>python -m pytest test_simple2.py
============================= test session starts =============================
platform win32 -- Python 3.4.3, pytest-3.3.2, py-1.5.2, pluggy-0.6.0
rootdir: C:\Temp\LocalDev\Python\farada, inifile:
collected 2 items

test_simple2.py s.                                                       [100%]

===================== 1 passed, 1 skipped in 0.06 seconds =====================

Things that still need to be worked out:

[ ] Command line input/parametrization for is_port_available function. Samples

kb1lqc commented 6 years ago

Thanks @lqdev yes other than the increment this could be useful. If the port is directly checked for then no command line options necessary. It would take this basic idea and include the serial/TUN checks performed with loopback in the other tests. However we would need to know what the hardware would do (loopback, or just normal radio transmissions).

There's probably a test that can be written into the firmware to help with this. For now just moving the basic versions of the serial/TUN test into a non-loopback format over the serial connection should be helpful, doesn't even have to assert any checks at the moment.

kb1lqc commented 6 years ago

@lqdev I've implemented a version of your suggested code! Instead of limiting it to test code I see this USB detection code useful enough that I implemented it in the faradayio module. This is boilerplate test code but I am able to successfully skip the test if a Faraday (or any USB serial device) is not attached to /dev/ttyUSB0!

Connected

(.venv) bryce@bryce-ubuntu:~/Documents/git/faradayio$ sudo .venv/bin/pytest -vk tunHardwareSendLoop
=============================== test session starts ================================
platform linux -- Python 3.5.2, pytest-3.3.2, py-1.5.2, pluggy-0.6.0 -- /home/bryce/Documents/git/faradayio/.venv/bin/python3
cachedir: .cache
rootdir: /home/bryce/Documents/git/faradayio, inifile:
plugins: cov-2.5.1
collected 38 items                                                                 

tests/test_tun.py::test_tunHardwareSendLoop PASSED                           [100%]

=============================== 37 tests deselected ================================
===================== 1 passed, 37 deselected in 0.35 seconds ======================

Unconnected

(.venv) bryce@bryce-ubuntu:~/Documents/git/faradayio$ sudo .venv/bin/pytest -vk tunHardwareSendLoop
=============================== test session starts ================================
platform linux -- Python 3.5.2, pytest-3.3.2, py-1.5.2, pluggy-0.6.0 -- /home/bryce/Documents/git/faradayio/.venv/bin/python3
cachedir: .cache
rootdir: /home/bryce/Documents/git/faradayio, inifile:
plugins: cov-2.5.1
collected 38 items                                                                 

tests/test_tun.py::test_tunHardwareSendLoop SKIPPED                          [100%]

=============================== 37 tests deselected ================================
===================== 1 skipped, 37 deselected in 0.34 seconds =====================

See my two commits above for the code that implemented these tests.

FaradayRF / faradayio