At the moment the robot-api tests depend on robotd, though don't really do much with the actual robotd boards. Instead they mainly use the BoardRunner as a mechanism to listen for the data sent over the socket (via injecting a message queue which it targets).
This approach appears simple on the surface however involves multiple processes and timeouts in the actual tests, meaning that there's a lot more complexity happening than appears at first.
While this approach also appears useful as integration level tests, as the underlying robotdBoards aren't used, there's nothing which actually validates that the data sent over the socket is going to work. Again the use of some of the robotd classes here creates the impression that more is being tested than is really the case.
Exploration
In #55 I've explored an alternative approach which avoids using the BoardRunner and instead mocks the socket on the robot-api side, connecting it directly to the realServoAssembly device. We then mock out the serial connection from that robotdBoard to the underlying arduino and primarily assert against the commands sent there.
This has its own downsides, notably that:
we're not testing a real socket
we're still mocking the request/response somewhere (albeit now a place which would be much harder to have a real endpoint for)
we're reliant on poking the internals of robotd in order to test robot-api
Suggestion
If we moved to a point where the majority of the transport layer (currently unix sockets) is swappable for an alternative backend, at least in code. If we don't actually implement a different backend for real use, I think that's fine; the aim here is to get better layering and thus testability.
Some of this is already present on the robotd side, with the extraction of the json wrapped socket that handles the data transport. I think robot-api would benefit from a similar approach.
Once we've extracted a mirror json wrapped socket which also handles reconnection (which I think is worth doing anyway), we can then override that connection much more easily in tests. A couple of ways that could be done are:
Dependency injection into the Boards -- have them take a connection rather than a _socketpath. This probably means that Robot sets up the connection objects, even if it doesn't actually cause them to connect. It could complicate the discovery and Board.serial aspects though.
Introduce a "backends" pattern such that the connection-type is selected (based on some config) at the point of constructing the socket. This is just as testable, though keeps us slightly more coupled to the idea of paths as the discovery mechanism and so may be a smaller change.
To support this for testing I suggest we would:
in robotd introduce a test harness which takes a connection-like object and behaves like the BoardRunner; we'd also be able to use that for testing the Board classes there
we might even want to look at separating some of BoardRunner's connection-nature from its process-nature to aid with this
in the robot-api tests, we could then:
inject an arbitrary connection into our Board api classes and have simpler testing behaviours
test the communication transport separately in a small number of tests, limiting the impact of needing real sockets, multiprocessing, etc. to only those tests which are actually testing that
Background
At the moment the
robot-apitests depend onrobotd, though don't really do much with the actualrobotdboards. Instead they mainly use theBoardRunneras a mechanism to listen for the data sent over the socket (via injecting a message queue which it targets).This approach appears simple on the surface however involves multiple processes and timeouts in the actual tests, meaning that there's a lot more complexity happening than appears at first.
While this approach also appears useful as integration level tests, as the underlying
robotdBoards aren't used, there's nothing which actually validates that the data sent over the socket is going to work. Again the use of some of therobotdclasses here creates the impression that more is being tested than is really the case.Exploration
In #55 I've explored an alternative approach which avoids using the
BoardRunnerand instead mocks the socket on therobot-apiside, connecting it directly to the realServoAssemblydevice. We then mock out the serial connection from thatrobotdBoardto the underlying arduino and primarily assert against the commands sent there.This has its own downsides, notably that:
robotdin order to testrobot-apiSuggestion
If we moved to a point where the majority of the transport layer (currently unix sockets) is swappable for an alternative backend, at least in code. If we don't actually implement a different backend for real use, I think that's fine; the aim here is to get better layering and thus testability.
Some of this is already present on the
robotdside, with the extraction of the json wrapped socket that handles the data transport. I thinkrobot-apiwould benefit from a similar approach.Once we've extracted a mirror json wrapped socket which also handles reconnection (which I think is worth doing anyway), we can then override that connection much more easily in tests. A couple of ways that could be done are:
Boards -- have them take a connection rather than a _socketpath. This probably means thatRobotsets up the connection objects, even if it doesn't actually cause them to connect. It could complicate the discovery andBoard.serialaspects though.To support this for testing I suggest we would:
robotdintroduce a test harness which takes a connection-like object and behaves like theBoardRunner; we'd also be able to use that for testing theBoardclasses thereBoardRunner's connection-nature from its process-nature to aid with thisrobot-apitests, we could then:Boardapi classes and have simpler testing behaviours