search

qca / open-plc-utils

Qualcomm Atheros Open Powerline Toolkit.
Other
352 stars 154 forks source link

Testability of open-plc-utils #34

Open ch-aurich opened 10 years ago

ch-aurich commented 10 years ago

Hi Folks,

I would like to discuss how we can improve stability of open-plc-utils. Since I2SE contributes to open-plc-utils we have seen quite some of changes that actually introduced malfunction of various tools which I do not want to discuss here...

The point is: when we change something we have no chance of testing the tools agains various chips / chipsets. This means that every feature we introduce to existing tools might break the function for other hardware then we actually develop for. The same applies for all other people implementing new features. Nobody is willing to try out the tools with all possible hardware setups.

The perfect solution for that would be an automated test lab - unless Qualcomm Atheros is willing to provide such a setup, I do not believe in that idea coming true.

Another solution would be to simulate the hardware counterparts... so that every packet gets the same response from some software as it gets from the Qualcomm Chips in real life. Again: this seems to be quite a lot work that QCA needs to do.

What I suggest is to start simple. We could use services like travis-ci for automated testing of verious file based tools like modpib, psout, psin and so on. Maybe we can add more stuff later.

If you agree with that suggestion I would like to start implementing some first test cases for QCA7000 PIB Files. This would mean that I upload some reference PIB Files to this repository. These could be used to perform modifications and the compare them to other PIB files that are also stored in the repository.

Kind regards, Christian

ffainelli commented 10 years ago

I suppose that there could be some PIB file redistribution issues involved. You could however try to download some of the firmware updates from e.g: Devolo and extract their PIB; those should have a proper license.

A while ago, I started writing an INT6300 emulator in order for me to test for regressions in the Wireshark packet-homeplug-av.c dissector, I could probably resurrect that if it is of any interest. The plan was also to allow for multiple stations to be spawned (multiple instances of the simulator) such you could also simulate a network and its properties.

razzlefratz commented 9 years ago

This is a great topic that is starting to get some interest. I can't address it now but lets not lose it.

ffainelli commented 9 years ago

I ended up pushing a very simplistic simulator program here: https://github.com/ffainelli/faifa/blob/master/simulator.c

we could extend it to support more commands than "get software version" of course ;)

razzlefratz commented 9 years ago

Thanks for this, Florian. We need a PLC zealot to finish this :-)