Testing Qt Code which requires EventHandling and UI

[kreuzberger]

🤔 What's the problem you're trying to solve?

Testing source code which requires Qt EventHandling (e.g. due to Network, Timer) and / or UI requires to haven an Q(Core)Applikation running and be executed in Thread or in the Main Thread depending on how to use the eventhandling or the ui. This seems not to be possible with the current implementation, where the cucumper-cpp test processing is done in the main loop.

✨ What's your proposed solution?

Fortunately there is a nomain library of cucumber-cpp which an alternativ implementation could use. Unfortunatly there is no code available to execute the Stream Reader in a non-blocking or event handling way. Maybe examples or basic implementations for the supported Test frameworks should be provided.

⛏ Have you considered any alternatives or workarounds?

Workaround is done by:

• linking against the nomain library
• Implement a Qt EventHandling friendly WireProtocol Handler
• this requires access to classes in "internal" namespace

📚 Any additional context?

The AsyncStreamReader original Code used in the attached files comes from https://gist.github.com/vmrob/38debbb80b93df6da4fa

Questions?

• Is this a valid use-case in the usage of cucumber-cpp?
• Is the usage of the nomain library intended?
• Are the required classes CukeEngineImpl, JsonWireMessageCodec, WireProtocolHandler, SocketServer intended and maintained as API classes?

[kreuzberger]

QtWireProtocolHandler.zip

Idea / Solution is to implement and use a Qt based WireProtocolServer

in the main of the test applikation:

  // initialize qt base wire protocol server
  QtWireProtocolHandler wire_protocol_handler( listenHost, port, parser.isSet( "verbose" ) );
  // start processung
  wire_protocol_handler.process();
  // run qt eventloop in main thread
  return app.exec();

Implementations (attached as zip)

class QtWireProtocolHandler : public QObject
{
  Q_OBJECT
public:
  explicit QtWireProtocolHandler( const std::string& host, int port, bool verbose );
  ~QtWireProtocolHandler() override;
  void process();
protected slots:
  void onClose();

private:
  std::unique_ptr<CukeEngineImpl>       cukeEngine      = nullptr;
  std::unique_ptr<JsonWireMessageCodec> wireCodec       = nullptr;
  std::unique_ptr<WireProtocolHandler>  protocolHandler = nullptr;
  std::unique_ptr<SocketServer> server = nullptr;
  bool                                  verbose         = false;
};

Which uses a custom Implementation of SocketServer

class QtTCPSocketServer : public QObject, public SocketServer
{
  Q_OBJECT
public:
  typedef unsigned short port_type;
  explicit QtTCPSocketServer( const ProtocolHandler* protocolHandler );

  void listen( const port_type port );
 void listen( const asio::ip::tcp::endpoint endpoint );

  asio::ip::tcp::endpoint listenEndpoint() const;

  void acceptOnce() override;

signals:
  void closed();

private:
  void doAcceptOnce( asio::basic_socket_acceptor<asio::ip::tcp>& acceptor );
  void timerEvent( QTimerEvent* ) override;

  asio::ip::tcp::acceptor            acceptor;
  std::unique_ptr<AsyncStreamReader> mAsyncStreamReader = nullptr;
  asio::ip::tcp::iostream            mStream;
};

[ursfassler]

@kreuzberger I strongly recommend to not include the Qt classes nor UI in the acceptance tests. And not timer, you don't want to make your tests slower than needed. Only include and test the business logic of the application. When doing so, you probably don't need the event handling. This removes asynchronous (non-deterministic) behavior, making the tests stable.

I had cases where we had queues (in Qt this would be queued connections) in the code under test. The solution was to dispatch the stuff in the queue until empty after every step. I think it was with QEventLoop::processEvents within AFTER_STEP hook. Or something like here.

[kreuzberger]

I agree with most of your doubts. Testing with no such constraints make live easier. Main advantage of the gherkin language is for me still to test user scenarios (not the gui) but if the app under the test is a rich desktop qt app and runs in a multi process enviroment i have to face it. Or use Squish. But testing the logic with squish is much more slower and complicated i think. But this is currently my intention, test the app instead of squish/gherkin with a different (and maybe better) approach.

But back to the questions:

• The existence of a nomain library is for me the hint that this library could be used.
• The names and used classes have an "internal" namespace, but if used as static library i can access them without any problems. The CUCUMBER_CPP_EXPORT is a hint for me that the classes could be used as an API even if builded as shared libraries?

[kreuzberger]

As currently a nomain library is provided and the wire protocol server could be replaced with the above implementation, the current work could be done without provided the code as "official" part. This requires the nomain library to be available in further releases, which might be intended. As currently there are no plans to change this, the issues could get closed cause solution could be implemented without requiring changes in the project.