As a tester/test I want to change the mock behaviour for a project

About situation

When other systems integrate sechub into their live cylce (this means not a build server integration, but an integration from another product...) they also want to integrate into their integration tests as well - so a special environment is necessary, which we call inside this document as INT environment.

When using commercial security products it can happen that integration tests will lead to additional license costs. To prevent this the INT environment can be setup to use mocked adapters. These adapters will not communicate with the real security products but instead return mocked product results. All logic, every behaviour inside SecHub is exactly the same except the communication with the security product. This technique is used by SecHub for integration testing itself.

At the moment the mock behaviour is only configured statically inside mockdata_setup.json and uses special targets to get green, yellow or red mocked product results . Also this behaviour was designed for internal sechub integration tests only, which is inconvenient for other systems because integration tests inside sechub could change and sechub configurations must be special prepared etc.

Tests/Testers wants to have possiblity to setup wanted result type (green,yellow,red) on their sechub projects by REST API. Green will contain only green results, yellow shall containg green and yellow results and red will contain green, yellow and red ones.

Solution

Related issue

We need #140 to be implemented to store project relevant data. Also we need to

INT environment setup

Integration tests from other systems require an INT environment where sechub is started with mocked_products but not integrationtest (in integration test profile there are some special anonymous access possiblities suitable for testing/temporary instances but not okay for a running stage. E.g email system is mocked and communiction can be fetched by everybody via REST ...)

REST interface

We provide a rest interface with URL pattern https://$serverName/project/$projectId/mockdata

and support PUT (containing JSON) GET (returning JSON)

The access to the project will be exactly like for any normal operation: User must have access to project (or must be an administrator). So even different systems can use this INT environment at same time.

JSON content

{ 
  "apiVersion" : "1.0",

   "codeScan" : {
         "result" : "green|yellow|red"   
   },
   "webScan" : {
         "result" : "green|yellow|red"   
   },
   "infraScan" : {
         "result" : "green|yellow|red"   
   }

}

Fallback

When not defined or data is an empty string, the defaults from mockdata_setup.json will be used! This makes it backward compatible to sechub integration tests. Calling systems are responsible to prepare their projects appropriate!

Future

It could be possible in another issue to provide special result data for testing - e.g. a code scan result shall contain a special location wanted by tester. If this wanted be aware about following: NEVER give the possibility to define product results inside JSON content! Reason: the main reason for sechub is the possiblity to change used products without affecting users or their setup. This is the same for INT testing. If sechub becomes modular, replaces default products etc. the integration tests must be still working! So we may not have any foreign product dependency here !

mercedes-benz / sechub

Usecase: A tester/test wants to change mock behaviour of a dedicated project #141