SmartBear / testengine-cli

Node based command line interface for ReadyAPI TestEngine.
Apache License 2.0
6 stars 3 forks source link

Does not support composite projects with relative paths to files #84

Open gpaciga opened 1 year ago

gpaciga commented 1 year ago

When using composite projects, testengine-cli is unable to package the project files properly. When trying to resolve the paths to files to package into a zip file, testengine-cli will produce the following error:

TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string. Received undefined

It seems to be because when parsing the composite project files, testengine-cli does not set resourceRoot attribute, which is necessary for determining the path to any files.

This is in contrast to non-composite projects which do set resourceRoot and are able to locate and package external files correctly.

Smartbear advises exporting a .zip file manually from ReadyAPI to work around this, but this is impractical in CI environments where each commit expects tests to be run, and a committed zip file can easily become out of sync with the actual tests and interferes with normal git workflows.

Instead, we are working around this by adding the following patch to bin/readyapi_project.js, in the parseComposite method, to use the same logic the parse method uses to find the resourceRoot of the project:

diff --git a/bin/readyapi_project.js b/bin/readyapi_project.js
index 7af487f..0cbe5e1 100644
--- a/bin/readyapi_project.js
+++ b/bin/readyapi_project.js
@@ -71,6 +71,12 @@ module.exports.parseComposite = function (pathname) {
     jsonProject['con:soapui-project']['con:testSuite'] = getCompositeTestSuites(pathname, jsonProject);
     result = postProcessStructure(jsonProject);
     result['name'] = jsonProject['con:soapui-project']['attr']['@_name'];
+
+    let resourceRoot = path.dirname(pathname);
+    if ( ('@_resourceRoot' in jsonProject['con:soapui-project']['attr']) && (jsonProject['con:soapui-project']['attr']['@_resourceRoot'].length > 0))
+        resourceRoot = jsonProject['con:soapui-project']['attr']['@_resourceRoot'];
+    result['resourceRoot'] = resourceRoot;
+
     result['projectFiles'] = jsonProject['projectFiles'];
     return result
 };
gpaciga commented 11 months ago

I notice there is an inconsistency in how resourceRoot is handled between TestEngine and ReadyAPI for the above patch I proposed. Since I copied that handling from the non-composite project code, the issue presumably exists there as well.

Given a project with its root settings file at /path/to/readyapi/project/settings.xml, ReadyAPI defaults resourceRoot to /path/to/readyapi but path.dirname(pathname) in the TestEngine CLI just resolves to the current working directory, wherever that is (since pathname is just ".", not an absolute path). Assuming you run testengine-cli in the same folder as your settings.xml file, it would effectively be /path/to/readyapi/project/. This means that e.g. file paths in a Data Source step will point to two different locations depending on whether or not you run the project with TestEngine CLI or with ReadyAPI.

I'm changing the fork we use to let resourceRoot = path.dirname(path.resolve(pathname));.

gpaciga commented 11 months ago

I've abandoned trying to get a fork of testengine-cli to work, and am instead using an external script to create a zip file and pass that to testengine-cli. Took some trial and error to figure out the right structure for the zip since it is undocumented, but for anybody else facing this: Data Source files need to be co-located with the root settings.xml file, regardless of the path used for them in <file> property.