Query for all qm components on a GC

[sergiomarsen]

Hello!

We are trying to get all testcases from a qm project. This project is configuration management enabled and has more than one components. All this components are contributing to a GC.

We are trying to query to this qm project with the GC project and GC stream, and the query works but only extracts the testcases from the default component.

Do you know if is there a way to extract test cases from all the components or any workaround to deal with this?

Thanks a lot in advance!

[barny]

Running oslcquery on a QM project with three components all contributing to a GC, specifying a GC project and configuration, I get results for test cases from all three components.

My commandline is

oslcquery -A qm,gc -G "SGC Production Stream" -O testcases.csv -p "SGC Quality Management" -E "SGC GC" -s *

Where:

• -A specifies the context roots for qm and gc
• -G specifies the GC stream name
• -O specifies the csv file name
• -p specifies the QM project name
• -E specifies the GC project name
• -s * selects all properties to be returned

This gets 36 results - one component has 30 test cases, the second has 3 and the third component also has 3.

So AFAICT oslcquery does what you're looking for, at least with 7.0.2 SR1 iFix025

What version+ifix are you using,

What's your commandline?

[barny]

Similarly querying for test plans by adding option -r TestPlanQuery I get 4+1+1 test plans from the three components

[barny]

Are you on the most recent version of elmclient? oslcquery shows the version when it runs, should be 0.26.2 I think.

[barny]

Here's a gotcha that might have got you - by default oslcquery caches responsed from the server for 7 days for everything except the query itself, which can cause problems when important things change - if you put option -W at the end of your commandline that will clear the cache at startup so everything will be retrieved at least once, which can work around important things having changed. Or use -WW to disable caching completely.

[sergiomarsen]

Hello, Thanks for the interest @barny , sorry didn't see the responses until now.

I confirmed I was using a bit old version from elm client, so upgraded it and followed your recommendation using -WW

However I can see this 2 issues:

1. While among all modules have a total of 1828 test cases (during the query seems to read them somehow) it finally only retrieves 540 entries. See pic below:

(Also tried using --pagesize 0 but exactly same result,

2. I have detected that the $uri attribute extracted only redirects to the qm test case for the default component, for the other extracted test cases the uri displays this error: AQXCM5008E The operation cannot be completed because the resource, https://****/qm/oslc_qm/contexts/_LRxjELO9Ee6SC_2Ft2FkkQ/resources/com.ibm.rqm.planning.VersionedTestCase/_Hs5UA8UJEe6uXKS9QbuYYQ, was not found.

Actually, what more disturb me now is the second point about the uris, as this is also happening also quering component by component (despite for default one). Have you ever experienced a similar issue? Thanks in advance!

[barny]

Your (2) is straightforward - the URI needs to be used with the configuration URL added to specify the test case fully, otherwise if you don't provide a configuration the uri is searched for in the default configuration in the default component and not found because it's not in that config. Same for RM artifact uris. The default config is the 'initial stream' of the original component of the project. Be aware that the default configuration can be archived so don't rely on URIs without configuration always working. You could modify the code that saves the result to add a column with the full uri-in-configuration like: URI?oslc_config.context=CONFIGURI

For (1) there was a defect in DN which had a similar effect of varying numbers of results but only when paging was in used, that's why by default paging is now disabled just like --pagesize 0. I don't have large ETM data to try reproduce this. If you're using -s * then try being more selective, does that change things? If you can get to a single query which shows varying numbers of results then you could create a support case? But get it to a query where you coan use the URL in a browser rest client (with headers) and see varying numbers of results, so you eliminate any possibility of elmclient being part of the problem.

[ukuko]

oslcquery -A qm,gc -G "SGC Production Stream" -O testcases.csv -p "SGC Quality Management" -E "SGC GC" -s *

hi, would you please provide an example to get Testcases just for specific component? I know we need to use local configuration, but after reading documentation I dont find the proper combination of flags. this thread provided the most helpful information so far.

If you wish, I can afterwards contribute to the project putting such examples to the README.

Thanks in advance!

[barny]

For an ETM project "SGC Quality Management", in component "SGC MTM" for (local) configuration "SGC MTM Production stream" this will get all the testcases:

oslcquery -A qm -p "SGC Quality Management" -C "SGC MTM" -F "SGC MTM Production stream" -O testcases.csv -s *

[ukuko]

somehow your suggestion above didnt work for me and I get the usual error: (I have to add that, as in this thread, we need to query with the GC project and GC stream)

requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://abcd.com/qm/oslc_config/resources/com.ibm.team.vvc.Component

[barny]

Are you using the most recent elmclient version 0.26? Which version of Python, is it from python.org? What version+ifix of ELM are you using?

I've found a defect in ETM 7.0.2, resolved in 7.0.3, which causes this sort of error when you aren't a member of the project - can you check if you're a member? If this is the problem then until you can upgrade to 7.0.3 the only really practicable workarounds are to be made a member of the project, or to get JazzAdmin privilege.

we need to query with the GC project and GC stream

Why? If you query in a GC context oslcquery will give you results from that GC context's contributions, i.e. not just one contribution/component.

[ukuko]

we are using: client: python: 3.10.12 (running in WSL) elmclient: 0.26.2

we run the query with user which is a member of the project and has following roles: Tool Administrator, Configuration Lead, Test Engineer, Test Lead, Tester

ETM: 7.0.2 SR1 -> is this then the issue?

[barny]

ETM: 7.0.2 SR1 -> is this then the issue?

If you're a project member then that defect shouldn't be the problem. Sorry I don't have any idea what else might be going wrong.

[sergiomarsen]

# https://github.com/IBM/ELM-Python-Client/issues/60#issuecomment-1991851739

For (1) there was a defect in DN which had a similar effect of varying numbers of results but only when paging was in used, that's why by default paging is now disabled just like --pagesize 0. I don't have large ETM data to try reproduce this. If you're using -s * then try being more selective, does that change things? If you can get to a single query which shows varying numbers of results then you could create a support case? But get it to a query where you coan use the URL in a browser rest client (with headers) and see varying numbers of results, so you eliminate any possibility of elmclient being part of the problem.

Hello, back to this topic, now we are experiencing this issue within project with only one component too. So it seems it was affecting large queries in general (not specifically for multiple components as it was my thought), regarding your suggestion on changing on -s selecting * or some specific attributes, affects specially in timing but not in succesfull results size.

In this case, extracts 432 entries out of 683 expected. Also using--pagesize 0 says query completed in 2 pages. Regarding the get it to a query where you coan use the URL in a browser rest client (with headers) you suggested, is there a way to extract somehow this query to test this directly as you say?

Thanks a lot for the support

[barny]

if you use the -V option for OSLC Query it will show the Full Query URL with local configuration (also without configuration) - use this in a REST client, you'll need to provide headers OSLC-Core-Version: 2.0 and Accept: application/rdf+xml. It's possible that ETM always pages the results, but you could try adding &oslc.paging=false to the URL to try suppress paging, or add &oslc.paging=true&oslc.pagesize=1000 as you have less than 1000 results this might return a single page. If there's more than one page the first result will include a tag oslc:nextPage.

[barny]

FYI In 0.26.3 there will be an explicit oslc.paging=false if --pagesize is 0, which is the default.