codice / ddf

DDF Distributed Data Framework - an open source, modular integration framework.
http://ddf.codice.org
Other
132 stars 181 forks source link

[2.26.x] Ensure SortedQueryMonitor uses query ThreadContext #6784

Closed derekwilhelm closed 5 months ago

derekwilhelm commented 6 months ago

What does this PR do?

When performing a query, the QueryOperations hands off the source execution of the queries to the SortedFederationStrategy. This class uses an ExecutorService to run the source queries in parallel within a thread pool where the results are gathered and processed with PostFederatedQueryPlugins in the SortedQueryMonitor class. Since these run in separate threads, they don't share the same thread context as the original query. This means that things like trace-id in the thread context are lost. Furthermore, if any of the PostFederatedQueryPlugins perform an action that sets or binds a user subject to the thread, then the thread in the thread pool will now have that user defined in its thread context going forward, including when it gets reused. This may easily happen if a PostFederatedQueryPlugin issues a new QueryRequest to the CatalogFramework. When multiple users are using the system, this has the effect of the source queries and PostFederatedQueryPlugins running in a thread with a different user than what issued the original request.

This PR fixes this issue by retrieving the original ThreadContext resources from the original query thread and passing them to the SortedQueryMonitor instance that is submitted to the ExecutorService. When the SortedQueryMonitor runs, it clears the thread context of the executor thread and resets it to original thread context resources from the original query thread. This maintains the user subject across the threads along with any other thread context variables that may be important.

Who is reviewing it?

@jlcsmith @pklinef

Select relevant component teams:

@codice/core-apis @codice/security

Ask 2 committers to review/merge the PR and tag them here.

@andrewkfiedler @glenhein

How should this be tested?

This is rather difficult to test with base DDF. You need to have a PostFederatedQueryPlugin that issues a query to the catalog framework to force it to modify the thread context. Then you can verify that the searches are executing as expected when querying against data that should be filtered out for some users but not others.

Alternatively, you can connect the debugger and set a break point in the SortedQueryMonitor class.
log in as two different users Create two different queries, one for each user (e.g. anyText like FOO and anyText like BAR) Submit the queries for the two different users When the break point hits in the SortedQueryMonitor, verify that the thread context while running contains the correct user for the query that matches the user submitting the original query Execute the query multiple times and verify the user subject in the SortedQueryMonitor's thread context is set appropriately each time

Any background context you want to provide?

What are the relevant tickets?

Fixes: #____

Screenshots

Checklist:

Notes on Review Process

Please see Notes on Review Process for further guidance on requirements for merging and abbreviated reviews.

Review Comment Legend:

jlcsmith commented 6 months ago

build now

cxddfbot commented 6 months ago

Internal build has been started, your results will be available at build completion.

cxddfbot commented 6 months ago

Build FAILURE See the job results in legacy Jenkins UI or in Blue Ocean UI.

derekwilhelm commented 6 months ago

build now

cxddfbot commented 6 months ago

Internal build has been started, your results will be available at build completion.

cxddfbot commented 6 months ago

Build FAILURE See the job results in legacy Jenkins UI or in Blue Ocean UI.

LinkMJB commented 6 months ago

build now

cxddfbot commented 6 months ago

Internal build has been started, your results will be available at build completion.

LinkMJB commented 6 months ago

build now

cxddfbot commented 6 months ago

Internal build has been started, your results will be available at build completion.

cxddfbot commented 6 months ago

Build SUCCESS See the job results in legacy Jenkins UI or in Blue Ocean UI.

jlcsmith commented 5 months ago

Hero will happen in a downstream project