aws-greengrass / aws-greengrass-shadow-manager

A GreengrassV2 Component that provides offline device shadow documents and optional synchronization to the IoT device shadow service.
Apache License 2.0
9 stars 6 forks source link

fix: only read sync from sync thread #171

Closed jcosentino11 closed 1 year ago

jcosentino11 commented 1 year ago

Issue #, if available:

Description of changes:

:warning: This change undoes the (unreleased) fix in https://github.com/aws-greengrass/aws-greengrass-shadow-manager/pull/166, which introduces a bug where the IPC thread could potentially be blocked, and takes a different approach by restricting sync info to being read by only one thread.

This change adjusts where SyncRequest#isUpdateNecessary is called, so that it is only called when a SyncRequest is executing on a sync thread. This is different than previous behavior, where an IPC or MQTT CRT thread could call this method, to read shadow sync information from the local database to make a decision on whether or not the SyncRequest should be put on the queue. This ensures that shadow sync information is only read and written to the local database from a single thread.

Since "unnecessary" requests aren't being filtered before being put on the RequestBlockingQueue anymore, there needs to be a filtering mechanism when the SyncRequest executes. Otherwise, the behavior fixed in https://github.com/aws-greengrass/aws-greengrass-shadow-manager/pull/106 will regress.

This behavior already works for requests that are not merged into a full sync by RequestMerger, because SyncRequests already check isUpdateNecessary at the beginning. To make it work for full syncs, we need to maintain a "history" of all requests that were merged together for a particular full sync request. Then, when the full sync executes, it can look through these "sub-requests", filter out the unnecessary ones, merge them all together, and execute the result. All of this behavior is wrapped in a new MergedFullShadowSyncRequest class.

The rest of the changes in the PR are cleanup from reverting https://github.com/aws-greengrass/aws-greengrass-shadow-manager/pull/166:

along with a bunch of flaky test fixes.

Why is this change necessary: To address a customer issue where cloud updates were being dropped due to a race condition when reading outdated sync information from the IPC thread. (more details of the original issue in https://github.com/aws-greengrass/aws-greengrass-shadow-manager/pull/166)

How was this change tested: Integration tests

Any additional information or context required to review the change:

Checklist:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

github-actions[bot] commented 1 year ago

Unit Tests Coverage Report

File Coverage Lines Branches
All files 84% 90% 79% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.model.MergedFullShadowSyncRequest 21% 26% 15% :x:
com.aws.greengrass.shadowmanager.sync.model.LocalUpdateSyncRequest 79% 85% 73% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.model.Direction 100% 100% 100% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.model.LocalDeleteSyncRequest 99% 98% 100% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.model.FullShadowSyncRequest 84% 97% 71% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.model.CloudDeleteSyncRequest 80% 78% 83% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.model.DirectionWrapper 100% 100% 0% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.model.OverwriteCloudShadowRequest 98% 95% 100% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.model.BaseSyncRequest 83% 93% 73% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.model.OverwriteLocalShadowRequest 98% 95% 100% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.model.CloudUpdateSyncRequest 74% 76% 71% :white_check_mark:
com.aws.greengrass.shadowmanager.ShadowManagerDatabase 0% 0% 0% :x:
com.aws.greengrass.shadowmanager.ShadowManager 75% 80% 71% :white_check_mark:
com.aws.greengrass.shadowmanager.ShadowManagerDAOImpl 100% 100% 100% :white_check_mark:
com.aws.greengrass.shadowmanager.AuthorizationHandlerWrapper 100% 100% 100% :white_check_mark:
com.aws.greengrass.shadowmanager.PubSubIntegrator 89% 93% 86% :white_check_mark:
com.aws.greengrass.shadowmanager.ShadowManager$1 43% 43% 0% :x:
software.amazon.awssdk.aws.greengrass.GeneratedAbstractDeleteThingShadowOperationHandler 67% 67% 0% :white_check_mark:
software.amazon.awssdk.aws.greengrass.GeneratedAbstractListNamedShadowsForThingOperationHandler 67% 67% 0% :white_check_mark:
software.amazon.awssdk.aws.greengrass.GeneratedAbstractUpdateThingShadowOperationHandler 67% 67% 0% :white_check_mark:
software.amazon.awssdk.aws.greengrass.GeneratedAbstractGetThingShadowOperationHandler 67% 67% 0% :white_check_mark:
com.aws.greengrass.shadowmanager.util.JsonMerger 94% 100% 89% :white_check_mark:
com.aws.greengrass.shadowmanager.util.SyncNodeMerger 91% 98% 84% :white_check_mark:
com.aws.greengrass.shadowmanager.util.DataOwner 100% 100% 0% :white_check_mark:
com.aws.greengrass.shadowmanager.util.ShadowWriteSynchronizeHelper 100% 100% 0% :white_check_mark:
com.aws.greengrass.shadowmanager.util.Validator 100% 100% 100% :white_check_mark:
com.aws.greengrass.shadowmanager.util.JsonUtil 94% 100% 88% :white_check_mark:
com.aws.greengrass.shadowmanager.ipc.BaseRequestHandler 77% 77% 0% :white_check_mark:
com.aws.greengrass.shadowmanager.ipc.GetThingShadowRequestHandler 100% 100% 100% :white_check_mark:
com.aws.greengrass.shadowmanager.ipc.IpcRateLimiter 100% 100% 100% :white_check_mark:
com.aws.greengrass.shadowmanager.ipc.PubSubClientWrapper 100% 100% 0% :white_check_mark:
com.aws.greengrass.shadowmanager.ipc.UpdateThingShadowIPCHandler 100% 100% 0% :white_check_mark:
com.aws.greengrass.shadowmanager.ipc.InboundRateLimiter 94% 89% 100% :white_check_mark:
com.aws.greengrass.shadowmanager.ipc.InboundRateLimiter$1 100% 100% 100% :white_check_mark:
com.aws.greengrass.shadowmanager.ipc.DeleteThingShadowRequestHandler 99% 99% 100% :white_check_mark:
com.aws.greengrass.shadowmanager.ipc.UpdateThingShadowRequestHandler 92% 95% 89% :white_check_mark:
com.aws.greengrass.shadowmanager.ipc.DeleteThingShadowIPCHandler 100% 100% 0% :white_check_mark:
com.aws.greengrass.shadowmanager.ipc.GetThingShadowIPCHandler 100% 100% 0% :white_check_mark:
com.aws.greengrass.shadowmanager.ipc.ListNamedShadowsForThingIPCHandler 100% 100% 100% :white_check_mark:
com.aws.greengrass.shadowmanager.ipc.model.Operation 100% 100% 0% :white_check_mark:
com.aws.greengrass.shadowmanager.ipc.model.PubSubRequest 100% 100% 0% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.strategy.PeriodicSyncStrategy 63% 77% 50% :x:
com.aws.greengrass.shadowmanager.sync.strategy.SyncStrategyFactory 100% 100% 100% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.strategy.BaseSyncStrategy 93% 97% 89% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.strategy.RealTimeSyncStrategy 93% 86% 100% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.IotDataPlaneClientFactory 55% 85% 25% :x:
com.aws.greengrass.shadowmanager.sync.RequestBlockingQueue 96% 99% 93% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.SyncHandler 71% 88% 53% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.IotDataPlaneClientWrapper 91% 91% 0% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.RequestMerger 79% 77% 81% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.CloudDataClient 74% 72% 77% :white_check_mark:
com.aws.greengrass.shadowmanager.model.configuration.ShadowSyncConfiguration 87% 93% 82% :white_check_mark:
com.aws.greengrass.shadowmanager.model.configuration.ThingShadowSyncConfiguration 76% 78% 75% :white_check_mark:
com.aws.greengrass.shadowmanager.configuration.ShadowDocSizeConfiguration 100% 100% 100% :white_check_mark:
com.aws.greengrass.shadowmanager.configuration.RateLimitsConfiguration 100% 100% 0% :white_check_mark:
com.aws.greengrass.shadowmanager.configuration.ComponentConfiguration 100% 100% 0% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.strategy.model.StrategyType 86% 92% 80% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.strategy.model.Strategy 90% 100% 80% :white_check_mark:
com.aws.greengrass.shadowmanager.model.ResponseMessageBuilder 100% 100% 0% :white_check_mark:
com.aws.greengrass.shadowmanager.model.ShadowDocument 81% 89% 73% :white_check_mark:
com.aws.greengrass.shadowmanager.model.ShadowStateMetadata 93% 98% 89% :white_check_mark:
com.aws.greengrass.shadowmanager.model.ErrorMessage 75% 100% 50% :white_check_mark:
com.aws.greengrass.shadowmanager.model.ShadowRequest 100% 100% 100% :white_check_mark:
com.aws.greengrass.shadowmanager.model.ShadowState 84% 87% 81% :white_check_mark:
com.aws.greengrass.shadowmanager.model.LogEvents 100% 100% 0% :white_check_mark:

Minimum allowed coverage is 65%

Generated by :monkey: cobertura-action against 7b737ffbc564802adb22a40facaf8d9acee88273

github-actions[bot] commented 1 year ago

Integration Tests Coverage Report

File Coverage Lines Branches
All files 72% 76% 68% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.model.MergedFullShadowSyncRequest 80% 85% 75% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.model.LocalUpdateSyncRequest 76% 83% 69% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.model.Direction 83% 91% 75% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.model.LocalDeleteSyncRequest 54% 58% 50% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.model.FullShadowSyncRequest 71% 79% 62% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.model.CloudDeleteSyncRequest 52% 54% 50% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.model.DirectionWrapper 100% 100% 0% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.model.OverwriteCloudShadowRequest 34% 43% 25% :x:
com.aws.greengrass.shadowmanager.sync.model.BaseSyncRequest 49% 57% 41% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.model.OverwriteLocalShadowRequest 29% 29% 0% :x:
com.aws.greengrass.shadowmanager.sync.model.CloudUpdateSyncRequest 62% 60% 64% :white_check_mark:
com.aws.greengrass.shadowmanager.ShadowManagerDatabase 100% 100% 100% :white_check_mark:
com.aws.greengrass.shadowmanager.ShadowManager 90% 92% 88% :white_check_mark:
com.aws.greengrass.shadowmanager.ShadowManagerDAOImpl 84% 91% 77% :white_check_mark:
com.aws.greengrass.shadowmanager.AuthorizationHandlerWrapper 100% 100% 100% :white_check_mark:
com.aws.greengrass.shadowmanager.PubSubIntegrator 22% 22% 21% :x:
com.aws.greengrass.shadowmanager.ShadowManager$1 14% 14% 0% :x:
software.amazon.awssdk.aws.greengrass.GeneratedAbstractDeleteThingShadowOperationHandler 100% 100% 0% :white_check_mark:
software.amazon.awssdk.aws.greengrass.GeneratedAbstractListNamedShadowsForThingOperationHandler 100% 100% 0% :white_check_mark:
software.amazon.awssdk.aws.greengrass.GeneratedAbstractUpdateThingShadowOperationHandler 100% 100% 0% :white_check_mark:
software.amazon.awssdk.aws.greengrass.GeneratedAbstractGetThingShadowOperationHandler 100% 100% 0% :white_check_mark:
com.aws.greengrass.shadowmanager.util.JsonMerger 64% 72% 56% :white_check_mark:
com.aws.greengrass.shadowmanager.util.SyncNodeMerger 76% 88% 64% :white_check_mark:
com.aws.greengrass.shadowmanager.util.DataOwner 100% 100% 0% :white_check_mark:
com.aws.greengrass.shadowmanager.util.ShadowWriteSynchronizeHelper 100% 100% 0% :white_check_mark:
com.aws.greengrass.shadowmanager.util.Validator 67% 68% 65% :white_check_mark:
com.aws.greengrass.shadowmanager.util.JsonUtil 81% 89% 74% :white_check_mark:
com.aws.greengrass.shadowmanager.ipc.BaseRequestHandler 77% 77% 0% :white_check_mark:
com.aws.greengrass.shadowmanager.ipc.GetThingShadowRequestHandler 87% 74% 100% :white_check_mark:
com.aws.greengrass.shadowmanager.ipc.IpcRateLimiter 100% 100% 100% :white_check_mark:
com.aws.greengrass.shadowmanager.ipc.PubSubClientWrapper 82% 82% 0% :white_check_mark:
com.aws.greengrass.shadowmanager.ipc.UpdateThingShadowIPCHandler 55% 55% 0% :white_check_mark:
com.aws.greengrass.shadowmanager.ipc.InboundRateLimiter 94% 100% 88% :white_check_mark:
com.aws.greengrass.shadowmanager.ipc.InboundRateLimiter$1 75% 100% 50% :white_check_mark:
com.aws.greengrass.shadowmanager.ipc.DeleteThingShadowRequestHandler 86% 72% 100% :white_check_mark:
com.aws.greengrass.shadowmanager.ipc.UpdateThingShadowRequestHandler 74% 76% 72% :white_check_mark:
com.aws.greengrass.shadowmanager.ipc.DeleteThingShadowIPCHandler 53% 53% 0% :white_check_mark:
com.aws.greengrass.shadowmanager.ipc.GetThingShadowIPCHandler 95% 95% 0% :white_check_mark:
com.aws.greengrass.shadowmanager.ipc.ListNamedShadowsForThingIPCHandler 61% 55% 67% :white_check_mark:
com.aws.greengrass.shadowmanager.ipc.model.Operation 100% 100% 0% :white_check_mark:
com.aws.greengrass.shadowmanager.ipc.model.PubSubRequest 100% 100% 0% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.strategy.PeriodicSyncStrategy 71% 92% 50% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.strategy.SyncStrategyFactory 100% 100% 100% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.strategy.BaseSyncStrategy 78% 85% 70% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.strategy.RealTimeSyncStrategy 83% 90% 75% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.IotDataPlaneClientFactory 77% 93% 61% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.RequestBlockingQueue 63% 71% 55% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.SyncHandler 82% 93% 71% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.IotDataPlaneClientWrapper 100% 100% 0% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.RequestMerger 61% 67% 56% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.CloudDataClient 51% 46% 57% :white_check_mark:
com.aws.greengrass.shadowmanager.model.configuration.ShadowSyncConfiguration 71% 81% 61% :white_check_mark:
com.aws.greengrass.shadowmanager.model.configuration.ThingShadowSyncConfiguration 76% 78% 75% :white_check_mark:
com.aws.greengrass.shadowmanager.configuration.ShadowDocSizeConfiguration 61% 73% 50% :white_check_mark:
com.aws.greengrass.shadowmanager.configuration.RateLimitsConfiguration 100% 100% 0% :white_check_mark:
com.aws.greengrass.shadowmanager.configuration.ComponentConfiguration 100% 100% 0% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.strategy.model.StrategyType 55% 69% 40% :white_check_mark:
com.aws.greengrass.shadowmanager.sync.strategy.model.Strategy 90% 100% 80% :white_check_mark:
com.aws.greengrass.shadowmanager.model.ResponseMessageBuilder 91% 91% 0% :white_check_mark:
com.aws.greengrass.shadowmanager.model.ShadowDocument 85% 93% 77% :white_check_mark:
com.aws.greengrass.shadowmanager.model.ShadowStateMetadata 83% 88% 78% :white_check_mark:
com.aws.greengrass.shadowmanager.model.ErrorMessage 97% 94% 100% :white_check_mark:
com.aws.greengrass.shadowmanager.model.ShadowRequest 88% 77% 100% :white_check_mark:
com.aws.greengrass.shadowmanager.model.ShadowState 96% 98% 94% :white_check_mark:
com.aws.greengrass.shadowmanager.model.LogEvents 100% 100% 0% :white_check_mark:

Minimum allowed coverage is 45%

Generated by :monkey: cobertura-action against 7b737ffbc564802adb22a40facaf8d9acee88273

jcosentino11 commented 1 year ago

https://github.com/aws-greengrass/aws-greengrass-shadow-manager/pull/171#discussion_r1110003904

@Nelsonochoam , Github isn't letting me add a comment....but yeah I'm in the process of adding tests, will kick it out of draft when everything is polished and ready