♻️ Refactor SSW Rules Widget

JackDevAU commented 2 months ago

Cc: @bradystroud @babakamyljanovssw @adamcogan

Hi Team,

Pain

Our current SSW Rules Widget relies on Azure Functions and a CosmosDB table, which adds unnecessary complexity and maintenance overhead to our system. The current issues we are facing is due to the builds and Azure Functions being out of sync and not storing all the data required.

What code could be improved?

The SSW Rules Widget codebase, specifically the parts that interact with Azure Functions to fetch rules.
A Single consistent retrieval method(s) that return the latest Rules information. Function is used and exported from the widget and used inside Rules + Widget.

Acceptance Criteria

[x] Add new fields title, uri, isArchived to history.json
[x] Refactor to use the history.json api instead of the Azure Functions
[x] Remove all Azure Functions previously used for fetching rules
[x] Remove the unnecessary table from CosmosDB
[x] Ensure all existing functionality is maintained
[x] Update documentation to reflect new architecture
[x] Performance benchmarks show no degradation
[ ] ~~Implement GitHub API integration to fetch GitHub user information~~
[ ] ~~All tests pass, including new tests for GitHub API integration~~
[ ] ~~Implement caching mechanism to optimise GitHub API calls (optional - might not be needed)~~

More Information

We'll need GitHub API credentials or tokens for authentication
Consider implementing a fallback mechanism in case of GitHub API unavailability
Ensure rate limiting considerations are accounted for in the new implementation

Screenshots

Figure: Current Widget Architecture Flow

Figure: Planned Widget Architecture Flow

Thanks!

babakamyljanovssw commented 2 months ago

Update

PRs drafted to add new fields to history.json

https://github.com/SSWConsulting/SSW.Rules.Functions/pull/59 https://github.com/SSWConsulting/SSW.Rules/pull/1440

babakamyljanovssw commented 2 months ago

Update:

There were some blockers due workflows were failing for SSW.Rules and SSW.Rules.Functions projects.

New Issues are created and PRs to resolve these blockers:

Issues:

PRs:

babakamyljanovssw commented 2 months ago

Update:

Drafted PR on SSW.Rules.Widget repo to refactor so it uses history.json and commits.json

https://github.com/SSWConsulting/SSW.Rules.Widget/pull/55

babakamyljanovssw commented 2 months ago

Update:

PRs to update widget version for SSW.Rules and SSW.People projects

babakamyljanovssw commented 2 months ago

cc: @bradystroud @JackDevAU

Update:

We are going to wait for 1 week and then proceed with removing unnecessary tables from CosmosDB

babakamyljanovssw commented 1 month ago

Update:

Drafted PR to remove unused Azure Functions for widget and entity context

https://github.com/SSWConsulting/SSW.Rules.Functions/pull/83

babakamyljanovssw commented 1 month ago

Update:

The UpdateLatestRules functions wasn't being used after last time I checked which is good and can be removed safely, but the GetLatestRules function was called a few times.

Figure: A few calls made to function after last check

Not sure where it is being called from, this function was being used in widget, but we have updated widget to new version to use history.json.

I checked the request logs for this function before widget upgrade and after widget upgrade for the 7 day period:

1400 logs are returned before widget upgrade
7 logs are returned after widget upgrade

Currently investigating where these calls are made from

babakamyljanovssw commented 1 month ago

Update

I couldn't find from where the GetLatestRules are being called yet. The calls are not frequent as it is used to be after recent changes to widget's data source. Currently I'm working on https://github.com/SSWConsulting/SSW.Rules/issues/1448 as top priority.

This work still in progress

babakamyljanovssw commented 1 month ago

✅ GetLatestRules and UpdateLatestRules functions were removed

babakamyljanovssw commented 1 month ago

Implement GitHub API integration to fetch GitHub user information

All tests pass, including new tests for GitHub API integration

Implement caching mechanism to optimize GitHub API calls (optional - might not be needed)

All above tasks are not required, as we don't need to call GitHub API to get user info because commits.json already has user information and widget is using it to get latest rules updated by specified user. For more info look at this PR - https://github.com/SSWConsulting/SSW.Rules.Widget/pull/55

babakamyljanovssw commented 1 month ago

✅ Done - removed LatestRules container from cosmos db

Figure: current cosmos db containers

SSWConsulting / SSW.Rules