sthewissen / MVP

Unofficial app to help MVPs manage their community activities
GNU General Public License v3.0
112 stars 31 forks source link

[Feature] Show overview of activities #2

Closed sthewissen closed 3 years ago

sthewissen commented 4 years ago

What

sthewissen commented 4 years ago

Updated the specs on this one.

sthewissen commented 4 years ago

Challenge here is to grab all the OpenGraph images for the entire backlog of activities. It's quite an intensive operation to run, so it probably needs to run in the background (maybe through something like Shiny?). Parts of the backlog will probably never change anymore since they are activities from outside of the current approval window. They will also probably need to be cached because we don't want to grab this data every time the overview is loaded. Both the image itself and the URL should be cached.

One caveat is that the URL could change for an item from outside of the app (e.g. through the web portal), making the cache stale. This means that there needs to be some sort of trigger to represent that OR we simply ignore that scenario and have the user edit the thing (without changing anything) to trigger a save and grab the new image.

sthewissen commented 4 years ago

Additional challenge We want to load these contributions progressively since there are people with decades worth of MVP contributions. My last experience with CollectionView and the Threshold functionality wasn't a glorious one though. @Depechie has an implementation that might work better, so keeping that in here for reference.

Depechie commented 4 years ago

If you are referring to my POC for the data loading/caching, it just uses the CollectionView threshold too :/ What are the issues you ran into in previous tries when using it?

And I do agree, somehow we should have more options, instead of what the API can brings us now. Not sure how communicative the API builders at Microsoft are? You mailed them before I guess?

sthewissen commented 4 years ago

Oh boy. I wish I could say something nice about them, but they are pretty much non-responsive. I think it's also built externally (which is odd to me but heck). Just no response whatsoever, even when going through internal MS channels so the odds of something changing on that end are close to zero.

What I had with CollectionView thresholding was that I saw items duplicating and seemingly the same API call being done a few times (which I couldn't quite explain yet).

sthewissen commented 4 years ago

Ok, looking at this issue once more, I scrapped a few features based on lacking API things:

Caching of stale data

The biggest issue I still have is grabbing all the data and caching it locally. Ideally, I want to have all the contributions stored locally and only ping the API sporadically when needed. There is a cutoff point (April 1st of each year I believe) after which everything before it is not editable anymore. This data should never be coming from the API anymore after an initial load.

However, creating some sort of caching mechanism that merges those two sets of potentially editable and uneditable activities is what is kind of boggling my mind right now. Especially as it's a shifting set as the next April 1st a new batch of things gets smacked into the "uneditable" bin.

sthewissen commented 3 years ago

So, verdict: There will be no caching of contributions. We will always poll the API for the latest set and progressively load more as the user scrolls. Managing a cache and keeping that in sync is just too much of a hassle for something that should be fairly simple and straightforward.