search

buttonmen-dev / buttonmen

Buttonmen - an online dice game
Other
16 stars 24 forks source link

Statistics for buttons #614

Open irilyth opened 10 years ago

irilyth commented 10 years ago

It'd be cool if there were a way to see statistics for buttons, e.g. win/loss records vs other buttons. Even fancier stuff could be nice too, like a way to see the W/L/T numbers for the games played between two buttons, or even to actually see the games themselves (if they're public or yours), etc.

cgolubi1 commented 10 years ago

FWIW, i have screen-scraped data from bmstats on the old site, so my first cut at this would be something similar to that --- the old site had, for each button:

cgolubi1 commented 10 years ago

I'm also hoping to be able to at some point list "legacy" stats for each button which is a holdover from the old site, i.e. actually load my canned data onto the site in some sort of read-only way. I don't even have a proposal for this yet --- when we first started working on the rewrite, i hoped i could just bring in the old stats and we could pick up where we left off, but that's not realistic because too many little things are new or different. But i do want to make the old stats available.

irilyth commented 10 years ago

Yeah, I was just gonna ask about the old site's stats. I think it'd be very cool to incorporate them somehow.

blackshadowshade commented 10 years ago

I think that it wouldn't be too hard to incorporate the stats from the old site into our new stats.

irilyth commented 10 years ago

This is my BFOTM for July. If there's way more than two hours of work and you want input on which parts I care about most, lemme know; or just work on whatever's best to work on. :^)

blackshadowshade commented 10 years ago

Right, after chatting on IRC, Chaos is going to kick this one off by considering how to import the stats from the old site.

irilyth commented 10 years ago

Some other thoughts from IRC:

I think pre-computing the win/loss stats for a couple of things is good. Things the old site had:

Of those, button vs button is the one I most care about, because my ongoing tournament/league things (Mad Robin and UBFC) both consume that (and don't care about anything player-related).

cgolubi1 commented 10 years ago

I put in my two hours looking at the stats i have downloaded from DHS, thinking about what to do with them, and starting to poke at reformatting the data correctly. Here's my thoughts about what we can do using the available data --- interested folks, weigh in:

First, what do the stats that i have for the old site actually look like? I have downloaded stats on a per-button basis (nothing on a per-player basis). For each button, i have:

For anyone who's lost track, the date info is important because i have this data basically as a side effect --- it was the cache of a script that downloaded new data whenever it needed data about a given button and found that its cache was missing or too old. So the upshot is that the newest file i have is dated 2012-02-23, while the oldest one is over a year older, 2011-02-18. So obviously if the data for button A was downloaded a year earlier than the data for button B, the two data sets might disagree about how many times buttons A and B played. Fortunately, there's no ambiguity about which data is better --- the more recent data will always be closer to an accurate representation of the matches between A and B on DHS.

Given that that's the sum total of the data i have, here's my proposal for how to put together usable "compiled DHS stats" which are of a sort which might be interesting enough to also compile for current data:

For button vs button data, corellate the A vs B data and the B vs A data, take the more recent of these, and report it in a dhs_site_button_vs_button_stats table with columns:

(button_a, button_b, games_completed, games_button_a_won, date_compiled)

For player/button pairs, report each one in a dhs_site_button_player_stats table with columns:

(button, player, games_completed_using, games_won_using, games_completed_against, games_won_against, date_compiled)

That doesn't get us overall data about button or player win percentages, and even the standalone context-free number for buttons which i have in my cache files isn't very useful, because there's not a standalone context-free count of games to go along with it, and that's pretty interesting for any sort of usable stats. However, we can reverse-engineer the button numbers by claiming that the set of games in the button_vs_button table is complete, and just adding up the games and wins for each button. Then we can make a table, dhs_button_summary_stats, with columns:

(button, games_completed, games_won, date_compiled)

Ditto for players: we can pretend the numbers obtained from the dhs_site_button_player_stats table are coherent, since they're the best we can get, and compile dhs_player_summary_stats with similar columns, though here the date_compiled column will be slightly more fictional, since the player data really will be a summation across time that doesn't actually represent truth as of any real moment in time:

(player, games_completed, games_won, date_compiled)

A general suggestion: we should not tie these stats to either current player ID or current button ID, but rather either store the old buttons/players by name, or make new tables, dhs_button and dhs_player, each with its own ID space. That way we can simply amend the current button and player tables to have a dhs_id column for that button or player's identity on DHS, which can either be set or null. This will let us make the right decisions for players who are using a different name on the new site, and for buttons which we've decided were "wrong" on the old site. (For example, the Sven record from DHS actually belongs to Little Sven on this site, and Sven on this site has no DHS site record.)

My theory is that we can generate similar summary tables for local data: button_vs_button_stats, button_player_stats, button_summary_stats, player_summary_stats, and create summary pages which spit those tables out. Then any page which shows stats for a button or for a player, can take a site=dhs (or whatever) flag which instead shows summary data for that button/player on DHS, and when we're showing overall information about a button or player, if that button or player has a non-null dhs_id, we can link to the appropriate DHS stats page.

In terms of actual data futzing, i got as far as compiling button vs. button data which i believe does the right thing in terms of getting the most recent available data for all button vs button matchups.

Here's the first few rows of output from that, just as an example:

button_a  button_b   games_completed  games_button_a_won  date_compiled
-----------------------------------------------------------------------
ABCGi     ABCGi      31               31                  1328639585
ABCGi     Abe Caine   2                0                  1328746716
ABCGi     Agatha      2                2                  1328639585
ABCGi     albertel    6                4                  1328639585
ABCGi     Aldric      1                1                  1328639585

I'm not 100% sure what the right way to handle mirror battle data is --- my cached data actually says that ABCGi played 62 mirror games and won 31 of them, but i don't believe that. I think either the original stats pages or my script mucked that up, and it must have actually been 31 games. (Among other things, it seems vanishingly unlikely that every single button was in an even number of mirror battles.)

Anyway, so, when compiling total stats about a button, i think the total number of games that button played in should include mirror battles, but for win percentage, you either have to do: A. exclude mirror battles when computing win percentage B. include mirror battles and assign the button half that many wins

I think A is clearly better --- if you use B, then playing more mirror battles will drag the button's win percentage closer to 50%, and i don't think that's what the overall win percentage statistic is for. But this seems needlessly complicated, so i am open to the assertion that i'm missing something blazingly obvious here.

Anyway. I think this is a good stopping point, because y'all can now weigh in on this proposal about how to store stat data.

AdmiralJota commented 10 years ago

This seems like a pretty good approach in general. A few thoughts:

cgolubi1 commented 10 years ago

I'm guessing that the reason the DHS data and the current data are in separate tables is because you want to base the foreign ID's off of separate lists of players and buttons? If so, then I think it'd make sense to name those columns things like dhs_button_id_a rather than just button_a, to make that explicit.

Seems fine.

Also, rather than game_completed and games_button_a_won, I'd propose having games_button_a_won and games_button_b_won. It makes the table more symmetric between the two buttons, which seems aesthetically nicer (and might make querying it easier too).

Also seems fine.

Why would we need a separate dhs_button_summary_stats table? Isn't it redundant with dhs_site_button_vs_button_stats?

I was thinking "in case we have to jump through any unexpected hoops to make a summary" and "for efficiency". Also, when adding stats about buttons, there's the usual "column A vs. column B" problem that you get whenever you make a table of ethernet cable connections. So either we need to double-enter all matchups, or we need to be able to select a button from either column. You may have a preference for which of those is better. Anyway, we may not strictly need a summary table, especially if we double-enter stats in the button_vs_button table.

Do we want the button/player tables to have a foreign ID column pointing at the DHS tables or do we want the DHS tables to have a foreign ID column pointing at the current tables?

Hmm, either way. I was assuming we'd want to select those things while looking at current data (e.g. while drawing a profile page), but in fact it's not that hard to do two selects, and your way has the advantage that if we turned up some hypothetical other source of third-party information about current players or buttons, we could more cleanly add that too.

For the pages, what do we show if they don't specify site=dhs? I'd suggest showing combined stats as our default, with both site=dhs and site=bw options to split them out. We could probably write a view that returned the combined data without too much trouble.

I think we should support combined stats (site=both or site=all or site=combined), but that new-site-only stats should be the default. A few reasons:

AdmiralJota commented 10 years ago

Also, are there any buttons that exist in the old data that we just don't have an equivalent button on the new site? If so, are we going to try to jump through hoops to include their stats, or are we just going to exclude them? (The latter seems like a perfectly reasonable alternative, although it means that the win percentages for the buttons that played against them might be slightly affected.)

cgolubi1 commented 10 years ago

Also, are there any buttons that exist in the old data that we just don't have an equivalent button on the new site? If so, are we going to try to jump through hoops to include their stats, or are we just going to exclude them? (The latter seems like a perfectly reasonable alternative, although it means that the win percentages for the buttons that played against them might be slightly affected.)

Short answer: no.

Longer answer: my list of button recipes from DHS, which i believe was complete for buttons that actually existed at the time i played on the site, is one of the inputs we initially used to populate the buttonweavers site. I believe that everything for which i have completed game stats is also represented in our database in some form. Several of the DHS buttons had stats listed for playing against two buttons named Alpha and Omega, but those buttons (assuming they even were buttons) never appeared on the site in any other form, so i don't have stats or recipes for them, and have simply omitted the stats about games nominally against them from my compiled stats.

AdmiralJota commented 10 years ago

I was thinking of ones that might have changed their recipes from the old site to the new one, but I guess for most of those, we're keeping both versions (e.g. Rold Rage). Wasn't there at least one fanatic whose recipe was based on the skill letters, where we changed those symbols and had to ask them if they wanted it revised?

Also, I believe that Alpha and Omega were just code names for when the old site used to allow users to make up a recipe for testing, so they should definitely be excluded from all the stats, as they weren't actual buttons.

cgolubi1 commented 10 years ago

Ah, yeah, i think we're changing Gryphon's recipe, since we're no longer planning to use a letter for plasma. That's not a big deal, IMO, well within the scope of recipes which will work differently just because our implementation is generally different.

Cool, re Alpha and Omega. I never knew.

irilyth commented 10 years ago

Thanks a lot for working on this!

One thought from me:

I'm not 100% sure what the right way to handle mirror battle data is --- my cached data actually says that ABCGi played 62 mirror games and won 31 of them, but i don't believe that. I think either the original stats pages or my script mucked that up, and it must have actually be 31 games.

My guess is that ABCGi was 31 - 31 in mirror battles, i.e. it won 31 games and lost 31 games. W + L = total, right? :^) Not if you're playing both sides.

Anyway, so, when compiling total stats about a button, i think the total number of games that button played in should include mirror battles

Sounds right. Should it count as two games or as one game? My intuition is that it's probably one, but something I can't quite put my finger on makes me think that maybe it should be two. Like, if you and I had a time-trial bike race, and we both rode the same bike, that bike would clearly have run the course twice, i.e. been in two races, in some sense. Dunno.

but for win percentage, you either have to do: A. exclude mirror battles when computing win percentage B. include mirror battles and assign the button half that many wins

I think A is clearly better --- if you use B, then playing more mirror battles will drag the button's win percentage closer to 50%, and i don't think that's what the overall win percentage statistic is for.

I agree, and I sort of thought that the old site did that -- that the win percentage didn't include mirror battles. Is that easy to check?

yawetag commented 10 years ago

My two cents: mirror battles should be ignored for button statistics purposes. It's obvious that any mirror battle means the button wins once and loses once.

I would even go so far as to say Echo and [enter other 'copy recipe' button] shouldn't factor into statistics either, as they are essentially mirrored themselves.

Andrew Senger asenger@gmail.com

On Wed, Sep 3, 2014 at 5:23 PM, irilyth notifications@github.com wrote:

Thanks a lot for working on this!

One thought from me:

I'm not 100% sure what the right way to handle mirror battle data is --- my cached data actually says that ABCGi played 62 mirror games and won 31 of them, but i don't believe that. I think either the original stats pages or my script mucked that up, and it must have actually be 31 games.

My guess is that ABCGi was 31 - 31 in mirror battles, i.e. it won 31 games and lost 31 games. W + L = total, right? :^) Not if you're playing both sides.

Anyway, so, when compiling total stats about a button, i think the total number of games that button played in should include mirror battles

Sounds right. Should it count as two games or as one game? My intuition is that it's probably one, but something I can't quite put my finger on makes me think that maybe it should be two. Like, if you and I had a time-trial bike race, and we both rode the same bike, that bike would clearly have run the course twice, i.e. been in two races, in some sense. Dunno.

but for win percentage, you either have to do: A. exclude mirror battles when computing win percentage B. include mirror battles and assign the button half that many wins

I think A is clearly better --- if you use B, then playing more mirror battles will drag the button's win percentage closer to 50%, and i don't think that's what the overall win percentage statistic is for.

I agree, and I sort of thought that the old site did that -- that the win percentage didn't include mirror battles. Is that easy to check?

— Reply to this email directly or view it on GitHub https://github.com/buttonmen-dev/buttonmen/issues/614#issuecomment-54376872 .

AdmiralJota commented 10 years ago

I think that Echo et al. should still count normally. For one, a game with Echo isn't automatically just a win and a loss; it's one or the other, like any match. For another, there might even be a real difference: players who tend to choose Echo might be more likely to be players who are good at mirror matches. Or inexperienced players might be more likely to pick Echo than experienced ones. Or whatever. And finally, if we did exclude them, then Echo wouldn't have any stats at all!

irilyth commented 10 years ago

I agree that Echo and Zero should get stats; my recollection is that they weren't 50% W/L on the old site, and in fact weren't the same percentage as each other either.

yawetag commented 10 years ago

My idea is simply statistics-based.

Echo/Zero, by the simple makeup of its recipe, should be at 50% lifetime, but would realistically be within some statistical bound of such (I doubt they were outside 48-52%). The same probably holds true if you look at each button individually against Echo/Zero - the exact percentage won't be 50%, but it won't be significantly far away from it. This number may be affected by the skill level of players who choose them, but statistics will also say that we have to assume the percentage of people choosing Zero/Echo are of the same percentage levels that choose other buttons.

The point is that if we're using a "dragging (or pulling) the numbers closer to 50%" argument to prevent mirror matches from adding to the statistics (of which I agree), the same should hold true for Zero/Echo matches, as they produce the exact same match as mirrors.

For example, there's probably a recipe that's horrible, one with a win percentage less than 10% (especially if I played it). If I were to run 100 games of crap_button vs. Echo, that percentage is going to jump closer to 50%, as each side should win 50% of mirror matches (statistically speaking). This "jump" would depend on the number of games already completed for the button (one with 100 completed matches will jump more than one with 1,000). And, the opposite would hold true if I chose a button with a win percentage above 90%.

I see no problem with showing the stats against Zero/Echo or Zero/Echo stats against others, but I don't think these stats should be included in the button's overall statistics, as they cause the same skewness of stats as mirror matches. I would hazard a guess that removing Zero/Echo/mirror matches from statistics will give you a much better view of the true abilities of each button.

irilyth commented 9 years ago

This is my BFOTM for October.

My top priority is being able to see the old stats on the site, which I'd like to use in a new overall-winning-percentage-based ladder tournament type thing.

cgolubi1 commented 9 years ago

I gave the button/player stats the same treatment, and looked at summaries of total games per button (which looked reasonable at a glance) and per player. I decided i should double-count button_vs_button after all, i.e. report ABCGi vs. Darrin and also Darrin vs. ABCGi, to make it easier to add stats for buttons without maintaining a separate button_summary table, per Jota.

Total games per player says my overall game count and win percentage on DHS were:

  6152  0.5496  glassonion

As it turns out, i have some sporadic stats about my DHS rank over time (hey, do i make fun of your hobbies?) --- here are the dates when i grabbed those stats from the player rank page which most closely approximate the beginning and end of my dataset... hmm, actually, let me take a step back. Here's what i actually have recorded in my file:

DATE:     RANK  WON  LOST SCORE   PERCENT
------------------------------------------------------------------------
20110221:   43  3204 2661 1704472 .5562
20120222:   38  3514 2866 1860376 .5592

I'm spelling this out because those percentages don't actually make any sense. I'm pretty sure i copied them from the rank table rather than computing them myself, but if you just look at the W/L numbers, it should be:

20110221: 3204 / (3204 + 2661) = 3204 / 5865 = .5463
20140222: 3514 / (3514 + 2866) = 3514 / 6380 = .5508

Given those percentages, and the fact that my compiled data is an arbitrary mixing of records collected between those dates, the sanity-check checks out perfectly.

But the win percentages i transcribed off DHS at the time are almost a full percentage point too high. I might have typoed them, of course, but i bet i didn't. Anyone have any idea/recollection of what was up with that? Did your site win percentage count in-progress games that you were ahead on? This doesn't really matter, of course --- the DHS compiled player win percentages aren't going to have any effect on the stats i'm compiling here because, with a couple of very very sporadic exceptions, no one has them. It's just surprising.

cgolubi1 commented 9 years ago

Alright. So i now have code which can turn my pickle files into a dict containing button_vs_button and button_player. That's as far as i can go with this project, and i don't want to commit my dict to the repo, because it has a bunch of detailed information about individual players' stats on DHS.

So the next thing that has to happen is that someone needs to create some tables to put this data in, and then i can load my actual data into them as part of a deployment.

cgolubi1 commented 9 years ago

I'm actually going to flag this as my October BFotM, in hopes that someone will work with me to get the database schemas setup so i can put the data in them before i lose track of it.

cgolubi1 commented 9 years ago

I'm designating this again as my November BFotM, so i think i get four hours on it. :>) I should take another stab at summarizing the database tables i believe we need, because that wandered somewhat during the thread on this ticket.

cgolubi1 commented 9 years ago

Okay, update of database table ideas based on Jota's feedback, plus stuff i hadn't spelled out:

Figure out how to identify DHS button IDs --- easiest is just add a column to the BW button table, dhs_button_id, which is non-null for any button that existed on DHS. Will that do, or any reason we need a separate table?

For button vs button data, corellate the A vs B data and the B vs A data, take the more recent of these, and report it in a dhs_site_button_vs_button_stats table with columns:

(dhs_button_id_a, dhs_button_id_b, games_button_a_won, games_button_b_won, date_compiled)

For player/button pairs, report each one in a dhs_site_button_player_stats table with columns:

(button, player, games_won_using, games_lost_using, games_won_against, games_lost_against, date_compiled)

We'll want to enter the button_vs_button data twice, once as A vs B and once as B vs A, for each of counting. With that, we probably don't need summary tables, but think about this.

ManvilleFog commented 9 years ago

For button vs button data, corellate the A vs B data and the B vs A data, take the more recent of these, and report it in a dhs_site_button_vs_button_stats table with columns:

Wouldn't the more accurate win rate be based on adding the A and B stats than taking the more recently compiled? It seems to me that just taking one of these sets is removing a very significant percentage of the usable data.

I'm also wondering if it would save time in the long run to just reassign the dhs_button_id with the buttonweavers button id.

cgolubi1 commented 9 years ago

Wouldn't the more accurate win rate be based on adding the A and B stats than taking the more recently compiled? It seems to me that just taking one of these sets is removing a very significant percentage of the usable data.

No. That's not what the data is. The data contains the lifetime statistics about a particular button --- it's just a question of when those statistics were compiled. So if the A stats are more recent than the B stats, then the A stats contain information about all games summarized in the B stats, as well as games that happened after the B stats ended.

blackshadowshade commented 9 years ago

I'm also wondering if it would save time in the long run to just reassign the dhs_button_id with the buttonweavers button id.

I don't know how Chaos has the data stored. If the data is simply based on button names, then this suggestion would be sensible. If, however, she actually has the DHS button ids, then there are a number of reasons that we might want to preserve the previous ids, including if anyone else has a copy of the stats of the old site that we might want to import. I know, for example, that fnord7 used to compile stats, and we might be able to access those eventually.

cgolubi1 commented 9 years ago

No, i don't have access to the old button IDs, so i have to make them up. However, i don't see conflating the DHS and BW IDs as useful. It limits flexibility without buying us anything. (For instance, right now, since DHS buttons have their own set of IDs, we don't have to worry about the fact that RandomBM has stats on DHS but doesn't yet have a button ID on BW. Sure, we can fix that by blocking the stats pull until the RandomBM pull is tested and debugged... but why? They don't have anything to do with each other. I don't see the win.)

Let's give DHS buttons and players their own independent IDs and map between them. It's the relational DB-ish thing to do, it's more flexible, and i'm not aware of any downside.

cgolubi1 commented 9 years ago

Designating this as my BotM for February. Next steps are to populate the DHS data on the dev and prod sites, and start working on API methods to access the data.

blackshadowshade commented 7 years ago

So, just noting that there is some progress on generating stats from the new site in Issue #2208.

blackshadowshade commented 7 years ago

My first proof-of-concept attempt to generate stats (based on my local branch 614_r_stats) produces a HTML page which can be found here: http://stats.dev.buttonweavers.com/ui/stats/button_stats.html

Comments are welcome.

Things I've thought of already:

irilyth commented 7 years ago

Is the plan to generate stats into HTML pages, rather than into data in the databse?

blackshadowshade commented 7 years ago

I think that a mixture of the two approaches might be appropriate. Some stats pages that are commonly consulted are probably better suited to be static HTML pages, while customised stats pages (e.g., those pertaining to a particular user) may need to be dynamically generated from supporting summary data stored in the database.

We know from the old site just how much server load statistics pages can generate. I don't want to end up copping out and adding the standard disclaimer "please don't do this more than 10 times a day", especially if static HTML pages are enough for most purposes.

dwvanstone commented 7 years ago

Thank you! This is wonderful!

Are the buttons (if any) that have played 0 games in this document too?

It would be great to have the Button Set name as another column, especially if it becomes sortable.

There are some buttons at the top I don't recognize from the production site: Skinny & Gordo.

blackshadowshade commented 7 years ago

No, buttons that haven't played any games (apart from mirror battles) aren't included at the moment.

Game 432 has Skinny, and game 416 has Gordo. This must have been before we disabled those buttons where we didn't support the full recipe.

irilyth commented 7 years ago

Here are some things I'd like to do with button vs button stats.

Just some ideas. Yay stats!

blackshadowshade commented 7 years ago

Regarding data formats, since what you're looking to do is import data so that you can do further processing, would CSV be enough?

irilyth commented 7 years ago

I think I care about the contents of the data much more than the format -- doesn't matter at all to me if it's CSV, JSON, whatever, as long as it's something easy for Python to parse. JSON is particularly easy, but CSV is probably doable, if that's easier to generate for some reason.

blackshadowshade commented 7 years ago

The reason I ask about CSV vs JSON is this: CSV produces a multiline file, while JSON produces a single string. In this case, because there are so many matchups, the string ends up being extremely long, so I wonder whether this is preferable to a multiline file.

I'm able to generate both (and other formats) equally easily, so I don't mind, I'm just asking about the practicalities for you.

cgolubi1 commented 7 years ago

I've only sort of been following this discussion, but you can pretty-print JSON so it shows up on multiple lines. Does that help with anything?

irilyth commented 7 years ago

I don't care one tiny bit whether the output has line breaks or not.

irilyth commented 7 years ago

To clarify, I don't mean that in a dismissive way. :^) The only thing I can think of is that more line breaks would increase the size or the dataset, so having fewer lines might make the data transfer faster. But I bet the difference would be totally undetectable.

blackshadowshade commented 7 years ago

@irilyth, I've put up a 32 MB JSON file for you:

http://stats.dev.buttonweavers.com/ui/stats/win_percentage_stats.json

(b1 is the button of interest, b2 is the opponent's button, wp is the win percentage, ng is the number of games played)

Is this roughly what you're looking for?

blackshadowshade commented 7 years ago

2207 has a number of colours that may be useful for the future.

irilyth commented 5 years ago

@irilyth, I've put up a 32 MB JSON file for you:

http://stats.dev.buttonweavers.com/ui/stats/win_percentage_stats.json

(b1 is the button of interest, b2 is the opponent's button, wp is the win percentage, ng is the number of games played)

Is this roughly what you're looking for?

I never got around to looking at this, but might, before the next season of Mad Robin!

A quick question: Are you re-generating this file when you generate stats each month? Or was it a one-time thing? If the latter, can you do it again as another one-time thing?

blackshadowshade commented 5 years ago

I regenerate all the files each time I run the stats, including this one.

Chaos is currently working on getting the R script running once a day, so you will have daily up-to-date stats when that's working.

PoshFrosh commented 1 month ago

Something I've thought of with regards to stats is how stats are affected when we make changes to the site? To know this we would have to compare stats from before the change to stats after the change. Therefore we'd need to be able to slice the stats by date.

For instance, how different will Pappy vs Bruno stats be now that we've implemented the extra die special rule? It would be cool to be able to compare (after enough games have been played with the new special in place).

This is super NOT important (how useful would it really be?). It's just something I thought of while reading this issue and figured I'd share. It might be better suited to creating a one-off bespoke report when notable changes are made, rather than a type of live data set as discussed above.

blackshadowshade commented 1 month ago

I agree that one-off bespoke reports are probably the best way to handle this case.