Option to extend transaction storage beyond 30 days

[WyndStryke]

I have been experimenting with a local version of the code which can store up to 3 months worth of data (default still 30 days). This adds this/last/prior month selectors. If this is of interest the source code is attached.

Not sure if you would want this functionality in the public version or not, since the main focus of ATT seems to be performance rather than adding loads of different features, and extending the stored data would have an obvious performance hit (simply adding the option should not have a negative effect, however).

It is a modified version of the official v1.11.2 to add support for up to 92 days (=3 months) worth of stored data instead of 30, and added new selectors for 'this month', 'last month', 'prior month', and 'all time'. This functionality is intended for guild leaders rather than normal guild members.

However, I would recommend that, for performance reasons, the extended period should only be used for a single guild (i.e., the GMs) rather than extending the data retention period for all guilds you are a member of.

Also note that the excellent new exports module may supply what GMs need, without the requirement to store more than 30 days of transaction data.

Caveats - Do not trust the new translations (I am only monolingual). Do not trust the new code (I am new to LUA). Similarly I am new to GitHub and don't know how to do branches / pull requests / etc. I have tested with about 60 days of data, but not across a year boundary or other edge conditions, and only on PC/NA with keyboard&mouse. I presume that os.time() function gives the calendar date on the client machine rather than the server, so 'this month' etc would refer to the local start time and end time..

I have attached the modified v1.11.2 code as a .zip

ArkadiusTradeTools_92_days.zip

Labels - enhancement, user request

[timothymclane]

If you could fork the repo and commit your changes, then link the appropriate branch from here, that would be helpful.

Thanks!

[WyndStryke]

I hope I've done this right, I forked it, downloaded GitHub Desktop, cloned it, merged my local changes, committed it to the fork, and then wrote the changes into the fork's master branch:

https://github.com/WyndStryke/arkadius-trade-tools

So hopefully you should be able to compare the fork with the main project and see what changes I made.

• Extended max transaction storage period from 30 days to 3 months
• Defaults stay at 30 days for performance reasons
• Extended sliders to max 92 days
• Added 'this month', 'last month', 'prior month', and 'all time' filters

Wishlist items

• Only add 'last month' and 'prior month' options if the number of days the user has selected is > 30, or > 60 respectively
• Adjust module sliders appropriately (so a user who has kept 30 days for all guilds should only see sliders for 30 days instead of 92 days)
• Raise a performance warning if the user selects > 30 days in the settings
• Review whether we should use the calendar date from the client PC or the server (i.e., because of time zones)
• Check the actual performance impact (should be zero if user keeps the 30 day default)
• Use the actual month names for the user's language in the combo box selectors instead of saying 'last month' etc.

[timothymclane]

Thanks for taking care of that. 👍

I haven't dug into things deeply yet, but it looks like you've covered all of the appropriate bases.

Your wish list covers the items I was going to bring up. Especially important, IMO, is the max value of the slider. Making that dynamic should be relatively simple, but we don't want to present options to the user which will just frustrate them.

A couple of notes:

• Everything in ATT is based upon UTC. Anything here should use that as well in order to be consistent.
• Using month names will provide significant challenges, as the combo boxes currently have static text and I'm not aware of a way to dynamically generate other options (such as a getter, etc.). Further investigation into the ESOUI combo box implementation would be necessary before this could be pursued.
• Performance will need to be tested with very large saved variables. We have several users which have hundreds of megabytes of saved data (10+ guilds). Ideally, a powerful machine should be able to still load within a reasonable amount of time.
  • Longer login times are fine, but timeouts shouldn't happen with modern machines.
  • Loading screen times (which is when data cleanup happens) need to not be severely impacted. All sales and purchases are iterated on every zone change, so either the time cost needs to be minimal, or saved data cleanup needs to be refined to happen at a different time. (Note: our cleanup code theoretically shouldn't run during the actual loading screens, but my previous testing has shown otherwise.)