Logging to files

[bdotsamir]

little unclear on what the structure of the log folder is. what would that look like?

the way I have it set up on my personal bots is that there's a log folder, then there are folders that have timestamps of the day, then inside of that folder there are separate log files.

like so:

[JackDotJS]

logs/archive should have old log files, compressed to .zip and grouped with other logs by month and year. (I'd actually prefer 7z since that seems to get the smallest file size of the 3 formats im familiar with, but I've struggled to find a library for this)

logs/crash contain copies of logs where the bot has stopped unexpectedly. makes it easier to track down problems instead of having to open a bunch of different log files one by one

logs/all is where i initially planned for all logs to go, but on 2nd thought they could probably just go into the root logs folder so it's kinda unnecessary lol

btw im also wondering if having our own logging-to-file system is necessary. I've had a brief look at the pm2 docs and it seems like they have their own logging solution. should we use that or continue making our own thing?

the thing im worried about is, if we have our own thing running in the same script as the bot, we could potentially miss key information in error/crash logs, since the logging script would stop at the same time.

this issue could PROBABLY be minimal since we're doing the whole sharding manager thing, but it still worries me.

[bdotsamir]

They do have their own logging solution yes, however I find that I rarely use it in favor of my own. Though, if you'd like to go the pm2 route, it would save on complexity :shrug:

[JackDotJS]

mmm maybe we can just continue with our own thing and see if we need anything more in-depth later

[bdotsamir]

the logs/archive one could be done with the compressing package

i assume we'd write all calls to Logger.fatal and Logger.error to the logs/crash day's file? Attaching to uncaughtException is kind of dangerous, same with unhandledRejection (since it'll cause the program to exit anyway).

logs/all: What would this look like? Would there be a single file with the day's timestamp and everything gets sent to it? Then after 24 hours it gets compressed and moved to logs/archive and a new day's file gets created?

[JackDotJS]

the logs/archive one could be done with the compressing package

i assume we'd write all calls to Logger.fatal and Logger.error to the logs/crash day's file? Attaching to uncaughtException is kind of dangerous, same with unhandledRejection (since it'll cause the program to exit anyway).

logs/all: What would this look like? Would there be a single file with the day's timestamp and everything gets sent to it? Then after 24 hours it gets compressed and moved to logs/archive and a new day's file gets created?

(this was answered over discord, for anyone wondering)

TL;DR:

• logging should start in index.ts, which is also where the sharding manager will be initialized
• log files should be written to logs/*.log, using date and time in filename
  • maybe something like YYYY-MM-DD-HH-MM-SS.log?
• add smt that'll allow us to move the files to the crashlog folder when needed
• add smt that'll compress all the old log files at the start of each month (should probably happen early in the script before the logging begins)

[JackDotJS]

@akii0008 with your permission i'd like to merge this PR now and continue working on it via the main branch (that being ts-rewrite, ofc)

i realize its not exactly finished, but neither is the rest of the bot and keeping these things separate seems like it'll just make development that much more of a headache bc of merge conflicts n all

[bdotsamir]

Go ahead if you'd like.

The reason I've been working primarily in PRs is because I'm still kind of learning your style. Your coding style, your contributing style. You may not like a piece of code that I write and have objections- with PRs it makes it easier to discuss changes before they land in the main branch.

If I was working alone on this then I'd just be committing right to main since there'd be nobody else to collaborate with, but I'm now working with you and I want to keep you in the loop before you git pull and have a merge conflict you weren't aware of.

I can continue to create PRs if you'd like, or alternatively, I can just commit straight to main and we can revert changes as needed. Both are equally valid in terms of style, although I'm not the biggest fan of the revert button lol

[bdotsamir]

@JackDotJS ready whenever you are

[JackDotJS]

The reason I've been working primarily in PRs is because I'm still kind of learning your style. Your coding style, your contributing style.

i have a style...?

You may not like a piece of code that I write and have objections- with PRs it makes it easier to discuss changes before they land in the main branch.

If I was working alone on this then I'd just be committing right to main since there'd be nobody else to collaborate with, but I'm now working with you and I want to keep you in the loop before you git pull and have a merge conflict you weren't aware of.

completely valid points and i rly appreciate the consideration.

thing is, this particular PR is for a key part of the bot and it's hard to continue development without it. logging is quite important x)

I can continue to create PRs if you'd like, or alternatively, I can just commit straight to main and we can revert changes as needed. Both are equally valid in terms of style, although I'm not the biggest fan of the revert button lol

i think doing both is fine. i get notifications whenever anyone does smt on the repo so i can deal with any little merge conflicts that pop up from time to time.

maybe we should keep PRs for other significant technical changes outside of foundational scripts like logging.