search

jim256 / ebay

0 stars 0 forks source link

Log file appending number after file extension on each run #11

Closed jim256 closed 4 years ago

jim256 commented 4 years ago

Each time the spider runs, the log filenames generated are in the following format: ebay.log ebay.log.1 ebay.log.2 ....

The only issue here is that it causes my editor to not recognize it as a log file and makes it a tad harder to read through without renaming each one each time :)

Would you be able to switch it so that it puts the appendage before the file extension? Also, now that I think about it - if you wouldn't mind, it'd be nice to have the log filenames contain the UTC date and time they were run, something like the following if it ran once per hour: ebay--11-7-19--11-24-00.log ebay--11-7-19--12-24-00.log ebay--11-7-19--13-24-00.log ...

james-carpenter commented 4 years ago

That's certainly possible. You just give up the log rotation that keeps only the last X log files. If you don't mind doing the cleanup of old log files some other way, we can put the timestamp in the log filename as you have above.

jim256 commented 4 years ago

@james-carpenter Oh. Right. I hadn't realized that's how that was working.

Really the only disadvantage is being able to read it in an editor with the highlighting. Any thoughts on a way around that?

james-carpenter commented 4 years ago

It depends on what editor you are using.

james-carpenter commented 4 years ago

Let me know how you'd like to handle. Simple either way.

james-carpenter commented 4 years ago

I added this as an option in the next push by overriding the --logfile argument with:

--logfile=logs\{spider}--{date}--{time}.log
jim256 commented 4 years ago

Perfect! That's great. Thanks a lot.