trilogy-group / tu2k22-vssubhang

0 stars 0 forks source link

Track 10: Intro to AWS #13

Closed subhang-ti closed 2 years ago

subhang-ti commented 2 years ago
subhang-ti commented 2 years ago

Screenshot 2022-08-13 at 6 33 00 PM Screenshot 2022-08-13 at 6 41 10 PM Screenshot 2022-08-13 at 6 43 06 PM

Screenshot 2022-08-13 at 6 47 15 PM

Screenshot 2022-08-13 at 6 47 59 PM

subhang-ti commented 2 years ago

Screenshot 2022-08-13 at 8 10 08 PM

subhang-ti commented 2 years ago

Bonus 1: If given daily logs I would like to store it in the S3 bucket in the following fashion s3//:/AppLogs///. This would ensure that if we want to search for a log for any particular date, we can input the date in the url and we can get in an O(1) time lookup. Use Athena to query logs and we can remove certain % of normal logs. Use Expiration Lifecycle method to remove old logs.

sirjan-cn commented 2 years ago

LGTM, if interested, read up more for bonus 1 here: https://repost.aws/questions/QUjQHGAO_nQd-PsIKsvfH3HA/partitioning-s-3-access-logs-to-optimize-athena-queries