Closed StayPirate closed 1 month ago
Sadly I'm unable to reproduce this.
The first run I see all the emails, as expected:
./rss2email cron -verbose foo@bar.com
time=2024-06-12T19:58:15.608+03:00 level=DEBUG source=rss2email/processor/processor.go:104 msg="about to process feeds" feed_count=1
time=2024-06-12T19:58:15.609+03:00 level=DEBUG source=rss2email/processor/processor.go:110 msg="starting to process feed" feed=https://www.sentinelone.com/feed/
time=2024-06-12T19:58:15.629+03:00 level=DEBUG source=rss2email/httpfetch/httpfetch.go:183 msg="fetching URL" feed.link=https://www.sentinelone.com/feed/ httpfetch.link=https://www.sentinelone.com/feed/ httpfetch.user-agent="rss2email (https://github.com/skx/rss2email)" httpfetch.insecure=false httpfetch.retry-max=3 httpfetch.retry-delay=5s httpfetch.frequency=15m0s attempt=1
time=2024-06-12T19:58:15.837+03:00 level=DEBUG source=rss2email/httpfetch/httpfetch.go:325 msg="response from request" feed.link=https://www.sentinelone.com/feed/ httpfetch.link=https://www.sentinelone.com/feed/ httpfetch.user-agent="rss2email (https://github.com/skx/rss2email)" httpfetch.insecure=false httpfetch.retry-max=3 httpfetch.retry-delay=5s httpfetch.frequency=15m0s url=https://www.sentinelone.com/feed/ status="200 OK" code=200 size=148118
time=2024-06-12T19:58:15.848+03:00 level=DEBUG source=rss2email/processor/processor.go:292 msg="feed retrieved" feed.link=https://www.sentinelone.com/feed/ entries=10
time=2024-06-12T19:58:15.848+03:00 level=DEBUG source=rss2email/processor/processor.go:379 msg="new entry found in feed" feed.link=https://www.sentinelone.com/feed/ title="Navigating the NVD Backlog | How to Stay Ahead in Vulnerability Management" link=https://www.sentinelone.com/blog/navigating-the-nvd-backlog-how-to-stay-ahead-in-vulnerability-management/
time=2024-06-12T19:58:15.850+03:00 level=DEBUG source=rss2email/processor/emailer/emailer.go:280 msg="preparing to send email" feed.link=https://www.sentinelone.com/feed/ email.link=https://www.sentinelone.com/blog/navigating-the-nvd-backlog-how-to-stay-ahead-in-vulnerability-management/ email.title="Navigating the NVD Backlog | How to Stay Ahead in Vulnerability Management" recipient=foo@bar.com method=sendmail
time=2024-06-12T19:58:15.898+03:00 level=DEBUG source=rss2email/processor/emailer/emailer.go:293 msg="email sent" feed.link=https://www.sentinelone.com/feed/ email.link=https://www.sentinelone.com/blog/navigating-the-nvd-backlog-how-to-stay-ahead-in-vulnerability-management/ email.title="Navigating the NVD Backlog | How to Stay Ahead in Vulnerability Management" recipient=foo@bar.com method=sendmail
time=2024-06-12T19:58:15.898+03:00 level=DEBUG source=rss2email/processor/emailer/emailer.go:300 msg="emails sent" feed.link=https://www.sentinelone.com/feed/ email.link=https://www.sentinelone.com/blog/navigating-the-nvd-backlog-how-to-stay-ahead-in-vulnerability-management/ email.title="Navigating the NVD Backlog | How to Stay Ahead in Vulnerability Management" recipients=1
time=2024-06-12T19:58:15.904+03:00 level=DEBUG source=rss2email/processor/processor.go:379 msg="new entry found in feed" feed.link=https://www.sentinelone.com/feed/ title="Block Attacks with SentinelOne’s AI-Powered CNAPP" link=https://www.sentinelone.com/blog/block-attacks-with-sentinelones-ai-powered-cnapp/
time=2024-06-12T19:58:15.906+03:00 level=DEBUG source=rss2email/processor/emailer/emailer.go:280 msg="preparing to send email" feed.link=https://www.sentinelone.com/feed/ email.link=https://www.sentinelone.com/blog/block-attacks-with-sentinelones-ai-powered-cnapp/ email.title="Block Attacks with SentinelOne’s AI-Powered CNAPP" recipient=foo@bar.com method=sendmail
time=2024-06-12T19:58:15.953+03:00 level=DEBUG source=rss2email/processor/emailer/emailer.go:293 msg="email sent" feed.link=https://www.sentinelone.com/feed/ email.link=https://www.sentinelone.com/blog/block-attacks-with-sentinelones-ai-powered-cnapp/ email.title="Block Attacks with SentinelOne’s AI-Powered CNAPP" recipient=foo@bar.com method=sendmail
time=2024-06-12T19:58:15.953+03:00 level=DEBUG source=rss2email/processor/emailer/emailer.go:300 msg="emails sent" feed.link=https://www.sentinelone.com/feed/ email.link=https://www.sentinelone.com/blog/block-attacks-with-sentinelones-ai-powered-cnapp/ email.title="Block Attacks with SentinelOne’s AI-Powered CNAPP" recipients=1
time=2024-06-12T19:58:15.960+03:00 level=DEBUG source=rss2email/processor/processor.go:379 msg="new entry found in feed" feed.link=https://www.sentinelone.com/feed/ title="Introducing Real-Time CWPP for Amazon ECS Fargate" link=https://www.sentinelone.com/blog/introducing-real-time-cwpp-for-amazon-ecs-fargate/
time=2024-06-12T19:58:15.962+03:00 level=DEBUG source=rss2email/processor/emailer/emailer.go:280 msg="preparing to send email" feed.link=https://www.sentinelone.com/feed/ email.link=https://www.sentinelone.com/blog/introducing-real-time-cwpp-for-amazon-ecs-fargate/ email.title="Introducing Real-Time CWPP for Amazon ECS Fargate" recipient=foo@bar.com method=sendmail
time=2024-06-12T19:58:16.008+03:00 level=DEBUG source=rss2email/processor/emailer/emailer.go:293 msg="email sent" feed.link=https://www.sentinelone.com/feed/ email.link=https://www.sentinelone.com/blog/introducing-real-time-cwpp-for-amazon-ecs-fargate/ email.title="Introducing Real-Time CWPP for Amazon ECS Fargate" recipient=foo@bar.com method=sendmail
time=2024-06-12T19:58:16.008+03:00 level=DEBUG source=rss2email/processor/emailer/emailer.go:300 msg="emails sent" feed.link=https://www.sentinelone.com/feed/ email.link=https://www.sentinelone.com/blog/introducing-real-time-cwpp-for-amazon-ecs-fargate/ email.title="Introducing Real-Time CWPP for Amazon ECS Fargate" recipients=1
time=2024-06-12T19:58:16.015+03:00 level=DEBUG source=rss2email/processor/processor.go:379 msg="new entry found in feed" feed.link=https://www.sentinelone.com/feed/ title="PinnacleOne ExecBrief | China’s “New Quality Productive Forces”" link=https://www.sentinelone.com/blog/pinnacleone-execbrief-new-quality-productive-forces/
time=2024-06-12T19:58:16.017+03:00 level=DEBUG source=rss2email/processor/emailer/emailer.go:280 msg="preparing to send email" feed.link=https://www.sentinelone.com/feed/ email.link=https://www.sentinelone.com/blog/pinnacleone-execbrief-new-quality-productive-forces/ email.title="PinnacleOne ExecBrief | China’s “New Quality Productive Forces”" recipient=foo@bar.com method=sendmail
time=2024-06-12T19:58:16.063+03:00 level=DEBUG source=rss2email/processor/emailer/emailer.go:293 msg="email sent" feed.link=https://www.sentinelone.com/feed/ email.link=https://www.sentinelone.com/blog/pinnacleone-execbrief-new-quality-productive-forces/ email.title="PinnacleOne ExecBrief | China’s “New Quality Productive Forces”" recipient=foo@bar.com method=sendmail
time=2024-06-12T19:58:16.063+03:00 level=DEBUG source=rss2email/processor/emailer/emailer.go:300 msg="emails sent" feed.link=https://www.sentinelone.com/feed/ email.link=https://www.sentinelone.com/blog/pinnacleone-execbrief-new-quality-productive-forces/ email.title="PinnacleOne ExecBrief | China’s “New Quality Productive Forces”" recipients=1
time=2024-06-12T19:58:16.084+03:00 level=DEBUG source=rss2email/processor/processor.go:379 msg="new entry found in feed" feed.link=https://www.sentinelone.com/feed/ title="The Good, the Bad and the Ugly in Cybersecurity – Week 23" link=https://www.sentinelone.com/blog/the-good-the-bad-and-the-ugly-in-cybersecurity-week-23-5/
time=2024-06-12T19:58:16.087+03:00 level=DEBUG source=rss2email/processor/emailer/emailer.go:280 msg="preparing to send email" feed.link=https://www.sentinelone.com/feed/ email.link=https://www.sentinelone.com/blog/the-good-the-bad-and-the-ugly-in-cybersecurity-week-23-5/ email.title="The Good, the Bad and the Ugly in Cybersecurity – Week 23" recipient=foo@bar.com method=sendmail
time=2024-06-12T19:58:16.133+03:00 level=DEBUG source=rss2email/processor/emailer/emailer.go:293 msg="email sent" feed.link=https://www.sentinelone.com/feed/ email.link=https://www.sentinelone.com/blog/the-good-the-bad-and-the-ugly-in-cybersecurity-week-23-5/ email.title="The Good, the Bad and the Ugly in Cybersecurity – Week 23" recipient=foo@bar.com method=sendmail
time=2024-06-12T19:58:16.133+03:00 level=DEBUG source=rss2email/processor/emailer/emailer.go:300 msg="emails sent" feed.link=https://www.sentinelone.com/feed/ email.link=https://www.sentinelone.com/blog/the-good-the-bad-and-the-ugly-in-cybersecurity-week-23-5/ email.title="The Good, the Bad and the Ugly in Cybersecurity – Week 23" recipients=1
time=2024-06-12T19:58:16.141+03:00 level=DEBUG source=rss2email/processor/processor.go:379 msg="new entry found in feed" feed.link=https://www.sentinelone.com/feed/ title="AWS Integrations | Enhancing Visibility & Powering Threat Hunting" link=https://www.sentinelone.com/blog/aws-integrations-enhancing-visibility-powering-threat-hunting/
time=2024-06-12T19:58:16.143+03:00 level=DEBUG source=rss2email/processor/emailer/emailer.go:280 msg="preparing to send email" feed.link=https://www.sentinelone.com/feed/ email.link=https://www.sentinelone.com/blog/aws-integrations-enhancing-visibility-powering-threat-hunting/ email.title="AWS Integrations | Enhancing Visibility & Powering Threat Hunting" recipient=foo@bar.com method=sendmail
time=2024-06-12T19:58:16.186+03:00 level=DEBUG source=rss2email/processor/emailer/emailer.go:293 msg="email sent" feed.link=https://www.sentinelone.com/feed/ email.link=https://www.sentinelone.com/blog/aws-integrations-enhancing-visibility-powering-threat-hunting/ email.title="AWS Integrations | Enhancing Visibility & Powering Threat Hunting" recipient=foo@bar.com method=sendmail
time=2024-06-12T19:58:16.186+03:00 level=DEBUG source=rss2email/processor/emailer/emailer.go:300 msg="emails sent" feed.link=https://www.sentinelone.com/feed/ email.link=https://www.sentinelone.com/blog/aws-integrations-enhancing-visibility-powering-threat-hunting/ email.title="AWS Integrations | Enhancing Visibility & Powering Threat Hunting" recipients=1
time=2024-06-12T19:58:16.193+03:00 level=DEBUG source=rss2email/processor/processor.go:379 msg="new entry found in feed" feed.link=https://www.sentinelone.com/feed/ title="PinnacleOne ExecBrief | Chips and Spies – Insider Threats as China Seeks to Evade Controls" link=https://www.sentinelone.com/blog/pinnacleone-execbrief-chips-and-spies/
time=2024-06-12T19:58:16.195+03:00 level=DEBUG source=rss2email/processor/emailer/emailer.go:280 msg="preparing to send email" feed.link=https://www.sentinelone.com/feed/ email.link=https://www.sentinelone.com/blog/pinnacleone-execbrief-chips-and-spies/ email.title="PinnacleOne ExecBrief | Chips and Spies – Insider Threats as China Seeks to Evade Controls" recipient=foo@bar.com method=sendmail
time=2024-06-12T19:58:16.242+03:00 level=DEBUG source=rss2email/processor/emailer/emailer.go:293 msg="email sent" feed.link=https://www.sentinelone.com/feed/ email.link=https://www.sentinelone.com/blog/pinnacleone-execbrief-chips-and-spies/ email.title="PinnacleOne ExecBrief | Chips and Spies – Insider Threats as China Seeks to Evade Controls" recipient=foo@bar.com method=sendmail
time=2024-06-12T19:58:16.242+03:00 level=DEBUG source=rss2email/processor/emailer/emailer.go:300 msg="emails sent" feed.link=https://www.sentinelone.com/feed/ email.link=https://www.sentinelone.com/blog/pinnacleone-execbrief-chips-and-spies/ email.title="PinnacleOne ExecBrief | Chips and Spies – Insider Threats as China Seeks to Evade Controls" recipients=1
time=2024-06-12T19:58:16.249+03:00 level=DEBUG source=rss2email/processor/processor.go:379 msg="new entry found in feed" feed.link=https://www.sentinelone.com/feed/ title="The Good, the Bad and the Ugly in Cybersecurity – Week 22" link=https://www.sentinelone.com/blog/the-good-the-bad-and-the-ugly-in-cybersecurity-week-22-5/
time=2024-06-12T19:58:16.251+03:00 level=DEBUG source=rss2email/processor/emailer/emailer.go:280 msg="preparing to send email" feed.link=https://www.sentinelone.com/feed/ email.link=https://www.sentinelone.com/blog/the-good-the-bad-and-the-ugly-in-cybersecurity-week-22-5/ email.title="The Good, the Bad and the Ugly in Cybersecurity – Week 22" recipient=foo@bar.com method=sendmail
time=2024-06-12T19:58:16.299+03:00 level=DEBUG source=rss2email/processor/emailer/emailer.go:293 msg="email sent" feed.link=https://www.sentinelone.com/feed/ email.link=https://www.sentinelone.com/blog/the-good-the-bad-and-the-ugly-in-cybersecurity-week-22-5/ email.title="The Good, the Bad and the Ugly in Cybersecurity – Week 22" recipient=foo@bar.com method=sendmail
time=2024-06-12T19:58:16.299+03:00 level=DEBUG source=rss2email/processor/emailer/emailer.go:300 msg="emails sent" feed.link=https://www.sentinelone.com/feed/ email.link=https://www.sentinelone.com/blog/the-good-the-bad-and-the-ugly-in-cybersecurity-week-22-5/ email.title="The Good, the Bad and the Ugly in Cybersecurity – Week 22" recipients=1
time=2024-06-12T19:58:16.307+03:00 level=DEBUG source=rss2email/processor/processor.go:379 msg="new entry found in feed" feed.link=https://www.sentinelone.com/feed/ title="Chained Detections | Revolutionizing Adaptive Threat Hunting" link=https://www.sentinelone.com/blog/chained-detections-revolutionizing-adaptive-threat-hunting/
time=2024-06-12T19:58:16.308+03:00 level=DEBUG source=rss2email/processor/emailer/emailer.go:280 msg="preparing to send email" feed.link=https://www.sentinelone.com/feed/ email.link=https://www.sentinelone.com/blog/chained-detections-revolutionizing-adaptive-threat-hunting/ email.title="Chained Detections | Revolutionizing Adaptive Threat Hunting" recipient=foo@bar.com method=sendmail
time=2024-06-12T19:58:16.364+03:00 level=DEBUG source=rss2email/processor/emailer/emailer.go:293 msg="email sent" feed.link=https://www.sentinelone.com/feed/ email.link=https://www.sentinelone.com/blog/chained-detections-revolutionizing-adaptive-threat-hunting/ email.title="Chained Detections | Revolutionizing Adaptive Threat Hunting" recipient=foo@bar.com method=sendmail
time=2024-06-12T19:58:16.364+03:00 level=DEBUG source=rss2email/processor/emailer/emailer.go:300 msg="emails sent" feed.link=https://www.sentinelone.com/feed/ email.link=https://www.sentinelone.com/blog/chained-detections-revolutionizing-adaptive-threat-hunting/ email.title="Chained Detections | Revolutionizing Adaptive Threat Hunting" recipients=1
time=2024-06-12T19:58:16.372+03:00 level=DEBUG source=rss2email/processor/processor.go:379 msg="new entry found in feed" feed.link=https://www.sentinelone.com/feed/ title="Partnering for Success | A Q&A with Brian Lanigan, SVP of Partner Ecosystem, SentinelOne" link=https://www.sentinelone.com/blog/partnering-for-success-a-qa-with-brian-lanigan-svp-of-partner-ecosystem-sentinelone/
time=2024-06-12T19:58:16.373+03:00 level=DEBUG source=rss2email/processor/emailer/emailer.go:280 msg="preparing to send email" feed.link=https://www.sentinelone.com/feed/ email.link=https://www.sentinelone.com/blog/partnering-for-success-a-qa-with-brian-lanigan-svp-of-partner-ecosystem-sentinelone/ email.title="Partnering for Success | A Q&A with Brian Lanigan, SVP of Partner Ecosystem, SentinelOne" recipient=foo@bar.com method=sendmail
time=2024-06-12T19:58:16.410+03:00 level=DEBUG source=rss2email/processor/emailer/emailer.go:293 msg="email sent" feed.link=https://www.sentinelone.com/feed/ email.link=https://www.sentinelone.com/blog/partnering-for-success-a-qa-with-brian-lanigan-svp-of-partner-ecosystem-sentinelone/ email.title="Partnering for Success | A Q&A with Brian Lanigan, SVP of Partner Ecosystem, SentinelOne" recipient=foo@bar.com method=sendmail
time=2024-06-12T19:58:16.410+03:00 level=DEBUG source=rss2email/processor/emailer/emailer.go:300 msg="emails sent" feed.link=https://www.sentinelone.com/feed/ email.link=https://www.sentinelone.com/blog/partnering-for-success-a-qa-with-brian-lanigan-svp-of-partner-ecosystem-sentinelone/ email.title="Partnering for Success | A Q&A with Brian Lanigan, SVP of Partner Ecosystem, SentinelOne" recipients=1
time=2024-06-12T19:58:16.421+03:00 level=DEBUG source=rss2email/processor/processor.go:446 msg="feed processed" feed.link=https://www.sentinelone.com/feed/ seen_count=0 unseen_count=10
time=2024-06-12T19:58:16.421+03:00 level=DEBUG source=rss2email/processor/processor.go:247 msg="all feeds processed" feed_count=1
The second time I see no emails:
./rss2email cron -verbose foo@bar.com
time=2024-06-12T19:58:26.804+03:00 level=DEBUG source=rss2email/processor/processor.go:104 msg="about to process feeds" feed_count=1
time=2024-06-12T19:58:26.804+03:00 level=DEBUG source=rss2email/processor/processor.go:110 msg="starting to process feed" feed=https://www.sentinelone.com/feed/
time=2024-06-12T19:58:26.837+03:00 level=DEBUG source=rss2email/httpfetch/httpfetch.go:183 msg="fetching URL" feed.link=https://www.sentinelone.com/feed/ httpfetch.link=https://www.sentinelone.com/feed/ httpfetch.user-agent="rss2email (https://github.com/skx/rss2email)" httpfetch.insecure=false httpfetch.retry-max=3 httpfetch.retry-delay=5s httpfetch.frequency=15m0s attempt=1
time=2024-06-12T19:58:27.052+03:00 level=DEBUG source=rss2email/httpfetch/httpfetch.go:325 msg="response from request" feed.link=https://www.sentinelone.com/feed/ httpfetch.link=https://www.sentinelone.com/feed/ httpfetch.user-agent="rss2email (https://github.com/skx/rss2email)" httpfetch.insecure=false httpfetch.retry-max=3 httpfetch.retry-delay=5s httpfetch.frequency=15m0s url=https://www.sentinelone.com/feed/ status="200 OK" code=200 size=148118
time=2024-06-12T19:58:27.063+03:00 level=DEBUG source=rss2email/processor/processor.go:292 msg="feed retrieved" feed.link=https://www.sentinelone.com/feed/ entries=10
time=2024-06-12T19:58:27.198+03:00 level=DEBUG source=rss2email/processor/processor.go:446 msg="feed processed" feed.link=https://www.sentinelone.com/feed/ seen_count=10 unseen_count=0
time=2024-06-12T19:58:27.198+03:00 level=DEBUG source=rss2email/processor/processor.go:247 msg="all feeds processed" feed_count=1
My feed list only has the one item:
./rss2email list -verbose
# 10 entries, aged 0-14 days
https://www.sentinelone.com/feed/
And the list of seen entries is correct:
./rss2email seen
https://www.sentinelone.com/feed/
https://www.sentinelone.com/blog/aws-integrations-enhancing-visibility-powering-threat-hunting/
https://www.sentinelone.com/blog/block-attacks-with-sentinelones-ai-powered-cnapp/
https://www.sentinelone.com/blog/chained-detections-revolutionizing-adaptive-threat-hunting/
https://www.sentinelone.com/blog/introducing-real-time-cwpp-for-amazon-ecs-fargate/
https://www.sentinelone.com/blog/navigating-the-nvd-backlog-how-to-stay-ahead-in-vulnerability-management/
https://www.sentinelone.com/blog/partnering-for-success-a-qa-with-brian-lanigan-svp-of-partner-ecosystem-sentinelone/
https://www.sentinelone.com/blog/pinnacleone-execbrief-chips-and-spies/
https://www.sentinelone.com/blog/pinnacleone-execbrief-new-quality-productive-forces/
https://www.sentinelone.com/blog/the-good-the-bad-and-the-ugly-in-cybersecurity-week-22-5/
https://www.sentinelone.com/blog/the-good-the-bad-and-the-ugly-in-cybersecurity-week-23-5/
Whoops, so that should be something bad in my configuration, which I haven't spotted yet. Thank you for confirmation and sorry if I wasted your time.
This issue is invalid, so closing.
No problem :)
If it was a config-problem that could be reported then please do open a new issue with the suggestion.
Not sure what happening but since 06-06-2024 I'm receiving the same notifications at every run from a specific feed. I didn't change anything other than having updated the container with newer released images.
When it fetches the feed
https://www.sentinelone.com/feed/
all the items are sent as new notifications. For instance, I have many copies of the following email:the
X-RSS-Link
ishttps://www.sentinelone.com/blog/pinnacleone-execbrief-chips-and-spies/
and if I search for it in the rss2email's local database, I can't find it.The only logs I get about that feed are the following, with
LOG_LEVEL=DEBUG
Can you reproduce the same issue with this feed?