haydenwoodhead
commented
2 years ago Sorry, this isn't an issue as such, just somewhere to open a discussion.
No worries. More than happy to have this discussion.
Firstly, can I ask, how many concurrent users could Burner.Kiwi handle using SQLite, or is this entirely limited by server resource? If this could handle say a million emails a day and 1,000 concurrent users, then I might go ahead and do this.
The answer is, I'm not sure. I've never actually bench marked this project to see what it can handle and I don't have any performance monitoring/analytics in place in "production". The current sqlite connection parameters need a bit of tuning but the common wisdom I see online is that sqlite and go can handle a lot of traffic with the correct params. Obviously, it would ultimately depend on what resources you're giving it. I think you'd need to run some benchmarks with the configuration you would like to fully answer that question.
Could any configuration of burner.kiwi handle this much traffic? Absolutely. I have no doubt you could horizontally scale this project to reach whatever traffic level you need.
Ideally I would like to run Burner.Kiwi in a server-less environment, but this would require MailGun, which is expensive. I see someone has talked about using SendGrid before which is a lot cheaper although this isn't built. Do you know if anyone has started such a thing? I might be interested in building this, although I might need a little help.
I would recommend against using serverless if you're going to be providing this sort of service for free to the internet. That was what I initially used when I first built this project, however, someone registered and sent thousands of emails to burner.kiwi. Serverless, of course, scaled to handle this and charged me several hundred dollars for what amounted to less than an hours worth of wall time.
As for sendgrid, it has not been implemented to my knowledge. This should be relatively simple to implement and I'm more than happy to point you in the right direction, do code review and merge it into this repo.
Also FYI if you weren't aware, burner.kiwi does have a built in SMTP server. It can receive it's own email without the need for an external provider. But that won't work in some serverless environments.
Apart from that, I am trying to think of alternative ways which Burner.Kiwi could handle incoming emails in a server-less environment. I am wondering if it would be possible to use AWA SES somehow? From my initial investigation emails could be processed by AWS SES and put into an S3 container, or, piped into a Lambda function. From there anything can happen, maybe onwards to MySQL, Postgres, DynamoDB? Likewise, an Azure LogicApps could also automatically pick up emails from an Outlook inbox and parse that into a database.
I'm sure that there are many ways you could do this in a serverless way. However, as I said above, I'd recommend against it to protect your wallet and just from a complexity perspective. It's also not a direction I want to take the project in, I've even been thinking about removing AWS lambda support because I don't use it and can't guarantee it works correctly. I'd be interested to see where you'd take it but you're on your own here.
Also more generally if you do truly handle that much traffic then burner.kiwi needs some other work. Off the top of my head it would need better logging and performance monitoring, work on the SMTP email parser to ensure that it handles most of the emails people throw at it, as well as some performance tuning.
Happy to keep discussing and help out where I can.
andyinspace
Hi all,
Sorry, this isn't an issue as such, just somewhere to open a discussion.
I currently own a temporary email service similar to that of Burner.Kiwi, although mine is written in PHP and relies heavily on IMAP. While this has worked reasonably well until now, the IMAP server is struggling to keep up with the number of concurrent connections to the single inbox so I am trying to find an alternative. Burner.Kiwi looks quite promising so far.
Firstly, can I ask, how many concurrent users could Burner.Kiwi handle using SQLite, or is this entirely limited by server resource? If this could handle say a million emails a day and 1,000 concurrent users, then I might go ahead and do this.
Ideally I would like to run Burner.Kiwi in a server-less environment, but this would require MailGun, which is expensive. I see someone has talked about using SendGrid before which is a lot cheaper although this isn't built. Do you know if anyone has started such a thing? I might be interested in building this, although I might need a little help.
Apart from that, I am trying to think of alternative ways which Burner.Kiwi could handle incoming emails in a server-less environment. I am wondering if it would be possible to use AWA SES somehow? From my initial investigation emails could be processed by AWS SES and put into an S3 container, or, piped into a Lambda function. From there anything can happen, maybe onwards to MySQL, Postgres, DynamoDB? Likewise, an Azure LogicApps could also automatically pick up emails from an Outlook inbox and parse that into a database.
Any thoughts?