Future of Postal? - Githubissues

jaydrogers commented 4 years ago

I am an open source maintainer myself and I get it -- maintaining open source projects suck. It's a huge time commitment and everyone wants everything for nothing.

The question is: What can contributors do to help?

Postal is very popular tool and is vital to many developer tool kits. The last commit was on July 17th. Seems like there is an increasing number of pull requests that are not being merged.

I'm totally understanding that life happens (and that's the highest priority), but trying to figure out how contributors can help keep Postal going? Is it donating money, time, or other resources?

It would just be interesting to get a discussion going of where Postal goes from here on out. Thanks for everything that you guys do!

Brightside56 commented 4 years ago

@jaydrogers , I think I will just fork it and merge everything myself

jaydrogers commented 4 years ago

I'm fully aware that anyone can fork it and start merging things, but I'm afraid that might just split the community.

What aTech did here is beautiful and I don't want to see it die. Looking at their site, it looks like they have a lot of other amazing apps going on too. I don't blame them one bit if they have their hands tied up with apps that actually pay their bills. Developers need to eat too! 😀

I want to be very respectful of their efforts and generosity with this project.

What I was hoping for out of this discussion was something along the lines of:

Getting a transparent update from the aTech of where things are at and what expectations the community should have for where Postal is going (possibly @adamcooke or @catphish?)
aTech letting us know what they need to continue the project (if that is granting access to more "core contributors" or even donation goals)

If aTech is too busy to respond to this thread (which is still very understandable), then we as a community need to figure out some sort of structure how to move forward.

It's very easy to hit the fork button. It's very difficult to keep a good community together.

It would be very interesting to hear @willpower232's thoughts too because he is a major resource to this project. He helps on almost every issue and probably knows most about this product other than the founders.

adamcooke commented 4 years ago

Thanks for your message @jaydrogers.

We're fully planning on keeping Postal running but we just haven't had the time to commit to much ongoing development. At the moment the 300+ issues is daunting so we need to instigate a policy of not handling any support on GitHuib. I think we'll set up a public mailing list for this, update any links to point people there for support and then close any support issues with a message pointing ticket users to the mailing list.

There are a few maintenance tasks that are needed here which I'd like to get around to quite soon. I imagine I'll have a bit of time to look at this before the end of the year. One thing that's missing (because we deleted it when it was open sourced because it was just reliable and incomplete) is a full and reliable test suite for testing key components. We would like to add this as a priority.

@willpower232 has been very helpful and I can't thank him enough for the help he's been to date.

Leave this with me for a bit and we'll see what we can do to help.

Brightside56 commented 4 years ago

I'm fully aware that anyone can fork it and start merging things, but I'm afraid that might just split the community.

No, my idea is just not more than use some useful PRs and provide possibility for someone else to use it) I'm also planning to dockerize it normally with ansible.

adamcooke commented 4 years ago

No, my idea is just not more than use some useful PRs and provide possibility for someone else to use it)

Hopefully we'll be able to get these merged soon once we've overcome the issue overload. I've already created the Google Group and I'll try and go through the issue list to find any legitimate issues which need attention vs those which are just support requests.

I'm also planning to dockerize it normally with ansible.

That'd be great! 👍

jaydrogers commented 4 years ago

Thanks @adamcooke! We greatly appreciate your insight and I am very excited to hear that there are more things coming for Postal.

I know what you are looking to do is a ton of work, so please let us know what you need for help.

At the moment the 300+ issues is daunting so we need to instigate a policy of not handling any support on GitHuib. I think we'll set up a public mailing list for this, update any links to point people there for support and then close any support issues with a message pointing ticket users to the mailing list.

If I could just share my experience in helping open source projects, I would highly recommend using Github for support:

Github's support experience is STANDARDIZED for everyone. This makes it much easier for people to find issues and help others (especially with code sharing, screenshots, video, etc)
If it is easier for the community to support eachother, then this puts less demand on the founders
Everything for the project will continue to be in one place making it very easy for new members to onboard
People have the option to subscribe to the entire repository or to their specific issue (Github basically a mailing list already, with better features specifically for developers)
The founders don't have to maintain yet another product for mailing lists

Separating features from actual support issues can easily be done with tags. See the examples with our tags and our issue templates (click on "New Issue") on the project we worked on: https://github.com/521dimensions/amplitudejs/issues

I can help you organize and go through these 300+ issues if needed.

Moving forward

Please let us know if there is anything that we can do to help. If you have any requirements for core-contributors, I think there would be a handful of us that could help out.

Let us know what you are most comfortable with. Thanks again for all of your work!

cmmata commented 4 years ago

I agree with @jaydrogers that having a separate forum / issue tracker could add some extra time to the core team, and I think it would be easier to have it all on Github. If the 300+ issues is really a problem, a good solution may be to have some community users triaging and tagging issues, so maintainers and core developers could focus on just selecting and resolving tasks.

By the way, I can offer some help with triage and tag if needed, because at work we have a Postal instance that I maintain.

jaydrogers commented 4 years ago

Glad to see that this thread is getting a lot of people offering their help! This community is filled with a lot of great people and I am excited to see where this goes.

Over the weekend I had some further thought. Long story short, I think the first steps for Postal needs more structure to community than it does new features. I see this being broken down into two chunks:

Step 1 Goals: Organize the community

Create standards of issue labels/tags to use for organizing all action items
Create issue templates to automatically organize newly created issues
Develop requirements for people to apply as "core-contributors" to save @adamcooke and @catphish valuable time. Create this as documentation so future members can also easily participate
Develop a way to organize a roadmap (using something like Github Projects is sweet. We can even develop a way for people to "vote" on the feature within Github)
Organize, address, or close existing issues on Github

Step 2 Goals: Re-ignite the development process

Merge or address pending pull requests
After organizing a roadmap and getting community feedback, execute a small release to get Postal back on the right track

Ideally, if "Step 1" is complete, "Step 2" could be done through community members submitting PRs for the new features. The ultimate goal would be to put as much work as possible to the community.

This would allow the community to do the work of keeping "the ship" running, but still giving @catphish and @adamcooke the steering wheel to guide "the ship".

Just some quick thoughts I had. Just want to be clear that I am not telling you guys what to do, but just offering some suggestions that came to mind as I thought about it more.

Thanks again for everything that you guys do!

jaydrogers commented 4 years ago

Just wanted to say thank you to @adamcooke, @catphish and @willpower232! The number of open issues have dropped by over 50% (150+ issues!) and there have been a number of PRs merged. Holy crap!

I've noticed the Github repo has been moved to a new organization as well (postalhq). It is very exciting to see these updates and know that Postal is far from dead.

Thanks for your hard work guys! Let us know if there is anything else that the rest of the community can help with.

willpower232 commented 4 years ago

:+1: if any of you happen to be experienced at ruby on rails with some time to kill, I think there are a fair few bugs (now labelled) without PRs to try your hand at.

I think the others are aiming to update the rails version and other dependencies at some point but there are definitely other bits worth investigating if nothing else.

adamcooke commented 4 years ago

@willpower232 has been great at getting the issues down to a sensible level and tagging things. We'll hopefully have a bit of time here to look at getting the more important things sorted soon.

jaydrogers commented 4 years ago

Thanks for the update! If there is anything else that we can help as a community, please let us know.

jaydrogers commented 4 years ago

I hate to be "that guy" to open this up again, but I am just curious to see where things are at. It was exciting to see things moving forward but it seems like momentum slowed down again (which is totally understandable).

I completely understand that this is an open-source project and life outside the computer is highest priority.

I am just looking for any sort of expectation of Postal's future or any method we as the community can help keep Postal growing. If there is anything that we can do to help, I am highly interested in volunteering (as I am sure others are too).

Any thoughts @adamcooke, @catphish, or @willpower232?

pjv commented 4 years ago

I am just looking for any sort of expectation of Postal's future or any method we as the community can help keep Postal growing.

Growing would be icing. Personally, I think it would be really good to merge a few bug fix PRs and get the dependencies up to date so Postal can keep running on updated OS versions.

Everything I host is now 100% dependent on Postal for SMTP. It would suck badly if it stopped working at some point by becoming incompatible with an internet that keeps moving forward.

Edit: And I should add that I would really dig in if I had any kind of ruby / rails experience, but I do not. Rails is a black box to me.

adamcooke commented 4 years ago

I still can't make any promises for huge amounts of our time but I have, today, pushed some updates to key dependencies and also started work on some Docker images. Personally, I'm looking to get things running on Kubernetes (which I have mostly achieved) now so will hopefully document those bits soon.

jaydrogers commented 4 years ago

I saw the commits earlier, holy crap! Thanks so much for the updates. It's very promising to see the dependencies get updated.

Is there anything that others can do to help merge some of the bug fixes?

Kubernetes is a great direction, but even getting some of the bugs fixed with a new version launched might be really helpful.

I feel like a lot of people are running their own customized versions of Postal until releases are officially launched.

I know your time is extremely valuable and just didn't know if you had a process in place for others to help review and squash bugs for you.

adamcooke commented 4 years ago

Sorting out some bugs would be great. The main problem is that we don't see the bugs ourselves (and we send millions of messages through our own Postal installs) so fixing them is always quite low priority.

Now I've got the app running on my machine, I may well find myself in a position to get some updates pushed out a bit more frequently.

jaydrogers commented 4 years ago

@willpower232 will be the best resource on what bugs need to be resolved. He is extremely active on working with everyone on bugs.

Maybe @willpower232 could give you a list of priorities of bugs that can definitely be replicated?

Other ideas

Getting issue templates of what every bug needs might be helpful too. See Fastlane for example: https://github.com/fastlane/fastlane/issues/new/choose

Having stuff like this will:

Automatically organize every issue
Put emphasis on the user submitting an issue to submit as much detail as possible (to save a core-contributor as much time as possible fixing issues)

Disclaimer about me: I have no experience in Ruby coding, but I do enjoy organizing tasks and design (see one of my repos here: https://github.com/521dimensions/amplitudejs). I also have Sysadmin background (this is how I found Postal).

If you need any help getting stuff like this done, I have no issues volunteering some time as long as my changes get merged (and you find my work valuable). I don't want you to think that I would be stealing your project from you either. @willpower232 would basically be my "go-to" guy if you were not available.

Just some thoughts. Let me know what you think.

willpower232 commented 4 years ago

Sorting out some bugs would be great. The main problem is that we don't see the bugs ourselves (and we send millions of messages through our own Postal installs) so fixing them is always quite low priority.

Firstly, I feel I should echo this sentiment. Apart from the PRs I've made, we don't experience any issues so replicating problems is difficult when time allows.

I'm hoping to build on the Dockerfile with docker-compose to create an easily managed test environment which should help. I also don't know much Ruby but copy-paste-tweak has served me well so far!

I can see there are 30 open issues tagged bug and some of them have a PR already. Reviewing those would be handy to see if they can be solved with minor code changes and small PRs.

@jaydrogers a PR with one or more github issue templates would be lovely, I keep meaning to but don't get a chance. Basically with some standard tips for what to include would be lovely. Something along the lines of:

postal status output
netstat -lnp output
if they've restarted after changing config if relevant
how much email they're trying to send
etc

I don't have the ability to create tags but it might be worth adding topic-related tags like ui-related, has-pr, dns-related, queue-related, and so on. Then the other issues could be categorised a bit easier.

@adamcooke I'd also suggest pinning this issue and #755 to the issues page as they are probably the most useful. With all the dependencies up to date, the next major step is merging the PRs in where possible. I'm not sure how much of my initial comment is still relevant: https://github.com/postalhq/postal/issues/755#issuecomment-467020995

oviliz commented 4 years ago

I don't know if a good news or not, from a Postal perspective:

Exciting news! We're delighted to announce that we've joined the @KrystalHosting family. Firstly, and most importantly, your service isn't changing and will continue to work as normal and our team is remaining exactly the same.

https://twitter.com/atechmedia/status/1240239030752686081?s=20 https://k.io/

jaydrogers commented 4 years ago

Wow, very interesting! Maybe @adamcooke could give some insight what that means for Postal?

Congrats on the sale guys, you guys definitely worked hard for it!

adamcooke commented 4 years ago

Krystal are also Postal users so nothing changes other than we may well have more resources to be able to work on it going forward. We've just opened the Krystal Open Source Slack channel which you're more than welcome to join to chat with me and the rest of the team here.

https://slack.k.io

clemens commented 3 years ago

Hey everyone,

my company uses Postal as well and we're eager to see it developed further or at least maintained (with relevant updates, such as the recent Rails update, and bugfixes) on a regular basis.

The thing that bothers me when reading through this thread is statements like this:

Sorting out some bugs would be great. The main problem is that we don't see the bugs ourselves (and we send millions of messages through our own Postal installs) so fixing them is always quite low priority.

Judging by its sheer number of stars and forks, Postal seems to be a very popular open source project, but at the same time it seems to be effectively unsupported by its creators/maintainers – but not due to the fact that they've abandoned it but because of lack of reproducibility of issues. From what I can see, this stems mostly from Postal not being set up in a way that facilitates contributions from other developers (= easy to contribute for developers, easy to accept contributions for maintainers).

The biggest culprit here is (in my opinion) the lack of an automated test suite: If the project had a proper automated test suite, you wouldn't have to try to reproduce issues manually and check that proposed fixes actually fixes them – these steps are time consuming and prone to errors. Instead, you could check the tests and see if they make sense and are correct – and if everything looks good and the overall test suite passes as well (= the change didn't break any existing functionality), a contribution can be merged with (reasonable) confidence and (relatively) low effort.

Is there any reason why the project basically doesn't have a test suite? Is there anything preventing the setup of such a test suite? Would this be a contribution that would be accepted?

Apart from that, I echo the sentiment mentioned above (https://github.com/postalhq/postal/issues/917#issuecomment-585422782): Contribution guidelines, issue templates etc. could be tremendously helpful in facilitating contributions (both for contributors as well as maintainers).

gerricom commented 3 years ago

Hello!

We started using Postal three years ago. It serves us very well and fills exactly our needs. As Ubuntu 16 is sunset and I also wanted to relocate our Postal instance to another hoster, I tried to reinstall it from scratch with Ubuntu 20. It kind of works but it is a way full of papercuts as one has to dig through all those issues here and get all the pieces together to get it running.

It is very sad to see, that this project is kind of hung. @willpower232 is doing his best to help everybody around here, but I really would like to see this project actively maintained.

Is there anything we users can do, to get this up and running again? Is it a financial issue? We would sponsor it in any case as far this is possible for us and I can imagine that there are many other users that would do the same.

Please send a sign of life! Thanks!

jaydrogers commented 3 years ago

I think this picture is the scariest part about running Postal in production right now:

We're getting to the point where there has to be some security implications on this (especially with the number of libraries Postal depends on).

At this point, it would be great to see some transparency of where things will be going. @adamcooke made some commitments today (which is exciting🤞), but it would be nice to see a roadmap laid out so people can help out and understand where this project is going.

I think a fork would fail (unless another team as talented as Krystal is able to support it).

I'm very grateful for everything that the Krystal (formerly Atech) team has done, but it's tough when the community can't do anything to help the product.

gerricom commented 3 years ago

For those not following it I want to point out that there is a sign of life in postal's slack channel 🥳

omueller commented 3 years ago

Just in case (cc: @jaydrogers @clemens), to join the slack channel: connect/subscribe under https://slack.k.io and then join the #postal channel.

jaydrogers commented 3 years ago

For those not on Slack, I just wanted to include that the conversation in Slack went stale as well. Hopefully the Krystal team will have time to update soon.

adamcooke commented 3 years ago

The number one thing that, at this point, would be hugely beneficial to the project would be to get a test suite in place - at least for the key components. Once that's sorted, we can confidently accept updates from a wider audience with less risk of things going wrong. If there is anyone in the community who can help with this, do let us know.

If anyone has any technical questions or needs any help getting started with working on the project itself, I'll always be happy to help on Slack. Just join and drop me a message. This, however, does not extend to question relating to installing or using the product - I'm afraid I don't have time to help with individual installs.

Right now, our team is busy working on other things which is why we haven't been able to do a huge amount on Postal. We do still love the product and do have the intention to spend some time on it but I don't have a solid date for that.

jaydrogers commented 3 years ago

Right now, our team is busy working on other things which is why we haven't been able to do a huge amount on Postal. We do still love the product and do have the intention to spend some time on it but I don't have a solid date for that.

We totally understand. We're super grateful for everything that you've done for Postal. Open Source is amazing, but time is always the restriction. At the end of the day, family and friends come first.

From the lens of "what's best for Postal", unfortunately someone will need to spend a lot of time to get it updated. The security risks of running Postal in production are becoming uncomfortably high. If we put that requirement to be your time, then the problem Postal is facing is only going to get worse.

Is it possible for you to offload some responsibilities to other community members? It would be great to still have you involved at the capacity that you want to be, but also have other community members contribute and be able to update the product for you.

Forking the project would be a major setback too. It would divide the community and/or face an "uphill battle" since the new project would have to regain the traction that Postal already has.

@willpower232 has been an incredible asset for years on this project. Are you at least willing to give him the ability to merge PRs? @willpower232 is this something you're comfortable and interested in?

I am very interested in hearing both of your thoughts.

willpower232 commented 3 years ago

Whilst I can sort of understand ruby, I am by no means proficient enough in it to fully understand all the ramifications of a PR so I wouldn't have myself as first choice for merging PRs.

Also even if PRs are merged, the actual release process is internal to Krystal so we as a community cannot actually bring changes to what is probably the vast majority of Postal users.

jaydrogers commented 3 years ago

Thanks for your honesty @willpower232! You're a great asset and glad to have you around.

I'm in the same boat... I don't understand much of Ruby, so I cannot be any help in that department either 😆

I could help in fixing this problem though:

the actual release process is internal to Krystal so we as a community cannot actually bring changes to what is probably the vast majority of Postal users.

We could use Github Actions to automate builds and deliver releases.

@adamcooke: What do you think? What's best for Postal moving forward?

Wait until you have more time?
Allow another Ruby developer who you are comfortable with for merging PRs and possibly adding features, tests, etc

Interested in hearing your thoughts.

jaydrogers commented 3 years ago

Lot's of great updates are happening now! Excited to see the next chapter of Postal! 🎉

postalserver / postal

Future of Postal? #917

Moving forward

Step 1 Goals: Organize the community

Step 2 Goals: Re-ignite the development process

Other ideas