Replace ownCloud with something more stable

[JoshData]

We've had dozens of issues with ownCloud. Most recently, editing events is totally broken (https://github.com/owncloud/calendar/issues/892) -- fortunately I noticed before making a new release of Mail-in-a-Box. Prior to that, ownCloud broke their release distributions and put one version under the name of a different version, and clobbered their own hash signature to hide the problem.

They're making me waste a lot of time trying to pin down their problems.

So I'm now going to be looking into replacing ownCloud with SabreDAV, which is ownCloud's DAV component, plus some new calendar/contacts front-end.

In this scenario, the Files app will be dropped with no replacement (it's not something I ever intended to support).

Suggestions welcome.

[mbardelmeijer]

:+1:

[nomandera]

I have to agree. From my use case all I have ever wanted is reliable contacts, calendar and notes that can be shared between users. I think this a natural match to mailinabox. Unfortunately the shared part seems very hard to achieve outside of OwnCloud and as we know it spends as much time broken as working.

When experimenting for alternatives I particularly liked http://baikal-server.com/

Interestingly on looking again it appears at least some work on shared contacts and calendars is in the works (https://github.com/netgusto/Baikal/issues/303)

[dhpiggott]

I'd be happy with a move away from ownCloud to SabreDAV.

I wouldn't be so sure about Baikal. Before I switched to MIAB I used Baikal to meet my *DAV requirements, but I found on a number of occasions that my contacts had been corrupted. Unfortunately my memory of the corruption issues I had isn't great, so I can't say with certainty that it was Baikal (I may have concluded it was an issue with CardDAV Sync which I was using at the time). It might even have only happened with Radicale which I used for some time before Baikal (maybe that was why I switched!). What I do know is that I've not had any corruption issues with MIAB + DAVdroid.

[PortableTech]

I certainly have my own frustrations with the OwnCloud calendar app where a deleted calendar can not be properly recreated. They certainly could do a better job on things.

That being said, my group has become rather hooked on the file sharing features so I would certainly hate so see that option go. I am sure I could find some other way to do it if I have to make a separate server for that purpose, but it sure was nice having it all in one place and using a common authentication scheme.

[RobWW]

One feature of Owncloud I really find useful is the calendar sharing. I looked at SabreDAV, but found it unclear if this capability is implemented. Hopefully this feature could be retained in any Owncloud replacement.

[nomandera]

I am struggling to find information about shared contacts, calendar and notes using SabreDAV. Obviously it is possible since OC do it but none of the other Card/Caldav projects seems to support it which is odd since they essentially all use SabreDAV as a back end. Its is doubly odd since you can pick any of these projects and find a request for it.

To me this and dropping filesharing are likely the only two viable push backs we would see from dropping OwnCloud.

Edit: If there was ever proof needed. I just auto updated from 8.0.5 to 8.1.1 on my test box. Whereby both contacts and calender apps were disabled, marked experimental, marked as not audited for for security and it is my risk to install. To fix they suggest i need to downgrade to v7 for it to work (yes even though i came from 8.0.5 where it worked). Nice one OwnCloud.

[DeviantEng]

I support this transition.

I run OwnCloud on my own network for file backup, and use the OwnCloud with MiaB only for contacts and calendar. I run MiaB on a small-ish VPS, and have limited room for file storage, so it was never going to be a file storage box for me.

+1 to ditch Owncloud and move to SabreDAV!

[JoshData]

And the latest from ownCloud: https://github.com/owncloud/core/issues/18224

[nstanke]

@JoshData Couldn't we run occ upgrade[1] twice? (As a workaround to release v0.13)

[1]https://github.com/mail-in-a-box/mailinabox/blob/master/setup/owncloud.sh#L185

[JoshData]

Let's take that conversation to https://discourse.mailinabox.email/t/v13-upgrade-issue-with-calendar/757/2.

[inspiredearth]

@JoshData I feel the same way. Found you're post whilst looking for alternatives. For the past year or so I've had to spend hours (literally) messing around with getting Contacts and/or Calendar to work each time I update OC. Sometimes Con and Cal are not able to be updated and/or enabled, and I go through a long winded process to uninstall and install them, along with various suggested hacks. I keep hoping the situation will improve. Added to that is the fact I've been having issues with Contacts not updating... that's been ongoing for many months. Lost count. Anyway... enough of a rant. I'll be looking into SabreDAV. Sharing of Calendars is a must for me though. Can anyone in this discussion confirm if it supports that? Thanks... Jonathan

[hnk]

re-post from the discourse forum:

While I like owncloud for the file-sharing part, I have always been, ... sceptical, about their QA.

Here is a good list of calDAV/cardDAV servers, just for reference https://github.com/Kickball/awesome-selfhosted#calendar-and-address-books4

I know you never wanted the whole file sharing part in miab, but I think it is a good fit for an email server, as it does allow the management and sending of larger attachments via email (as links) in a controlled environment. Many people need to send files around regularly, and offering that is a great thing IMO.

Maybe someone can give input to something like pydio2. I have read good things about it.

[PortableTech]

I could certainly live without owncloud if we had some other, ideally integrated, file storage method. As was mentioned earlier, the ability to move large files into a share location, or store easily accessible common repositories makes things so much simpler, and honestly keeps mailboxes smaller and more manageable. Far to often without such a solution, I see people take that large powerpoint document, or other such item and email it to 50 people, now it takes 51x the space (counting the sender) of the file size in the mail stores. That just does not seem ideal.

Add to that that many commercial email services out there set their attachment limit to 10MB, or sometimes, even 5MB, email can become quite limiting by itself.

I did look at Pydio as suggested earlier and it looks nice, honestly cleaner than owncloud. I think it is well worth consideration.

In the end it would be great if we could get someway for RoundCube to be able to sync with the contacts list for ActivSync. Right now I just export and import it from RoundCube to owncloud from time to time, but automating that would make end users much happier.

[nomandera]

So as I interpret the discussion to date, any replacement for OwnCloud,

Must provide:

• calendar
• address book
• multi user file sharing

lack of this following features will count as a regression for many users (me included):

• multi user shared calendar
• multi user shared address book

This to me sounds like we may be looking at using multiple projects and joining them with central authentication. Whilst this all may seem pretty obvious it is nice to see that no one is making a case for any other unofficial OwnCloud addons.

Also technically the only file sharing requirement so far could be handled with maildir de-duplication.

[PortableTech]

I received the following recommendation from a friend of mine. I have not used it myself, but it does is developed on GitHub.

http://baikal-server.com/

Baikal seems to focus on doing Address Books and Calendars well, and nothing else.

[nomandera]

There is some reference to shared Address Books and Calendars on the development branch. See third post of this thread. However for the best part of a year there are no code changes linked to that ticket or any movement on the whole branch in 6 months.

Apart from that it is an excellent product.

[hnk]

Baikal looks nice, but from what I understand there is no way to define an external authentication system. (EDIT: As I wrote this I did indeed find a way to implement imap auth. See a little further down.) Also looking at the Roadmap I can see that 'import' and 'export' features are still only planned. This would be a blocker for migration.

The only project that I could find that handles imap authentication is radicale. It does calDAV and cardDAV. There is an option for rights management, but it is file based. It should be possible to get this connected to a web interface somehow to manage calander sharing.

sabre/dav is also a nice option and I found a comment that proposes a proper way to authenticate via imap on the baikal github issues page. Combined with an online calendar browser like AgenDAV I think it is possible to get a good working solution. But the sabre/dav documentation says that calendar sharing is hard to implement. That is a minus.

Combining something like sabre/dav with agenDAV and a cardDAV plugin for roundcube like this one would provide a good solution with tools that are built to do one thing, but do it right.

But I think there must be a lot more discussion about the tools in question and more inspection about the viability of a switch to a new tool. Questions like

• Can the new tools be installed automatically via our available toolset?
• Can the new tools be configured automatically via our available toolset?
• Can we migrate existing installations to the new tools, or
• Do we have to create a new, backwards incompatible, version of miab and if so
• Does that old version receive bug-fixes and security patches as well?

come to mind. This is something that needs to be well evaluated by @JoshData for each available tool option before making a decision.

I am more than happy to test tools and see how well they might integrate with MIAB in my spare time. If you ( @JoshData ) want to define a set way of going about this task, let me know.

[JoshData]

The only absolute requirements are an address book and a sane upgrade path. Calendar, calendar sharing, and file sharing are nice-to-haves, but let's not get carried away.

So far I'm leaning toward Sabredav because it means a clean separation of the backend storage and UI, which means we can swap in any UI as we like, and since it's the backend within ownCloud we know it's mature (and as far as I can tell, not related to any of ownCloud's issues).

[DeviantEng]

I agree with @JoshData. Contacts/address book is a must, but the rest are definitely nice-to-haves.

[nomandera]

Its not as easy to say "nice-to-haves" when you are taking features away users already have and need.

But I have to agree, more than anything OwnCloud needs to go.

[nstanke]

@JoshData Did you start working on a Sabre/Dav integration?

[JoshData]

No. I don't know when I'll have time for it either yet.

[nstanke]

@JoshData Thanks for letting me know. I'll start working on a POC then.

[CaffTech]

How tied in with the rest of the system is owncloud? Would it be easy to alter the install to prompt whether to install or bypass owncloud? This could be temporary fix until Sabre/Dav is working.

[nomandera]

For what it is worth I just upgraded my local dedicated OC setup and again it broke calendars and contacts. I have fixed them yet again but it adds weight to the argument that the bit we actually use in MIAB is the least reliable part of OC.

Also it would seem that you cant share contacts anymore either (essentially breaking the entire contact app for me).

[bzg]

I've not read the full thread but I don't see CozyCloud (https://cozy.io) mentioned here. Maybe too full-featured for the purpose at hand, but worth considering?

[brylie]

:+1: for Cozy.io

[frankrousseau]

:+1 for Cozy :)

[h8h]

Cozy is quite nice, but can you share you calendars? As far as I know you can invite friends for each events, but not share the whole calendar with them.

[PortableTech]

Cozy looks pretty promising. will have to play around with it some more.

[nstanke]

-1 for Cozy. Because it's in NodeJS.

[nomandera]

Attractive to look at but it seems you cant share contacts either.

[zoof]

What about DAVical? It appears to still be under active development and I've used it for years for both calendaring and contacts. There are two attractive web based front ends: caldavzap and caldavmate.

[nstanke]

I absolutely would vote for DAVical, seems to be by far the best calendar implementation, but I guess PostgreSQL is to clumsy for MiaB.

[nomandera]

I had never even seen it before and playing with the demo and reading its specs I like what I see so far.

Will try and get time to test it properly ASAP

[zoof]

And for contact integration with roundcube, I used https://github.com/christian-putzke/Roundcube-CardDAV. There are also calendar plugins that are supposed to work with caldav that I haven't used -- none of my users are interested and I use Thunderbird/Lightning.

[rockman73]

Maybe all MIAB needs is to be a really awesome MAIL server and nothing else for now? Fruux is a perfectly acceptable service to use as a workaround. Sync.com for cloud storage.

Perhaps allow the admin to (un)select ownCloud from the MIAB installation? This would be a nice option for those who just need a MIAB. :)

[aspdye]

or we just work together with the ownClouders to create someting Great ;)

[rockman73]

Sure, but right now it appears owncloud is unreliable. Why used a hacked-up Card/CalDAV implementation when Fruux (the folks who maintain SabreDAV) works nearly perfectly? Isn't the point of MIAB is an easily to stand up and maintain email server? "Take back control of your email with this easy-to-deploy mail server in a box." owncloud just degrades the experience in this context.

[brylie]

The cloud storage and collaboration aspects are really important as well. In essence email is only one part of the picture. We need a unified system, so that we can liberate ourselves from GMail, Google Drive, and Google Calendar.

On 12 November 2015 03:08:54 EET, rockman73 notifications@github.com wrote:

Sure, but right now it appears owncloud is unreliable. Why used a hacked-up Card/CalDAV implementation when Fruux (the folks who maintain SabreDAV) works nearly perfectly? Isn't the point of MIAB is an easily to stand up and maintain email server? "Take back control of your email with this easy-to-deploy mail server in a box." owncloud just degrades the experience in this context.

---

Reply to this email directly or view it on GitHub: https://github.com/mail-in-a-box/mailinabox/issues/514#issuecomment-155965351

Sent from my Android device with K-9 Mail. Please excuse my brevity.

[dhpiggott]

@rockman73 I don't doubt that Fruux build a high quality product and provide a good service. But to me CalDAV & CardDAV are core to what it is to provide email services. As I perceive one of MIAB's goals to be to enable redecentralization, it should continue providing integrated CalDAV & CardDAV support so that our boxes remain independent appliances.

At risk of going off at a tangent, but to illustrate my own feeling about how key CalDAV & CardDAV are, it continues to surprise me that Thunderbird does not integrate CalDAV & CardDAV support and that it has to be added by plugins. Which is partly why I use Evolution instead.

I'd sooner see ownCloud left in place than removed entirely -- bugs or not. I've not actually been affected by the ownCloud issues, presumably because I don't use its web UI at all -- I just sync Evolution and DAVdroid with it. Which is why I'd be equally happy for it to be replaced with a separate backend and frontend (or not, because we could always add one later -- in an ideal world, there'd be a single web UI that could replace Roundcube and the ownCloud UI, providing email, contacts and calendaring all in one).

@nstanke What do you mean by "I guess PostgreSQL is to clumsy for MiaB"?

[JoshData]

We need a unified system

What people need isn't the issue. People need a lot of things. The limiting factor is that I don't have time to be the maintainer of a project that takes up more of my time. So as I've said, I want Mail-in-a-Box to provide an address book (because you can't email people if you don't know their address). We've gotten calendars and files along for the ride so far, but if dropping those features makes my life easier, then I'll do that (but we'll find a way to offer users an upgrade path to some other service).

[PortableTech]

Well said Josh. Mail is clearly our core concept here, and address books are core to that. In this statement there is no doubt. As to Calendars and File Sharing, they are nice, really nice. They are what many of us view as a collaboration suite, and clearly is the path that the majority of providers have taken, but that does not mean it has to be completely unified.

As to Calendar, it is not even currently integrated into roundcube, not directly anyway. File Sharing is not either, but there are a ton of third party solutions. If we can at minimum work out the ability for third party services, of which we have named several in this thread, to work with the existing user database, then there can always exist the option for individuals to integrate and manage independently of the MiaB control panel the other services.

Personally I like ownCloud, but sadly like so many things that try to do everything well, they tend to do mediocre at best. At it stands, I have been fortunate in not experiencing the majority of bugs, but I know others have. Some of the other solutions that have been discussed focus on doing one thing well, and often that can lead to a better result. much like the core install of MiaB does it's job very well.

No matter which way this all goes in the end, your efforts continue to be greatly appreciated Josh as well as all those who help contribute to this project.

[DeviantEng]

Mail and carddav are the primary needs, IMO.

I run MiaB on a VPS with a 30GB drive, so after web site space, and mailbox space, room for file storage is very little.

[brylie]

The limiting factor is that I don't have time to be the maintainer of a project that takes up more of my time.

How could more developers contribute to maintaining this project? What are some ways we can encourage community stewardship?

[nomandera]

Devils advocate. Could we work on DAVical as a standalone docker as both a proof of concept of DAVical itself and potentially using docker containers as modules to compliment traditional apt-get'ing required components.

My thought is that if viable this modular approach makes the effort to assist modular as well reducing the very steep curve to offer any meaningful dev help here.

[dhpiggott]

Nice idea -- using Docker for *DAV services. I've kept out of all discussions around Dockerising MIAB until now, but flipping it around like this is something I could support -- using Docker only for some aspects of MIAB, where it can be advantageous to do so. It could simplify prototyping/development of the ownCloud replacement.

I nearly commented asking why you favour DAVical and what we would use to provide CardDAV support because from the name and first glance of the website it sounded like DAVical doesn't provide it. In case anyone else nearly makes the mistake though, this bit of the website corrected my misunderstanding:

DAViCal creates two collections automatically when a user is created. In recent versions these are called 'calendar' and 'addressbook'. Some software also makes it easy to create more calendars and addressbooks, or you can create more through DAViCal's web interface, also.

[JoshData]

@brylie I appreciate you asking that. Answering questions on the forum, helping to triage/fix issues and pull requests, and testing possible future changes (like Davical) would all be helpful.

@anoma I don't think mixing issues is going to help, and I really don't see a future for docker here anymore. (#590) (That's not a pronouncement - just a prediction.)

[zoof]

I just got DAVical working on a different VPS instance so I'm happy to help with putting together a pull request -- I will need some help/guidance though since I have no familiarity with nginx and I'm not sure what I can install and run in MiaB without clobbering something else.

[nomandera]

@JoshData I see what you are saying but I dont think we are mixing issues. The problem is most of us could install Davical and develop with it but few of us could integrate it into the complex MIAB appliance. Also in doing do you pollute the appliance itself making it no longer MIAB. Net result few people are doing any dev.

However if you could have DAVical working within MIAB but isolated in a container simply by issuing a "docker run miab/davical" command many of the hurdles and problems go away.

As if to highlight my point @zoof just hit this exact scenario. This is not a coincidence it is exactly what docker is designed to do.