Migrate from Firebase to self hosted backend

[devmount]

Is your feature request related to a problem? Please describe. Firebase is great, performs well and is limited in the free tier (which is totally fine of course). But to make SongDrive completely FOSS I more and more like the idea of providing a realtime backend, which is self hostable, more flexible and not limited to request count per time.

SongDrive is currently build on top of Firestore, a document/collection based realtime database from Firebase/Google. It uses CRUD calls to FB endpoints (creating, reading, updating and deleting documents) and listeners for changes in each collection. And it uses the Firebase authentication to hold user records and their permissions (4 different roles).

Main questions are:

• Does SongDrive really need to be realtime everywhere? It's neat, but is it necessary?
• Which features really require realtime? Like Setlist Sync, song key changes, etc.
• How much effort is the creation of a self-hosted backend? Which solutions are a good fit?
• How much effort is the migration from Firestore to the new backend?

Describe the solution you'd like Just recently, Laravel Reverb got released, which could be a solution.

Describe alternatives you've considered Supabase could be an alternative, but it's pretty much the same in green (literally) and using Postgres instead of doc-based approach.

Additional context Before extending the current data structure as of #145, #163, #201 and #202, this should be decided first. Tagging @EinEnton for reference.

[EinEnton]

I honestly love this 👍 I think Firebase is a great choice to getting something of the ground to show a functional design idea. But once an idea got traction it is either a costly dependency (in a business venture), a risk for future features that are not natively supported and a blocker for, as you stated FOSS and developer engagement.

To discuss options there are in my opinion two main areas to explore:

• Architecture selection
• Technology selection

On the architecture side are question like: Should it be database agnostic? Should it support a local-only mode? Will there be an extended administration capabilities (admin portal with user + rights management...)? Is any multi-tenancy (aka SaaS 😏 ) suppose to be supported? Scale? And of course the mentioned real-time-ness of the intermediate layer.

The technology goes into: Language/stack selection, database selection and team skills

My first decision would be the need for horizontal scalability needs (the only really expensive and tricky feature Firebase solved IMHO). Single instance websocket servers are incredibly easy to build now-a-days

[devmount]

Wow, thanks for the quick and thorough reply! Great starting points for exploration. I'll come back to this after giving it some thought and having something more concrete to discuss.

[devmount]

Ok here are some thoughts open for discussion.

1. Architecture

1.1 Should it be database agnostic?

Yes, but a soft yes :) The backend should definitely have some kind of persistence layer / PDO / ORM (which should be pretty much standard today) and support a range of database systems to choose from or switch to. But being fully db agnostic is not necessarily needed.

1.2 Should it support a local-only mode?

Yes. To some extend this was already implemented with service workers in the past, but I got some caching issues (don't remember the details), so I dropped it. But this is definitely wanted.

1.3 Will there be extended administration capabilities?

Yes. This already has a frontend (currently calling the Firebase auth API which should be replaced). The frontend admin section can either be kept and just call endpoints from a new API, or be dropped in favor of a simple backend UI for all admin stuff. The latter would slim down the frontend.

1.4 Is any multi-tenancy suppose to be supported?

In the long run, this would be nice. This would enable us to provide SongDrive to different churches / orgs without them having the setup work. But this also comes with issues regarding licensing, payment etc. that would need to be solved.

1.5 And of course the mentioned real-time-ness of the intermediate layer

Yeah, tbh back in 2016, SongDrive was the project/playground for me to learn Vue.js together with Firebase which provided an amazing experience of reactivity. So having a completely reactive UI is still a result of me fanboying over realtime 😅 Today I would say, realtime is only needed for:

• List of setlists
• List of songs
• Synchronized setlist navigation
• Setlist modification, especially song key changes
• Song modification

2. Technology

Well, since this a single-dev-project the "team-skills" are pretty much limited to that one person 😅 I am most familiar with the LAMP stack. Let's see how a Laravel API would perform on the above noted architecture points:

• 1.1: It's not fully agnostic but it natively supports MariaDB, MySQL, PostgeSQL and SQLite
• (1.2: Not relevant for backend, this would be read only from a local cache)
• 1.3: Laravel Authentication
• 1.4: Laravel Tenancy
• 1.5: Laravel Reverb

Concerning horizontal scalability: I never thought that this would be a problem for the use cases of SongDrive. I mean it's a tool for musicians and there are only so much people that fit on a stage simultaneously 😂 So at most there would only be 10-15 devices using SongDrive at the same time. Or did you mean something else here?

---

Any concerns, doubts, fears, suggestions, ideas?