FirebaseExtended / firebase-queue

MIT License
786 stars 108 forks source link

Node.js strategy : firebase-queue vs building API #75

Closed lusa closed 7 years ago

lusa commented 7 years ago

Hello, I have a question on a general strategy of using Firebase on Node.js. I haven't seen many example applications of this aside from strategies involving firebase-queue. I want my application to scale when deploying to Google App Engine, which uses automatically scaling to add/remove CPUs as needed. I chose GAE because it sounded easy to deploy and scale Node.js apps. I am not a veteran node.js developer, hence opting for PaaS solutions.

My backend does not currently need to do any blocking actions, in fact it just needs to perform Firebase operations like updating or transactions on a database reference (at least I believe these are non-blocking). I've chosen to use the firebase-queue anyway, since I really like the ease of pushing "requests" onto a tasks queue and have firebase-queue automatically delegate a worker to processing this task, across all my Node app instances.

I've stress tested firebase-queue and noticed that the queue can't keep up with heavy load, such as 1000 tasks/second. I understand this is because of the limitation of the transaction with the queue. So now I'm thinking that using firebase-queue for all my requests is not as efficient as building an API to handle my operations on a firebase database, such as the example below: https://github.com/NickMoignard/node-firebase-restful/blob/master/server.js

I assume an API built like the above would route requests to only one instance of a Node application, which could then modify a Firebase reference. My assumption is this would be faster than a queue, at the tradeoff of holding up traffic if blocking requests happen (still assuming firebase operations are non-blocking). Is this correct? Is there a better strategy?

cbraynor commented 7 years ago

This kind of question is probably better suited to the firebase-talk Google group, where a wider audience will be able to chime in with their own experiences.

As for blocking/non-blocking in Javascript - it's a little more nuanced. Basically the Firebase SDK operations to the Realtime Database are asynchronous (they don't block) but you probably should wait to respond to a client until after you have confirmation that any writes have succeeded. Blind writes using something like push IDs to prevent collisions are quicker, of course, but transactions allow data to be read from a location if that's important to the application logic.