LiveQuery for real time messaging - Discussion and Thoughts and Questions

[ranhsd]

Hi,

In my app i currently use LiveQuery for the "real time" parts. Currently it serve the following:

1. User Presence - when a user go online/offline
2. User profile update - when a user update his/her profile all of the user friends are being updated in real time
3. Friend request accepted - When a pending friend request was accepted on the other side and the current user is online then it will get notified
4. Other in-app notifications

Now I have another part in my app there i let users to chat in realtime with each other. Currently only one user can chat with only another user but in the future we will also add groups so i design my MongoDB schema to support both cases. Currently i am implementing the real time chat via LiveQuery and it works fine. The big advantage to use LiveQuery is pretty straightforward... it allows me to store all the info in MongoDB, support media out of the box (images, videos etc.) and not count on other 3-party solution that i start to investigate (e.g. firebase, pubnub etc.)

The question that i wanted to ask if you think that it's safe to go with LiveQuery for my real time chat or should i go with another 3-party provider? I am a huge fan of parse-server and i really like the LiveQuery concept but i am wondering if it is good for real time messaging.

to summarize this, my questions are (based on my current tech stack (see below)):

1. Scalability - can i serve hundred of thousands of connections with this solution?
2. Performance - What about the performance? I want to the message will be sent to other side as fast as possible (because we are talking about "real time messaging")
3. Flexibility - If i want to enrich my solution with more capabilities like: user is typing, user read the message, last seen and more...

My Tech Stack

1. Docker image for Parse Server (currently 2 replicas)
2. Docker image for Parse Live Query server (currently 3 replicas)
3. Docker image for Parse Dashboard (1 replica)
4. NGINX for load balancing 5.. Redis server for pub/sub and other key-value queries
5. Docker image for MongoDB with 3 replicas
6. Google cloud storage - for hosting images, videos and other binary content
7. All images are running inside a kubernetes cluster which deployed on GKE (google container engine)
8. My cluster is auto scaled means that i can increase or decrease the number of nodes and i can also increase my replicas dynamically for parse server or live query servers.

Thanks in advanced for your input!

[dpoetzsch]

I am implementing a similar thing at the moment.

It does work fine with live query for now. However, I can't tell you about scalability as we haven't scaled so much yet. I'd be also interested in experiences here.

We easily implemented stuff like "read" and "last seen" in our model. I was quite happy about the flexibility. Performance-wise we couldn't complain so far.

Still, be careful on how to design your data model: Live Query currently only supports simple, non-relational queries as discussed in issue #2946. This was a small issue for me when trying to have group chats as I didn't want to copy all receivers into each message at first.

[ranhsd]

Hi @dpoetzsch , thanks for your response.. Where your parse-server and parse-livequery servers are deployed ?

[dpoetzsch]

We currently work with heroku but are only in a closed beta phase so far.

[ideastouch]

Hi, I'm also using LiveQuery and Parse-Server both together at same Heroku app. I have also still running Parse.com so I have a daemon on Heroku that search for changes done through Parse.com server and then I send directly to LiveQueryServer at heroku a _onAfterSave message. About the kind of queries

// Chat is a PFObject extender object.
var query = Chat.query()
// row customerUser is pointer to CustomerUser
// CustomerUser is _User extender object.
query.whereKey("customerUser", equalTo: customerUser);
query.whereKey("readByCustomer", equalTo: false) ;
// Appointment is a PFObject extender object.
var queryAppointment = Appointment.query();
// row date is a Date Object.
queryAppointment.whereKey("date", greaterThan: new Date());
queryAppointment.whereKey("cancelled", notEqualTo: true)
// row appointment is pointer to Appointment.
query.whereKey("appointment", matchesQuery: queryAppointment)

BTW: I also use a modificated ParseLiveQuery Dictionary::init, see my comments at https://github.com/ParsePlatform/ParseLiveQuery-iOS-OSX/issues/40#issuecomment-255806307

[malhal]

I was wondering about the issue around missing object changes, i.e. it likely depends on the order of subscribe and find.

[faizsameerahmed96]

The best way according to me is use an established real time service. You can create your own easily through Socket.io or use an existing service like firebase.

[blainefarr]

@ranhsd @dpoetzsch I'm interested in this as well. We're just now evaluating Live Query vs a 3rd party tool for chat and group chat / scalability is the big question. What did you end up using?

How difficult has it been to build chat in with Live Query so far? We may go down that path for MVP and migrate later as need dictates.

[ranhsd]

Hi @blainefarr - I think that the LiveQuery is great for "real time database" scenarios where you want to get an update about the status of one or more objects ... it is also possible to use LiveQuery for real time messaging scenario (like i did) but it will require a bit more work on your side. Think about all the features that you have in a simple chat application like:

• User Presence - in order to track on user presence (online/offline) i added a boolean field to my user class. From the client i subscribe to changes in this field and then i handle the result in my client app via KVO. But it's not really "true" because in order to know if your user is really offline or online you need to work on the Installation class and check that all the installations of specific users are offline so for now i assume that each user will use one device.

• Chat rooms - I created a class which called ChatRoom. This collection contains all the list of chat rooms. Each entity contains an array of participants which is another class. I need to create another class for each participant because i want to save additional metadata for each one of the participant like: last seen etc.

• Count number of unread messages - this feature seems to be straight forward but it's not... I can easily implement this feature in a not efficient way but currently i am still struggling how to do it. I was thinking about using count of number of messages since data but it's not really helpful because the count api return only a number but i need this count for each one of the chat rooms that i have and iterating on the chat rooms and calling to count api is not efficient ... The best solution that i am thinking to do is to execute a query on the ChatMessage class with and select only the objectIds and in the client sort it and see how many records i have for each one of the ChatRooms. I can do it because i store a reference of ChatRoom inside the ChatMessage class.

If you have more ideas it will be great if you can share them here...

[davidruisinger]

Hi all, I'm currently building a small chat feature within our app using the LiveQuery implementation. My main concerns are all security related. Besides the fact, that LiveQueries (at least for me) do not respect class level permissions (see: https://github.com/ParsePlatform/parse-server/issues/3427) I'm still not sure how to secure my Messages.

I basically use 2 parse classes for my implementation:

• Chat: -- participants => array of _User pointers
• Message: -- author => pointer to _User -- chat => pointer to _Chat -- content => string with the message

I don't have a problem with my Chat class beeing publicly readable since it only show some user ids. So I focused on securing the Message class.

Initial attempt I initially thought I could use class level permissions and restrict read access to the author. Even if those class level permissions would be respected by LiveQueries, this wouldn't work since that would mean the user would only subscribe to messages he wrote himself.

Current solution I think it might work to add a beforeSafe function in Cloud code to set some custom ACL which would set read access to all participants of the chat. When a new user enters the chat, he would NOT have access to previous/old messages but could read along all messages written from now on (which is quite similar to common apps like whatsapp)

Do you guys have any other idea or tips on how to secure a chat application running on parse?

[ranhsd]

Hi @flavordaaave What about create ACL's (for read/write) only for the list of participants that you have under the Chat object?

[Samigos]

Hi everyone!

I'm in the same place as you were! I have a Parse Server where I run my app and store my db and I have another one for LiveQuery...

That's where I'm stuck... I've seen Parse's documentation, but it's not much explanatory! Can someone please tell me why do I need redis and how can I use it?

[flovilmart]

@Samigos what's not clear about the redis usage? Consider opening an issue on the docs repository if the notes about scaling LiveQuery are not clear.

[Samigos]

@flovilmart Thanks for responding! It says how to connect to a redis IP, but it doesn't clarify what we need to do redis-side... (I don't mean the exact steps, but what needs to be done)

For people like me, that hadn't heard redis before, it's a bit tough!

[flovilmart]

There's nothing to do redis-side, it's a Key-Value store, just follow any documentation about redis to start it up

[Samigos]

That's great! So if I got it right, I need to deploy somewhere the redis-server, correct? If so, is it a good idea to install it on the server where my Parse Server is deployed, or should I do it on a separate one?

Thanks in advance!

[Samigos]

Sorry if here's not the place to ask these questions, but just now things started clearing up in my head, about how to proceed.

[ranhsd]

@Samigos - If you are planning to run it in production then the best will be to run it on a separate service. By service i refer to either: server, container, etc.

[THEDOWNCOUNTRY]

Hello guys,

I am using the latest JS client to connect to Parse-Server LiveQuery I notice that when I close my app there is a disconnect log that is triggered I need some help to use that event to updated the user account status... Any input or code snippets are welcome. Thanks in advance.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Etto91]

someone can help me? https://github.com/parse-community/parse-server/issues/1906#issuecomment-616805834

[maxiqsoft]

I found a solution.

I've added one line of code to the file /node_modules/parse-server/lib/LiveQuery/ParseLiveQueryServer.js

The line is sessionToken: client.sessionToken, and in my case in line 459.

With that you are able to recognize who disconnected from the websocket by the users sessionToken.

Before it was just possible to get the sessionToken when the user connected.

With that you can make a function to get the user object from the Session class and save it maybe in an array.

This is where I added the line:

(0, _triggers.runLiveQueryEventHandlers)({
        event: 'ws_disconnect',
        clients: this.clients.size,
        sessionToken: client.sessionToken,
        subscriptions: this.subscriptions.size,
        useMasterKey: client.hasMasterKey,
        installationId: client.installationId
      });