No updates when belongs_to association is declared as index

[sillevl]

Hi,

I have two models, that have a one-to-many relation (belongs_to and has_many). Everything works fine if I don't declare an index on the belongs_to relation. Them moment I add the index declaration I have no more updates that are send to the client.

With the index declared, I get the following messages in the server console/log:

Finished "/cable/" [WebSocket] for 127.0.0.1 at 2016-11-04 12:20:28 +0100
Closing cursor: #<NoBrainer::Streams::ConcurrentAsyncHandler::AsyncQueryHandler:0x000000056a6b98 @handler=#<RethinkDB::CallbackHandler:0x000000056a6df0 @callback=#<Proc:0x000000055aaa50@/home/sille/.rvm/gems/ruby-2.3.0/gems/nobrainer_streams-0.2.0/lib/nobrainer_streams.rb:31 (lambda)>>, @msg=[152, [[41, [[15, ["weathers"]]], {"index"=>[73, [:id]]}]]], @all_opts={:db=>r.db("ttn_weather_dashboard_development")}, @token=1, @conn=#<RethinkDB::Connection:44913440 (localhost:28015) (Default DB: r.db("ttn_weather_dashboard_development")) (open)>, @opened=true, @closed=false>
Closing cursor: #<NoBrainer::Streams::ConcurrentAsyncHandler::AsyncQueryHandler:0x000000056bc678 @handler=#<RethinkDB::CallbackHandler:0x000000056bc8d0 @callback=#<Proc:0x000000056bfaf8@/home/sille/.rvm/gems/ruby-2.3.0/gems/nobrainer_streams-0.2.0/lib/nobrainer_streams.rb:31 (lambda)>>, @msg=[152, [[39, [[41, [[15, ["weathers"]]], {"index"=>[73, [:id]]}], [69, [[2, [r(3)]], [17, [[170, [[10, [3]], :sensor_id]], "41ovpOFKNAmyqt"]]]]]]]], @all_opts={:db=>r.db("ttn_weather_dashboard_development")}, @token=2, @conn=#<RethinkDB::Connection:44913440 (localhost:28015) (Default DB: r.db("ttn_weather_dashboard_development")) (open)>, @opened=true, @closed=false>

I am new to Rethinkdb and Nobrainer, so I cannot figure this out. Is this a bug? or am I using it wrong?

The models I declared in my application are:

class Weather
  include NoBrainer::Document
  include NoBrainer::Document::Timestamps

  field :temperature, :type => Float
  field :humidity, :type => Float
  field :pressure, :type => Float

  index :created_at

  belongs_to :sensor, required: true#, index: true
end

and

class Sensor
  include NoBrainer::Document
  include NoBrainer::Document::Timestamps

  field :name, :type => String
  field :deveui, :type => String, unique: true, index: true
  field :description, :type => Text

  has_many :weathers
end

My channel looks like this:

class SensorsChannel < ApplicationCable::Channel
  include NoBrainer::Streams

  def subscribed
    stream_from Sensor.find(params[:id]).weathers
  end

  def unsubscribed
    # Any cleanup needed when channel is unsubscribed
  end
end

[tinco]

Oh I'm not sure. Could you check in the RethinkDB UI if the index has been created? This gem is rather experimental and I know there's a bug where it crashes if a collection doesn't exist in the DB yet, maybe that also holds for indexes.

[sillevl]

The rake nobrainer:sync_schema returns this: [NoBrainer] Create index Weather.sensor_id and indexStatus() returns me this.

{
    "function": <binary, 96 bytes, "24 72 65 71 6c 5f..."> ,
    "geo": false ,
    "index": "sensor_id" ,
    "multi": false ,
    "outdated": false ,
    "query": "indexCreate('sensor_id', function(var1) { return var1("sensor_id"); })" ,
    "ready": true
}

[extem]

I encountered the same problem, but I solved it with index as lambda.

ex) index :sensor_id_index, ->(doc){ doc['sensor_id'] }

[tinco]

Heh, that's a coincidence, are you both working with sensor data in rethinkdb? What led you to RethinkDB and this Gem if you don't mind me asking @extem ?

[extem]

Thank you for your reply. I developed a real time medical and healthcare application using nobrainer_streams. It works very well.

There was a problem that did not work when using a secondary index in the condition of stream_from. I accidentally found that specifying an index with lambda works. Index names may be related. . .

RethinkDB + ActionCable + nobrainer_streams has good performance and I think it's a great idea. While this mechanism makes the system very simple, it can create very cool applications.

I hope RethinkDB + nobrainer + nobrainer_streams will evolve.

[tinco]

If I ported it to another database that has change feeds like mongodb, would you still choose it, or is there some other nice feature in RethinkDB that's really useful to you?

On Wed, Jul 24, 2019, 15:38 extem notifications@github.com wrote:

Thank you for your reply. I developed a real time medical and healthcare application using nobrainer_streams. It works very well.

There was a problem that did not work when using a secondary index in the condition of stream_from. I accidentally found that specifying an index with lambda works. Index names may be related. . .

RethinkDB + ActionCable + nobrainer_streams has good performance and I think it's a great idea. While this mechanism makes the system very simple, it can create very cool applications.

I hope RethinkDB + nobrainer + nobrainer_streams will evolve.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/tinco/nobrainer_streams/issues/1?email_source=notifications&email_token=AAAFQBDOMMR3N2PZ42LEQ6LQBBLLRA5CNFSM4CVFXVUKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2WLO6A#issuecomment-514635640, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAFQBAD7I4YW2ILBOTGUNTQBBLLRANCNFSM4CVFXVUA .

[extem]

Other databases will use connection pooling, but can you use a lot of ChangeFeeds like RethinkDB? My application creates ChangeFeeds with 5-6 stream_from in one session (client), but for my application, there are no performance issues with increasing number of clients.

In the case of mongodb(It is old information) https://stackoverflow.com/questions/48411897/severe-performance-drop-with-mongodb-change-streams

[extem]

nobrainer_stream is the best design for me. However, I am worried about the future RethinkDB.

Next time, I wanted to try to see if postgresql's listen notify and ActionCable can implement features like nobrainer_stream.

[extem]

I wrote something wrong about the index. RethinkDB cannot use order_by when using Changefeed. NoBrainer will automatically add an order(:id) if no order is specified.

Using order_by results in the following error internally: “RethinkDB: Cannot call changes on an eager stream”

It works if you specify without_ordering instead of order. Ex) stream_from ChatItem.where (chat_room_id: params ['chat_room_id']). without_ordering, include_initial: true, include_states: true (chat_room_id is an index)

I'm sorry, My English isn’t very good. Thank you.

[tinco]

Thanks! Does that mean the issue is resolved for you? Should I add some documentation somewhere? Or perhaps should I modify the code to automatically add without_ordering?

[extem]

Thank you for your reply.

Yes, my issue has been solved. Previously, queries involving secondary indexes did not work. Because you can use secondary indexes, you can use nobrainer_stream for tables with many records.

This issue only occurs when using a secondary index as a query. rethinkdb_monkeypatch.rb Add error handling to the callback (res) Then you can confirm the cause. The following error is returned but not logged.

{"t" => 18, "e" => 3000000, "r" => ["Unable to change eager stream calls." "]," b "=> [0]}

Thank you very much