search

RADAR-base / RADAR-RestApi

RESTful interface to access near real-time data
Apache License 2.0
2 stars 0 forks source link

Return Connected or Disconnected info about sources #48

Closed yatharthranjan closed 6 years ago

yatharthranjan commented 7 years ago

The sources information returned by the Rest API should also contain whether a source is currently connected or disconnected. A source is considered connected if its currently streaming data. This information is essential for dashboard.

yatharthranjan commented 7 years ago

Right now the best solution is to check the time of the last entry in the MongoDb because the dashboard needs this as soon as possible. Later we can look into the monitors, which already have disconnection information, and use them to write this info into a kafka topic which can then be read by mongoDb connector to store this info in MongoDb and then the Rest api can read it from MongoDb.

nivemaham commented 6 years ago

Looks like so. I will close the other

nivemaham commented 6 years ago

Currently, we get the earliest and latest time-stamps published from source-monitors from radar-backend, which effectively gives the value for effectiveTimeFrame for a source of a subject in a project. We can use the effectiveTimeFrame.endDateTime to determine whether a source is CONNECTED or DISCONNECTED or UNKNOWN maybe ( if we don't have any input on timeStart and timeEnd).

But, what is under question is on what condition we determine whether a source is connected or not?

Some of my thoughts are

  1. If theendDataTime is before predefined timeout e.g. (last 15 minutes/ 1 hours/ 1 day for example).
  2. Given that there is a huge variation of frequencies, can should may be configure what is the timeout for each source-type to determine whether a source of that kind is DISCONNECTED. (will require changes on MP and/or schemas)
  3. Calculate the time-out based on the least frequency of source-data assigned to a source-type. ( Would this make frequencies mandatory for each source-data)

What do you think @yatharthranjan @blootsvoets ? Let's discuss it here, so the decisions remain logged for future reference.