Shiva not writing data to MySQL database

[ghost]

Hi there,

I see that Shiva is successfully writing data to the db every 120 mins. May I ask what the implications are of lowering the schedulertime so that data is written to the db, say, every 10 minutes?

# Duration (in minutes) to be passed to shivascheduler (integer value only)
schedulertime : 120

I can see the following information in the analyzer lamson.log file. I understand what the "dumps data into maindb" and "sends data on hpfeeds" actions do but what does the "resets global counter" action do please?

2016-10-11 17:54:55,034 - root - INFO - Shiva scheduler, which dumps data into maindb, resets global counter and sends data on hpfeeds, started at 2016-10-11 17:54:55.034385 and would execute every 120 minutes Thanks in advance for your help.

[b0nd]

Hi,

Personal engagements have kept me very busy for last couple of days; will get back to you in 2-3 days.

Cheers! Sumit On 12-Oct-2016 8:11 AM, "roxy9990" notifications@github.com wrote:

Is data written to the db only every 120 mins?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/shiva-spampot/shiva/issues/20#issuecomment-253015136, or mute the thread https://github.com/notifications/unsubscribe-auth/AD-hlYKKL3i38rok2O_bGssJ_vxwXD-eks5qy99ggaJpZM4KT_bq .

[RahulBinjve-zz]

Hi,

Yes, the data is written after 120 minutes / or whatever time you mention in the config file. To check if there's any issue, might I ask you to change the schedulertime variable in shiva.conf to a small value, such as 5 or 10 and restart the analyzer. This would ask SHIVA to push data to DB every 5/10 minutes.

Let us know if you find any issues after that.

Thank you.

On Wed, Oct 12, 2016 at 1:41 AM, b0nd notifications@github.com wrote:

Hi,

Personal engagements have kept me very busy for last couple of days; will get back to you in 2-3 days.

Cheers! Sumit On 12-Oct-2016 8:11 AM, "roxy9990" notifications@github.com wrote:

Is data written to the db only every 120 mins?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <https://github.com/shiva-spampot/shiva/issues/20#issuecomment-253015136 , or mute the thread https://github.com/notifications/unsubscribe- auth/AD-hlYKKL3i38rok2O_bGssJ_vxwXD-eks5qy99ggaJpZM4KT_bq .

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/shiva-spampot/shiva/issues/20#issuecomment-253031254, or mute the thread https://github.com/notifications/unsubscribe-auth/ACvYlavoQ7EE8cNeLSEKdGytRD6E-NyDks5qy-1pgaJpZM4KT_bq .

Regards, Rahul Binjve

Twitter: @c0dist

[c0dist]

May I ask what the implications are of lowering the schedulertime so that data is written to the db, say, every 10 minutes?

SHIVA, by design, uses two databases: ShivaTemp and Shiva. There's a separate script shivamaindb.py, that fetches data from temporary database (ShivaTemp) and then co-relates that data with the data in main database (Shiva). Therefore, the schedulertime time is there to give time to shivamaindb.py to complete it's task.

We would not recommend it keeping lower than, say, 20-30 minutes. Depending upon the number of records you have in your main database, it would be safe to give the script some time to finish.

Also note that the main db script waits for a little more than 3 minutes when executed to let hpfeeds script do its job. Ref - https://github.com/shiva-spampot/shiva/blob/master/analyzer/core/shivamaindb.py#L582

Thank you.

[ghost]

Thank you for your response

On Oct 11, 2016, at 10:29 PM, c0dist notifications@github.com wrote:

Oh it seems like I replied to unedited comment.

May I ask what the implications are of lowering the schedulertime so that data is written to the db, say, every 10 minutes?

SHIVA, by design, uses two databases: ShivaTemp and Shiva. There's a separate script shivamaindb.py, that fetches data from temporary database (ShivaTemp) and then co-relates that data with the data in main database (Shiva). Therefore, the schedulertime time is there to give time to shivamaindb.py to complete it's task.

We would not recommend it keeping lower than, say, 20-30 minutes. Depending upon the number of records you have in your main database, it would be safe to give the script some time to finish.

Also note that the main db script waits for a little more than 3 minutes when executed to let hpfeeds script do its job. Ref - https://github.com/shiva-spampot/shiva/blob/master/analyzer/core/shivamaindb.py#L582

Thank you.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.