derekpierre commented 2 years ago

Update monitor dependencies
Limit large monitor logging taking up much of the available storage space over a long enough period of time.
Remove InfluxDB dependency
- Replace existing use of InfluxDB for events, with just scraping for error events for the past 2 periods - the events are rare, and getting events is not a heavy operation.

codecov-commenter commented 2 years ago

Codecov Report

Merging #103 (ae4bc31) into main (34d67c6) will decrease coverage by 0.35%. The diff coverage is 56.41%.

@@            Coverage Diff             @@
##             main     #103      +/-   ##
==========================================
- Coverage   50.46%   50.10%   -0.36%     
==========================================
  Files          11       11              
  Lines        1072      922     -150     
==========================================
- Hits          541      462      -79     
+ Misses        531      460      -71

Impacted Files	Coverage Δ
monitor/cli/_utils.py	`36.84% <ø> (-3.16%)`	:arrow_down:
monitor/dashboard.py	`19.37% <16.66%> (-0.47%)`	:arrow_down:
monitor/crawler.py	`45.42% <42.85%> (+2.71%)`	:arrow_up:
monitor/charts.py	`16.90% <100.00%> (ø)`
monitor/cli/main.py	`54.43% <100.00%> (+0.40%)`	:arrow_up:
monitor/components.py	`27.72% <100.00%> (-1.41%)`	:arrow_down:
monitor/db.py	`98.07% <100.00%> (+2.37%)`	:arrow_up:
monitor/layout.py	`94.11% <100.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 34d67c6...ae4bc31. Read the comment docs.

derekpierre commented 2 years ago

@vepkenez, @KPrasch The status monitor is currently running this code, with one small modification to comment out the collection of events for the crawler - still working through that minor issue.

-        network_events = self.check_network_events()
+        network_events = [] #self.check_network_events()

It works locally, but not on the server for some reason. Remote Connection timeout.

In the meantime, the monitor is running this code with the nucypher version located here - https://github.com/derekpierre/nucypher/tree/ease-up-monitor-logs - which is basically v5.3.1 with one cherry-picked commit from main - https://github.com/derekpierre/nucypher/commit/8cc598ed4c2ae40a9ecdb2b588eb4424d2273445. This is causing the logs to fill up relatively quickly. The logging limitations will be adjusted once I get a sense for what size the logs are after a few days without the excessive logging from before.

Once 6.0 is released, which would include that change, we can remove that git-specific nucypher dependency.

derekpierre commented 2 years ago

Filed a follow-up issue #104 for the aforementioned events issue.

The monitor is now running the code as-is from this PR.

nucypher / nucypher-monitor

InfluxDB...It was time to say goodbye #103

Codecov Report