search

MassProspecting / docs

Public documentation, roadmap and issue tracker of MassProspecting
http://doc.massprospecting.com/
1 stars 0 forks source link

Glitch: Server resources leaking #334

Open leandrosardi opened 1 week ago

leandrosardi commented 1 week ago

I have a cloud VPS 5 rented to Contabo.

In such a server I run many instances of the same processes profile.rb, each profile running and automating a chrome browser.

Each process also perform API calls to a central server.

Also, there is a launcher process who connects a database to know which processes should be running.

The problem is that after a few hours of running, I cannot longer connect the server via SSH and processes start failing.

I installed New Relic and start monitoring servers.

I need an OS admin to find the root of the problem?

leandrosardi commented 18 hours ago

Exception Backtrace 1

Net::ReadTimeout with #
/usr/local/rvm/rubies/ruby-3.1.2/lib/ruby/3.1.0/net/protocol.rb:219:in `rbuf_fill'
/usr/local/rvm/rubies/ruby-3.1.2/lib/ruby/3.1.0/net/protocol.rb:193:in `readuntil'
/usr/local/rvm/rubies/ruby-3.1.2/lib/ruby/3.1.0/net/protocol.rb:203:in `readline'
/usr/local/rvm/gems/ruby-3.1.2/gems/net-http-0.2.2/lib/net/http/response.rb:42:in `read_status_line'
/usr/local/rvm/gems/ruby-3.1.2/gems/net-http-0.2.2/lib/net/http/response.rb:31:in `read_new'
/usr/local/rvm/gems/ruby-3.1.2/gems/net-http-0.2.2/lib/net/http.rb:1615:in `block in transport_request'
/usr/local/rvm/gems/ruby-3.1.2/gems/net-http-0.2.2/lib/net/http.rb:1606:in `catch'
/usr/local/rvm/gems/ruby-3.1.2/gems/net-http-0.2.2/lib/net/http.rb:1606:in `transport_request'
/usr/local/rvm/gems/ruby-3.1.2/gems/net-http-0.2.2/lib/net/http.rb:1579:in `request'
/usr/local/rvm/gems/ruby-3.1.2/gems/selenium-webdriver-4.10.0/lib/selenium/webdriver/remote/http/default.rb:119:in `response_for'
/usr/local/rvm/gems/ruby-3.1.2/gems/selenium-webdriver-4.10.0/lib/selenium/webdriver/remote/http/default.rb:77:in `request'
/usr/local/rvm/gems/ruby-3.1.2/gems/selenium-webdriver-4.10.0/lib/selenium/webdriver/remote/http/common.rb:59:in `call'
/usr/local/rvm/gems/ruby-3.1.2/gems/selenium-webdriver-4.10.0/lib/selenium/webdriver/remote/bridge.rb:620:in `execute'
/usr/local/rvm/gems/ruby-3.1.2/gems/selenium-webdriver-4.10.0/lib/selenium/webdriver/remote/bridge.rb:94:in `get'
/usr/local/rvm/gems/ruby-3.1.2/gems/selenium-webdriver-4.10.0/lib/selenium/webdriver/common/navigation.rb:32:in `to'
/usr/local/rvm/gems/ruby-3.1.2/gems/selenium-webdriver-4.10.0/lib/selenium/webdriver/common/driver.rb:135:in `get'
/home/blackstack/code1/sdk/lib/enrichment_types/ApolloRPA_Enrichment.rb:80:in `scrape'
/home/blackstack/code1/sdk/lib/enrichment_types/ApolloRPA_NameAndLinkedInUrlToEmail.rb:36:in `do'
/home/blackstack/code1/sdk/p/profile.rb:911:in `block in 
'
/home/blackstack/code1/sdk/p/profile.rb:903:in `each'
/home/blackstack/code1/sdk/p/profile.rb:903:in `
leandrosardi commented 18 hours ago

Exception Backtrace 2

Net::ReadTimeout with #
/usr/local/rvm/rubies/ruby-3.1.2/lib/ruby/3.1.0/net/protocol.rb:219:in `rbuf_fill'
/usr/local/rvm/rubies/ruby-3.1.2/lib/ruby/3.1.0/net/protocol.rb:193:in `readuntil'
/usr/local/rvm/rubies/ruby-3.1.2/lib/ruby/3.1.0/net/protocol.rb:203:in `readline'
/usr/local/rvm/gems/ruby-3.1.2/gems/net-http-0.2.2/lib/net/http/response.rb:42:in `read_status_line'
/usr/local/rvm/gems/ruby-3.1.2/gems/net-http-0.2.2/lib/net/http/response.rb:31:in `read_new'
/usr/local/rvm/gems/ruby-3.1.2/gems/net-http-0.2.2/lib/net/http.rb:1615:in `block in transport_request'
/usr/local/rvm/gems/ruby-3.1.2/gems/net-http-0.2.2/lib/net/http.rb:1606:in `catch'
/usr/local/rvm/gems/ruby-3.1.2/gems/net-http-0.2.2/lib/net/http.rb:1606:in `transport_request'
/usr/local/rvm/gems/ruby-3.1.2/gems/net-http-0.2.2/lib/net/http.rb:1579:in `request'
/usr/local/rvm/gems/ruby-3.1.2/gems/selenium-webdriver-4.10.0/lib/selenium/webdriver/remote/http/default.rb:119:in `response_for'
/usr/local/rvm/gems/ruby-3.1.2/gems/selenium-webdriver-4.10.0/lib/selenium/webdriver/remote/http/default.rb:77:in `request'
/usr/local/rvm/gems/ruby-3.1.2/gems/selenium-webdriver-4.10.0/lib/selenium/webdriver/remote/http/common.rb:59:in `call'
/usr/local/rvm/gems/ruby-3.1.2/gems/selenium-webdriver-4.10.0/lib/selenium/webdriver/remote/bridge.rb:620:in `execute'
/usr/local/rvm/gems/ruby-3.1.2/gems/selenium-webdriver-4.10.0/lib/selenium/webdriver/remote/bridge.rb:551:in `find_elements_by'
/usr/local/rvm/gems/ruby-3.1.2/gems/selenium-webdriver-4.10.0/lib/selenium/webdriver/common/search_context.rb:78:in `find_elements'
/home/blackstack/code1/sdk/lib/profile_rpa/apollo_rpa.rb:102:in `people_search_row_to_lead_hash'
/home/blackstack/code1/sdk/lib/enrichment_types/ApolloRPA_Enrichment.rb:124:in `block in scrape'
/home/blackstack/code1/sdk/lib/enrichment_types/ApolloRPA_Enrichment.rb:115:in `each'
/home/blackstack/code1/sdk/lib/enrichment_types/ApolloRPA_Enrichment.rb:115:in `scrape'
/home/blackstack/code1/sdk/lib/enrichment_types/ApolloRPA_NameAndLinkedInUrlToEmail.rb:36:in `do'
/home/blackstack/code1/sdk/p/profile.rb:911:in `block in 
'
/home/blackstack/code1/sdk/p/profile.rb:903:in `each'
/home/blackstack/code1/sdk/p/profile.rb:903:in `
'