search

cloudsidedev / appside

Multitenant environment automation.
http://cloudside.ch
GNU Affero General Public License v3.0
38 stars 7 forks source link

Raise default Vagrant memory value to 4GB #31

Closed ivomarino closed 6 years ago

ivomarino commented 7 years ago

Considering the today most devs have machines with at least 8 or 16GB of physical RAM we could start considering to increase the default Vagrant memory value to 4GB, actually it's set to 2GB as defined in https://github.com/ttssdev/appflow/blob/master/Vagrantfile#L44.

Especially MySQL intensive tasks and queries could take advantage of more memory, actually some atlantis nodes crash during while executing some heavy queries, like reported by @hubeRsen and @grappler.

ocean90 commented 7 years ago

machines with at least 8 or 16GB of physical RAM we could start considering to increase the default Vagrant memory value to 4GB,

That might be true, but this VM isn't the only process which is running on our machines. Think about IDEs, Chrome, Slack, and maybe other VMs which are running parallel to the atlantis VM. IMO the memory value should only be increased if we know for sure that low memory is the real cause for this or if it's just a faulty setting somewhere.

ivomarino commented 7 years ago

makes sense to me, thanks @ocean90, so let's investigate what pushes the memory usage high.

ivomarino commented 7 years ago

We should run some benchmarks e.g. with https://github.com/wg/wrk for http and some MySQL bench/stress test tool.

ivomarino commented 7 years ago

For MySQL something like this could do https://www.howtoforge.com/how-to-benchmark-your-system-cpu-file-io-mysql-with-sysbench#-mysql-benchmark-

grappler commented 7 years ago

@ivomarino What is the progress on this as I need to restart atlantis at least twice a day and it takes time to get back as I get "503 Service Unavailable" in the first couple of minutes after reloading it.

ivomarino commented 7 years ago

@grappler so the main issue where missing images? is CPU load OK now?

grappler commented 7 years ago

The stability has improved a bit. I don't think the issue is the missing images. Here are log files from monit. This was from Jan 16th. https://required.slack.com/files/grapplerulrich/F3S11LUCD/pasted_image_at_2017_01_16_09_54_am.png https://required.slack.com/files/grapplerulrich/F3SJFH8NA/monit_logfile.txt

I can't increase the memory to 4GB as I have troubles now when Appflow takes a full 2GB.

I have implemented Dominik's htaccess fix for the images. Let see if that makes any difference.

ivomarino commented 7 years ago

@grappler allright, let's see, thanks for update;)

grappler commented 7 years ago

I am getting now 503 Service Unavailable.

image

ivomarino commented 7 years ago

@grappler uptime of the VM? ssh atlantis "uptime"

grappler commented 7 years ago

@ivomarino uptime, 3h 8m

ivomarino commented 7 years ago

let's check which service is not running:

ssh atlantis
sudo service apache2 status ; sudo service haproxy status ; ps aux | grep varnish

paste output, thanks.

grappler commented 7 years ago

Atlantis was unresponsive again today morning.

grappler commented 7 years ago

Atlantis was unresponsive again after 36 mins

The result from sudo service apache2 status ; sudo service haproxy status ; ps aux | grep varnish

 * apache2 is running
debug1: multiplexing control connection
debug1: channel 3: new [mux-control]
debug1: channel 4: new [client-session]
debug1: multiplexing control connection
debug1: channel 5: new [mux-control]
debug1: channel 6: new [client-session]
debug1: multiplexing control connection
debug1: channel 7: new [mux-control]
debug1: channel 8: new [client-session]
debug1: channel 5: free: mux-control, nchannels 9
debug1: client_input_channel_req: channel 6 rtype exit-status reply 0
debug1: client_input_channel_req: no sink for exit-status on channel 6
debug1: channel 6: free: client-session, nchannels 8
debug1: channel 3: free: mux-control, nchannels 7
debug1: client_input_channel_req: channel 4 rtype exit-status reply 0
debug1: client_input_channel_req: no sink for exit-status on channel 4
debug1: channel 4: free: client-session, nchannels 6
debug1: multiplexing control connection
debug1: channel 3: new [mux-control]
debug1: channel 4: new [client-session]
debug1: channel 3: free: mux-control, nchannels 7
debug1: client_input_channel_req: channel 4 rtype exit-status reply 0
debug1: client_input_channel_req: no sink for exit-status on channel 4
debug1: channel 4: free: client-session, nchannels 6
ivomarino commented 6 years ago

for now we remain on 2G which allows to run multiple VMs also on Macs with e.g. 8G RAM.