kymr / daily-study

2 stars 0 forks source link

[Dictation] Configure Varnish Cache with a NGINX web server #22

Closed kymr closed 6 years ago

kymr commented 6 years ago

Link

kymr commented 6 years ago

Hi of them So in this video, I want to show you how you can configure a varnishing caching server. So basically, I want doing I am running two of two six in a four service here. This is my cache serve here and this is my web server. This webserver is actually already producing web page. I will change this server port the moment This is the port 80 for http traffic. What you going to need to do is changing your web server port to another port other than 80. So we can do is edit the default config. So if nginx. this is the if you strap install. So this is the default file. So I am going to change the listen 80 to listen 8080. And I am going to delete the at the sixth line. I am going to reload nginx that. This is changed at the port nginx actually listen for http traffic. As you can see now it is updated the listen up. Um Okay. So What's we doing now? We can now work on getting varnish install. So flicking ever to my cache server. the Why caching work? just before we do install it. Essentially, the http web request would come from the user's browser. It would hit the caching server. And if the caching server knows the data which is requesting, for example if the request at post page or image pages no ma. And we knows what the data is you know asked before what cached, then will serve pass it to user. If not the request come in, the cache we'd look up. would we look up on a cache essentially. It going to go forward it to the webser, and then it will query and bring it back and send back to the user. So essentially within sending traffic from the web to the caching server to the web server and then back. So it is a kind of very top level way it works. Actually packet self doesn't reach to the web server cause it returns internally. But it is so of the way. Nonetheless, so we are going to do is install Varnish now. The default version in debian repository is a little bit older than what is available if you compile it from source. So if you do want latest that stable release in a lot higher than wasn't the repo so your choice. So we are going to do not install Varnish. Make sure you do not that the anyway. Just to get the release version of software. That's already installed. Now one thing to note. Both this servers on the same LAN. So the traffic is going over the LAN rather than WAN. So essentially. I will show you in a second. Both having 10 O ip addresses. So this one here 10.131.32.200 ip address. And the webserver has a 10.131.33.39 ip address. So I am just going to copy the internal ip of the webserver. And on the caching server, we can now edit a couple of files. Now because we are using systemd on this version of bam to server. We need to actually edit the systemd service script. So that can be found for using vim. I don't think of sugar vim installed. Oh I have cool. shes nah. This is a fresh install on a burn to edit digilotion oh it say. Would we show? Anyway we can go into /var/lib/system.d/ oh I sorry. /lib/system.d/system/ and then is varnish. service file. And in here we essentially change the port the here we listen on soft the a flag here rather than 6081. we will use full 80. And we save this. And I am going to use systemctl daemon-reload to apply previous change. So we became do now is configure the actual Varnish application itself. So /etc/varnish/default.vcl In here this host port have backend default. We just need to change this port. Now changing nginx port to 8080 isn't mandatory. So you could leave it 80. But I've of. That's the way I do anyway. So you just going to see 8080 on a. That's the actually webserver backend. Not the port 80 at your varnish server. So I get the internal IP. I paste it in here. We are going to save that enqute. And then we will do systemctl restart varnish. And now if we open up the private browser to the ip address of our public ip of our varnish server. As you can see if we do ifconfig. ngin 206. This is it here. So we are now getting the nginx page. http page. So there is a tool for statistics. You can do it with Varnish. One of them is Varnishstat. When you guide that up, you can see the most important more you look at is hit miss. So the cache hit is essentially you can see a total of 3 request had been made, and the cache is been used once. I find normally when using the cache it takes couple of times for it to really cache the data. It will seems above two. So we refresh this again. Why not load. Still one. Let's try it maybe a Here it yeah. It is going up to a cache hit is two. The connection is going to go up. Because of how http works creates it constant. That you http session is a constant steam. So that is going to go, that is going to keep going up for the connections. But actual cache hit so we refresh it few times now going to. I am trying to say is that you want to you don't want to cache hits. The cache hits is going up you know doing it does job. If the miss is going up higher than hit then something is not correctly configured. So Last I want to do really for this video, I will just show you how you cache use varnish cache. It is very very simple it's up just get it working. There is a tone of stuff you can do in the background to really configure Varnish. To make this is really really cool thing. So you can tell it What to cache and What not cache. How big you want to cache to be? How often you want to clear cause it is cache? So sometimes actually on the webserver, if we go to default intall location for the actual file of webserver. We edit this. Let's say Youtube. We should need to do reload on the webserver. Okay was time. I did it early in hard to actually restart the Varnish service for to delete it caching and recreated opt it was time. But it actually routing through to the webserver. So yeah that is pretty much it. How you can use Varnish cache to start caching haste your web pages?

kymr commented 6 years ago

hi everyone so in this video I want to

show you how you can configure a varnish

caching server so basically what I'm

doing is I'm running to a bun to 16:04

servers here so this is my cache server

and this is my web server this web

server is actually already producing a

web page and will change the server port

but at the moment is this in the on port

port 80 for HTTP traffic so what you're

going to need to do is change your web

server port to another port other than

80 so we're gonna do is I'm gonna edit

the e default config so for nginx this

is literally just a straight-up install

so especially the default file and I'm

going to change it from listen 82 listen

8080 and I'm delete the ipv6 line and I

reload in genetics there so let's just

changed the port that nginx essentially

listens for HTTP traffic as you can see

now it's updated the listener okay so

once you've done that we can now work on

getting varnish installed so flicking

over to my cache server the way the

caching works just before we do install

it is essentially the HTTP web request

would come from the users browser it

would hit the the caching server and if

the caching server knows the data that

it's requesting for example if is

requesting a post page or an image page

or something like that and it knows what

that data is because it's you know been

asked it before and it would have cached

it then it was so that's the user if not

the real quest will come in the cache

will do a lookup the cache Excel will do

a lookup on its cache essentially if you

can't find it then it afforded it to the

web server and then it will query the

website and then I'll bring it back and

then I'll send back send it back to the

user so essentially within sending

traffic from the web to the caching

server to the web server and then back

so it's kind of in a very top level way

it works

the actual packet itself doesn't reach

the webserver because it's all done

internally but it's sort of than that

way nonetheless so what we'll do is

we're install varnish now the default

version in Debian repositories is a

little bit older than what's available

to you in a comparable source so if you

do want the latest stable release and

it's a lot higher than what's in the the

repo so your choice

so we're going to do is do an app to

install varnish make sure you do an apt

update anyway just to get the latest

versions of the software so a little

install now one thing to note both these

servers are on the same LAN so the

traffic is going be going over a LAN

rather than a one so essentially I'll

show you in a second they both have

internal IP addresses so this one here

has a ten one thirty one thirty two

hundred IP address and the web server

has a ten one thirty one thirty thirty

nine IP address so I'm going to copy the

in term IP of the web server and on the

caching server we can now edit a couple

of files now because we're using the

system D on this version of Ubuntu

server we need to actually edit the

system D service scripts so that can be

found we use vim I don't think I've got

them installed oh yeah cool excuse that

this is a freshly installed on ubuntu so

a digitalocean I would say was it sure

anyway we can go into var

Lib system D system

sis DD oh sorry assistants I live system

the system and then it's the varnish but

service file and in here we have said

she just need to change the port that it

listens on so after the the a flag here

rather than six zero eight one will use

port 80 and will save this and then we

need to do a systemctl daemon reload to

apply those changes so what we can do

now is configure the natural varnish

application itself so an Etsy varnish

default dot V CL in here this host bit

here for the backend defaults we just

need to change this port now changing in

genetics port to 8080 isn't mandatory so

you could leave it at eighty but I've

just that's the way I do it anyway so

it's only one IC 8080 is the web stuff -

see the web server back-end not the the

port 80 of natural varnish server so I'm

gonna get the internal IP and I'm gonna

paste that in here and we're gonna save

that and quit it and then we'll do a

systemctl restart and varnish and now if

we open up the proper browser to the IP

address of our public IP of our varnish

server as you can see if you're doing it

big and in 206 this is it here so we're

now getting the nginx page HTTP page so

and there's some cool statistics that

you can do with varnish one of them is

varnish stats

and when you load that up you can see

the most important one your loo look at

is in this so the cache hit is

essentially you see a total of three

requests have been made and the cache

has been used once I found normally when

using the cache it takes a couple of

times for it to really cache the data it

always seems to be above two so if we

refresh this again when that loads still

one let's try maybe yeah there ya gone

up to cache hits to the connections are

going to go up because of how HTTP works

it creates a constant the actual HTTP

session is a constant stream so that's

going to be key that's going to keep

going up for the connections but the

actual cache hit yeah cuz I've refreshed

a few times it's now gone up but what

can I say is that you want to keep your

eye on the cache hit the cache it's

going up then you know it's doing its

job if the Miss is going up pile and the

hit then something's not correctly

configured so that's what I wanted to do

really for this video just show you how

you can actually use varnish cache is

very very simple to set up just to get

it working but there's a ton of stuff

that you can do in the background to

really configure varnish to make it do

some really really cool things that you

can tell it what to cache what not what

to cache want onto cache how big you

want the cache to be how often you want

it to clear it's because as cache so

sometimes actually on the web server if

we go to the default install location

for the actual file on the web server we

edit this obviously YouTube we shouldn't

need to do reload on the web server ok

waste time I did it earlier and I had to

actually restart the varnish service for

it to the LEAs cache and recreate it but

obviously it worked this time but you

know that just shows your day it's

actually rooting through to the web

server so yeah that's pretty much it

that's how you can

use vanish cash to start cashing HTML

web pages