Varnish是一款高性能的开源HTTP加速器,Squid服务也是，两者的关系就像apache跟nginx，Varnish更年轻轻量,Squid更成熟稳重。

Varnish4.0工作工作流程如下图各个点说明vcl_recv：接收请求vcl_pass, 跳过缓存vcl_hit, 缓存命中vcl_miss, 缓存丢失vcl_pipe, 非httpvcl_purge, 手动清楚缓存为过期vcl_synth, 手动清楚缓存过期同步vcl_deliver 发送给客户端每个节点上的变量作用域如下图

内建变量：

1、req.*：request，表示由客户端发来的请求报文相关；

req.http.User-Agent, req.http.Referer, ...

2、bereq.*：由varnish发往BE主机的httpd请求相关；

3、 beresp.*：由BE主机响应给varnish的响应报文相关；

4、 resp.*：由varnish响应给client相关；

5、obj.*：存储在缓存空间中的缓存对象的属性；只读；

常用变量：

bereq.http.HEADERS

bereq.request：请求方法；

bereq.url：请求的url；

bereq.proto：请求的协议版本；

bereq.backend：指明要调用的后端主机；

req.http.Cookie：客户端的请求报文中Cookie首部的值；

req.http.User-Agent ~ "chrome"

//-------------------------------------

beresp.http.HEADERS

beresp.status：响应的状态码；

reresp.proto：协议版本；

beresp.backend.name：BE主机的主机名；

beresp.ttl：BE主机响应的内容的余下的可缓存时长；

//-------------------------------------

obj.hits：此对象从缓存中命中的次数；

obj.ttl：对象的ttl值

//-------------------------------------

server.ip

server.hostname

//-------------------------------------

client.ip

---

Varnish的特长体现在缓存命中vcl_hit，以及缓存清理vcl_purge方便。

简单使用说明

1、 安装

[root@node1 ~]# yum install varnish

2、 查看

[root@node1 ~]# rpm -ql varnish/etc/varnish   #配置文件目录/etc/varnish/default.vcl      #配置各Child/Cache线程的缓存策略；/etc/varnish/varnish.params   # 配置varnish服务进程的工作特性，例如监听的地址和端口，缓存机制；/run/varnish.pid/usr/bin/varnishadm  #客户端脚本/usr/bin/varnishtest  #测试工具程序：#Shared Memory Log交互工具：/usr/bin/varnishhist/usr/bin/varnishlog/usr/bin/varnishncsa/usr/bin/varnishstat/usr/bin/varnishtop/usr/sbin/varnishd  #主程序/usr/sbin/varnish_reload_vcl  #编译配置文件#启动脚本/usr/lib/systemd/system/varnish.service/usr/lib/systemd/system/varnishlog.service     #持久化日志  二选一/usr/lib/systemd/system/varnishncsa.service    #持久化日志  二选一

3、配置文件

[root@node1 varnish]# vim /etc/varnish/varnish.paramsRELOAD_VCL=1   #启动程序后能否reload配置文件VARNISH_VCL_CONF=/etc/varnish/default.vcl   #VARNISH_LISTEN_PORT=80 #默认6081 端口   如果直接当web服务器用改80# Admin interface listen address and port  管理端口地址VARNISH_ADMIN_LISTEN_ADDRESS=127.0.0.1VARNISH_ADMIN_LISTEN_PORT=6082VARNISH_SECRET_FILE=/etc/varnish/secret  #密钥#VARNISH_STORAGE="malloc,256M"  #缓存VARNISH_STORAGE="file,/www/data/varnish,1g"VARNISH_USER=varnishVARNISH_GROUP=varnish

[root@node1 data]# mkdir varnish/cache[root@node1 data]# chown -R varnish.varnish varnish

4、启动

[root@node1 varnish]# systemctl start varnish[root@node1 varnish]# ss -lntup|grep varnishtcp    LISTEN     0      128       *:80                    *:*                   users:(("varnishd",pid=1765,fd=7))tcp    LISTEN     0      10     127.0.0.1:6082                  *:*                   users:(("varnishd",pid=1764,fd=6))tcp    LISTEN     0      128      :::80                   :::*                   users:(("varnishd",pid=1765,fd=8))

5、缓存策略后台配置

[root@node1 varnish]# vim /etc/varnish/default.vclbackend default {    .host = "192.168.1.202";    .port = "80";}

6、重载

Loading vcl from /etc/varnish/default.vclCurrent running config name isUsing new config name reload_2018-06-06T09:17:17VCL compiled.VCL 'reload_2018-06-06T09:17:17' now activeavailable       0 bootactive          0 reload_2018-06-06T09:17:17Done

7、admin管理

[root@node1 varnish]# varnishadm -hvarnishadm: invalid option -- 'h'usage: varnishadm [-n ident] [-t timeout] [-S secretfile] -T [address]:port command [...]    -n is mutually exlusive with -S and -T[root@node1 varnish]# varnishadm -S /etc/varnish/secret -T 127.0.0.1:6082200-----------------------------Varnish Cache CLI 1.0-----------------------------Linux,3.10.0-693.el7.x86_64,x86_64,-sfile,-smalloc,-hcritbitvarnish-4.0.5 revision 07eff4c29Type 'help' for command list.Type 'quit' to close CLI session.varnish> vcl.list  #配置文件版本200available       0 bootactive          0 reload_2018-06-06T09:17:17#切换vcl配置  每次配置文件加载都会保存下来，在这里可以切换vcl.use boot200VCL 'boot' now activevcl.use reload_2018-06-06T09:17:17200VCL 'reload_2018-06-06T09:17:17' now active#查看默认配置选项vcl.show -v reload_2018-06-06T09:17:17#线程数量param.show thread_pools200thread_pools        Value is: 2 [pools] (default)        Default is: 2        Minimum is: 1param.show thread_pool_max   #min200thread_pool_max        Value is: 5000 [threads] (default)        Default is: 5000        Minimum is: 100#设置参数varnish> param.set thread_pools 4200#存储查看varnish> storage.list200Storage devices:    storage.Transient = malloc    storage.s0 = file#后端查看varnish> backend.list200Backend name                   Refs   Admin      Probedefault(127.0.0.1,,8080)       1      probe      Healthy (no probe)default(192.168.1.202,,80)     1      probe      Healthy (no probe)#--------------------------------------------------------------------------#非交互式[root@node1 varnish]# varnishadm -S /etc/varnish/secret -T 127.0.0.1:6082 statusChild in state running

规则编写

默认规则查看,很重要，因为varnish先读取客户端配置，在读取默认规则

[root@node1 varnish]# varnishadm -S /etc/varnish/secret -T 127.0.0.1:6082 vcl.listavailable       0 bootavailable       0 reload_2018-06-06T09:17:17available       0 reload_2018-06-06T10:43:58available       0 reload_2018-06-06T11:08:20available       0 reload_2018-06-06T12:52:47available       0 reload_2018-06-06T13:04:19active          0 reload_2018-06-06T13:06:08[root@node1 varnish]# varnishadm -S /etc/varnish/secret -T 127.0.0.1:6082 vcl.show -v reload_2018-06-06T13:06:08

1、不检查缓存

sub vcl_recv {        if (req.url ~ "(?i)^/(login|admin)") {                return(pass);        }}

2、合成错误码

sub vcl_recv {    if (req.http.User-Agent ~ "(?i)curl"){        return(synth(403));    }}marvindeMacBook-Pro:~ marvin$ curl http://192.168.1.200/admin.php              
Error 403 Forbidden
    
Forbidden
    
Guru Meditation:
    
XID: 196684
    
    
    
Varnish cache server
  

3、默认vcl_recv策略

sub vcl_recv {       if (req.method != "GET" &&       req.method != "HEAD" &&       req.method != "PUT" &&       req.method != "POST" &&       req.method != "TRACE" &&       req.method != "OPTIONS" &&       req.method != "DELETE") {                        /* Non-RFC2616 or CONNECT which is weird. */                        return (pipe);       }        if (req.method != "GET" && req.method != "HEAD") {            /* We only deal with GET and HEAD by default */            return (pass);        }        if (req.http.Authorization || req.http.Cookie) {            /* Not cacheable by default */            return (pass);        }            return (hash);}

4、指定资源去除cookie做缓存

sub vcl_recv {    if (req.url ~ "(?i)\.(jpg|jpge)$" ){                unset req.http.Cookie;                return(hash);    }}sub vcl_backend_response {    if (beresp.http.cache-control !~ "s-maxage") {        if (bereq.url ~ "(?i)\.(jpg|jpeg|png|gif|css|js)$") {            unset beresp.http.Set-Cookie;            set beresp.ttl = 3600s;        }    }}

5、客户端ip

vcl_recv： 如果多级代理需要加 ，默认是有X-Forwarded-For if (req.http.X-Forwarded-For){              set req.http.X-Forwarded-For = req.http.X-Forwarded-For + "," + client.ip;        }else {            set req.http.X-Forwarded-For = client.ip;        }

6、缓存修剪(实用)

#白名单设置  掩码写在外面acl purgers{#       "192.168.1.104";        "127.0.0.1"/8;}sub vcl_recv {    if(req.method == "PURGE"){        if (!client.ip ~ purgers){                return(synth(403));        }        return(purge);    }}#默认规则里面的行为    sub vcl_purge {    return (synth(200, "Purged"));}

以上三次请求，第一次删除缓存，第二次没有命中（添加缓存），第三次命中缓存

在交互模式下(用于临时清理)   这种方式很有用ban req.url ~ ^/javascript   #清空所有/javascript 开头的ban req.url ~ ^/            #清空所有#配置文件清空if (req.method == "BAN") {    ban("req.http.host == " + req.http.host + " && req.url == " + req.url);    # Throw a synthetic page so the request won't go to the backend.    return(synth(200, "Ban added"));}

7、绑定多个后台

import directors;#默认是backend defaultbackend websrv1 {    .host = "192.168.1.201";    .port = "80";}backend websrv2 {    .host = "192.168.1.202";    .port = "80";}sub vcl_init {    #轮询    new websrvs = directors.round_robin();    websrvs.add_backend(websrv1);    websrvs.add_backend(websrv2);    #权重    new websrvs3 = directors.random();    websrvs3.add_backend(websrv1,1);    websrvs3.add_backend(websrv2,2);}#可以区分出来 动态页面跟静态页面sub vcl_recv {       if (req.url ~ "(?i)\.php$") {                set req.backend_hint = websrv1;        } else {                set req.backend_hint = websrvs.backend();        }}    #---------------------------------------------------------#会话绑定sub vcl_init {    new h = directors.hash();    h.add_backend(one, 1);   // backend 'one' with weight '1'    h.add_backend(two, 1);   // backend 'two' with weight '1'}sub vcl_recv {    // pick a backend based on the cookie header of the client    set req.backend_hint = h.backend(req.http.cookie);}

8、健康状态监测

probe www_probe {    .url = "/index.html";    .timeout = 1s;    .interval = 4s;    .window = 5;   #探测5次 成功3次算在线    .threshold = 3;    .expected_response：期望的响应码，默认为200；}backend websrv1 {    .host = "192.168.1.201";    .port = "80";    .probe = www_probe;}backend websrv2 {    .host = "192.168.1.202";    .port = "80";    .probe = www_probe;}[root@node1 varnish]# varnishadm -S /etc/varnish/secret -T 127.0.0.1:6082 backend.listBackend name                   Refs   Admin      Probedefault(192.168.1.201,,80)     9      probe      Healthy (no probe)websrv1(192.168.1.201,,80)     5      probe      Healthy 5/5websrv2(192.168.1.202,,80)     5      probe      Healthy 5/5在交互模式下手动设置下线backend.set_health websrv Sickbackend.set_health websrv Healthy

服务进程配置

DAEMON_OPTS="-p thread_pool=3  thread_pool_min=5 -p thread_pool_max=500 -p thread_pool_timeout=300"thread_pool 2 线程池 小于等于cputhread_pool_min 100 每个线程池最小几个线程 默认100thread_pool_max 5000 最大几个线程  默认5000thread_pool_timeout 300.000 空闲线程的最长时常  超过thread_pool_min个数后 空闲的超过这个时间就会被杀死thread_queue_limit 20  线程池队列长度默认20  可以增加thread_pool_add_delay 0.000 [seconds]  增加不延迟thread_pool_destroy_delay  1 second 杀死延迟一秒thread_pool_fail_delay 0.200 [seconds] 创建失败时候，间隔多少时间在重新创建client：send_timeout 600.000   发送响应包给客户端timeout_idle 5     保持连接超时空闲时常  可调增加timeout_req  2      接收客户端请求报文首部  超过2秒放弃cli_timeout    admin 管理工具请求超时时常

backend BE_NAME {    ...    .connect_timeout = 3.5s;    .first_byte_timeout = 60s;    .between_bytes_timeout = 60s;}

log

1、varnishstat - Varnish Cache statistics

-1    -1 -f FILED_NAME     -l：可用于-f选项指定的字段名称列表；    MAIN.cache_hit     MAIN.cache_miss    # varnishstat -1 -f MAIN.cache_hit -f MAIN.cache_miss    # varnishstat -l -f MAIN -f MEMPOOL[root@node1 varnish]# varnishstat -1 #一次显示所有[root@node1 varnish]# varnishstat -1 -f MAIN.cache_hit -f MAIN.cache_missMAIN.cache_hit              63         0.00 Cache hitsMAIN.cache_miss             18         0.00 Cache misses

2、varnishtop - Varnish log entry ranking

-1     Instead of a continously updated display, print the statistics once and exit.    -i taglist，可以同时使用多个-i选项，也可以一个选项跟上多个标签；    -I <[taglist:]regex>    -x taglist：排除列表    -X  <[taglist:]regex>[root@node1 varnish]# varnishtop -i RespStatus  #压测 状态码

3、日志文件，一般不开启，一级代理采用

[root@node1 varnish]# systemctl start varnishncsa[root@node1 varnish]# tail /var/log/varnish/varnishncsa.log192.168.1.104 - - [06/Jun/2018:20:54:42 +0800] "GET http://192.168.1.200/robots.txt HTTP/1.1" 404 571 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.181 Safari/537.36"