Keepalived+LVS 高可用负载均衡集群部署实战

一、背景与概述

1.1 为什么需要高可用负载均衡

在生产环境中，单点故障是系统稳定性的最大威胁。当流量增长到单台服务器无法承载时，我们需要：

  • 负载均衡：将流量分散到多台后端服务器，提高整体处理能力
  • 高可用：当主节点故障时，自动切换到备用节点，保证服务连续性
  • 健康检查：自动检测后端服务器状态，剔除故障节点

Keepalived+LVS 组合提供了成熟的四层负载均衡高可用解决方案，广泛应用于互联网企业的生产环境。

1.2 技术选型对比

方案 层级 性能 高可用 配置复杂度 适用场景
Keepalived+LVS L4 ⭐⭐⭐⭐⭐ 内置 中 大流量、高性能要求
Nginx+Keepalived L7 ⭐⭐⭐⭐ 需配置 低 HTTP/HTTPS 流量
HAProxy+Keepalived L4/L7 ⭐⭐⭐⭐ 需配置 中 通用负载均衡
F5 BIG-IP L4/L7 ⭐⭐⭐⭐⭐ 内置 低 企业级、预算充足
Cloud LB (ALB/NLB) L4/L7 ⭐⭐⭐⭐ 内置 极低 云原生环境

1.3 架构设计

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
                     ┌─────────────────┐
                     │     用户请求     │
                     └────────┬────────┘
                              │
               ┌──────────────┴──────────────┐
               │        Virtual IP (VIP)     │
               │         192.168.1.100       │
               └──────────────┬──────────────┘
                              │
         ┌────────────────────┼────────────────────┐
         │                    │                    │
         ▼                    ▼                    ▼
┌─────────────────┐  ┌─────────────────┐  ┌─────────────────┐
│   Master Node   │  │  Backup Node 1  │  │  Backup Node 2  │
│  (Keepalived)   │  │  (Keepalived)   │  │  (Keepalived)   │
│   192.168.1.10  │  │   192.168.1.11  │  │   192.168.1.12  │
│   + LVS (DR)    │  │   + LVS (DR)    │  │   + LVS (DR)    │
└────────┬────────┘  └────────┬────────┘  └────────┬────────┘
         │                    │                    │
         └────────────────────┼────────────────────┘
                              │
         ┌────────────────────┼────────────────────┐
         │                    │                    │
         ▼                    ▼                    ▼
┌─────────────────┐  ┌─────────────────┐  ┌─────────────────┐
│  Real Server 1  │  │  Real Server 2  │  │  Real Server 3  │
│   192.168.2.10  │  │   192.168.2.11  │  │   192.168.2.12  │
│   (Web/App)     │  │   (Web/App)     │  │   (Web/App)     │
└─────────────────┘  └─────────────────┘  └─────────────────┘

1.4 LVS 工作模式

模式 全称 特点 适用场景
NAT Network Address Translation 请求和响应都经过 LVS，可跨网段 测试环境、小规模部署
DR Direct Routing 响应直接返回客户端，性能最高 生产环境推荐
TUN IP Tunneling 通过 IP 隧道转发，可跨机房 异地多活、跨数据中心

本文以 DR 模式 为例进行部署，这是生产环境最常用的模式。

二、环境准备

2.1 服务器规划

角色 主机名 IP 地址 系统 说明
Master lb-master 192.168.1.10 CentOS 7.9 主负载均衡器
Backup lb-backup1 192.168.1.11 CentOS 7.9 备负载均衡器
Backup lb-backup2 192.168.1.12 CentOS 7.9 备负载均衡器
Real Server web01 192.168.2.10 CentOS 7.9 后端 Web 服务器
Real Server web02 192.168.2.11 CentOS 7.9 后端 Web 服务器
Real Server web03 192.168.2.12 CentOS 7.9 后端 Web 服务器
VIP - 192.168.1.100 - 虚拟 IP

2.2 网络要求

  • LVS 节点与 Real Server 必须在同一二层网络（DR 模式要求）
  • 所有服务器之间网络互通
  • 关闭防火墙或配置相应规则
  • 禁用 SELinux 或配置相应策略

2.3 系统初始化

在所有服务器上执行：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
# 关闭防火墙
systemctl stop firewalld
systemctl disable firewalld

# 关闭 SELinux
setenforce 0
sed -i 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config

# 配置内核参数 (所有服务器)
cat >> /etc/sysctl.conf << EOF
# 允许 IP 转发
net.ipv4.ip_forward = 1

# 忽略 ICMP 重定向
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0

# ARP 相关 (Real Server 需要)
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.default.arp_ignore = 1
net.ipv4.conf.default.arp_announce = 2

# 增加连接跟踪表大小
net.netfilter.nf_conntrack_max = 1000000
net.nf_conntrack_max = 1000000
EOF

# 应用配置
sysctl -p

# 安装基础工具
yum install -y vim net-tools telnet curl wget

三、部署 LVS+Keepalived

3.1 安装软件包

在 LVS 节点（lb-master, lb-backup1, lb-backup2）上执行：

1
2
3
4
5
6
# 安装 ipvsadm 和 keepalived
yum install -y ipvsadm keepalived

# 验证安装
ipvsadm --version
keepalived --version

3.2 配置 Keepalived

3.2.1 Master 节点配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
cat > /etc/keepalived/keepalived.conf << 'EOF'
! Configuration File for keepalived

global_defs {
   # 通知邮箱 (可选)
   notification_email {
     ops@example.com
   }
   notification_email_from {
     keepalived@example.com
   }
   smtp_server smtp.example.com
   smtp_connect_timeout 30
   
   # 路由器标识，同一 VRRP 组必须相同
   router_id LVS_MASTER
   
   # 启用 VRRPv3 (支持 IPv6)
   vrrp_version 3
   
   # 脚本执行间隔
   vrrp_script_chk interval 2
}

# 健康检查脚本
vrrp_script chk_http_port {
    script "/etc/keepalived/scripts/check_lvs_health.sh"
    interval 2
    weight -20
    fall 3
    rise 2
}

# VRRP 实例定义
vrrp_instance VI_1 {
    # MASTER 或 BACKUP
    state MASTER
    
    # 网络接口
    interface eth0
    
    # 虚拟路由器 ID，同一 VRRP 组必须相同 (0-255)
    virtual_router_id 51
    
    # 优先级，MASTER 应该更高 (1-255)
    priority 100
    
    # VRRP 通告间隔 (秒)
    advert_int 1
    
    # 认证配置
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    
    # 虚拟 IP 地址 (可配置多个)
    virtual_ipaddress {
        192.168.1.100/24 dev eth0 label eth0:0
        192.168.1.101/24 dev eth0 label eth0:1
    }
    
    # 健康检查脚本引用
    track_script {
        chk_http_port
    }
    
    # 状态切换通知脚本
    notify_master "/etc/keepalived/scripts/notify_master.sh"
    notify_backup "/etc/keepalived/scripts/notify_backup.sh"
    notify_fault "/etc/keepalived/scripts/notify_fault.sh"
    
    # VIP 添加/删除脚本
    notify "/etc/keepalived/scripts/notify_vip.sh"
}

# LVS 配置
virtual_server 192.168.1.100 80 {
    # 调度算法
    # rr: 轮询
    # wrr: 加权轮询
    # lc: 最少连接
    # wlc: 加权最少连接
    # lblc: 基于局部性最少连接
    # lblcr: 带复制的基于局部性最少连接
    # dh: 目标地址哈希
    # sh: 源地址哈希
    # sed: 最少预期延迟
    # nq: 永不排队
    lb_algo wlc
    
    # LVS 模式
    # DR: 直接路由 (推荐)
    # NAT: 网络地址转换
    # TUN: IP 隧道
    lb_kind DR
    
    # 持久连接超时时间 (秒)
    # 同一客户端 IP 在超时时间内始终转发到同一 Real Server
    persistence_timeout 60
    
    # 协议
    protocol TCP
    
    # 健康检查配置
    delay_loop 6
    latency_threshold 3000
    protocol TCP
    
    # 后端服务器配置
    real_server 192.168.2.10 80 {
        weight 1
        # 健康检查类型
        HTTP_GET {
            url {
                path /health
                status_code 200
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
        # 状态变化通知脚本
        notify_down "/etc/keepalived/scripts/real_server_down.sh 192.168.2.10"
        notify_up "/etc/keepalived/scripts/real_server_up.sh 192.168.2.10"
    }
    
    real_server 192.168.2.11 80 {
        weight 1
        HTTP_GET {
            url {
                path /health
                status_code 200
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
        notify_down "/etc/keepalived/scripts/real_server_down.sh 192.168.2.11"
        notify_up "/etc/keepalived/scripts/real_server_up.sh 192.168.2.11"
    }
    
    real_server 192.168.2.12 80 {
        weight 1
        HTTP_GET {
            url {
                path /health
                status_code 200
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
        notify_down "/etc/keepalived/scripts/real_server_down.sh 192.168.2.12"
        notify_up "/etc/keepalived/scripts/real_server_up.sh 192.168.2.12"
    }
}

# HTTPS 服务配置 (可选)
virtual_server 192.168.1.100 443 {
    lb_algo wlc
    lb_kind DR
    persistence_timeout 3600
    protocol TCP
    
    delay_loop 6
    latency_threshold 3000
    protocol TCP
    
    real_server 192.168.2.10 443 {
        weight 1
        HTTP_GET {
            url {
                path /health
                status_code 200
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
    
    real_server 192.168.2.11 443 {
        weight 1
        HTTP_GET {
            url {
                path /health
                status_code 200
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
    
    real_server 192.168.2.12 443 {
        weight 1
        HTTP_GET {
            url {
                path /health
                status_code 200
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}
EOF

3.2.2 Backup 节点配置

Backup 节点配置与 Master 基本相同，主要修改以下内容：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
# 在 lb-backup1 和 lb-backup2 上
cat > /etc/keepalived/keepalived.conf << 'EOF'
! Configuration File for keepalived

global_defs {
   router_id LVS_BACKUP1  # 每个节点唯一
   vrrp_version 3
}

vrrp_script chk_http_port {
    script "/etc/keepalived/scripts/check_lvs_health.sh"
    interval 2
    weight -20
    fall 3
    rise 2
}

vrrp_instance VI_1 {
    state BACKUP           # 改为 BACKUP
    interface eth0
    virtual_router_id 51   # 与 Master 相同
    priority 90            # 低于 Master (如果有多个 Backup，依次递减)
    advert_int 1
    
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    
    virtual_ipaddress {
        192.168.1.100/24 dev eth0 label eth0:0
        192.168.1.101/24 dev eth0 label eth0:1
    }
    
    track_script {
        chk_http_port
    }
    
    notify_master "/etc/keepalived/scripts/notify_master.sh"
    notify_backup "/etc/keepalived/scripts/notify_backup.sh"
    notify_fault "/etc/keepalived/scripts/notify_fault.sh"
    notify "/etc/keepalived/scripts/notify_vip.sh"
}

# virtual_server 配置与 Master 相同
virtual_server 192.168.1.100 80 {
    lb_algo wlc
    lb_kind DR
    persistence_timeout 60
    protocol TCP
    
    delay_loop 6
    latency_threshold 3000
    protocol TCP
    
    real_server 192.168.2.10 80 {
        weight 1
        HTTP_GET {
            url {
                path /health
                status_code 200
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
    
    real_server 192.168.2.11 80 {
        weight 1
        HTTP_GET {
            url {
                path /health
                status_code 200
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
    
    real_server 192.168.2.12 80 {
        weight 1
        HTTP_GET {
            url {
                path /health
                status_code 200
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}
EOF

3.3 配置 Real Server

在 Real Server（web01, web02, web03）上执行：

3.3.1 配置 VIP 绑定脚本

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
cat > /etc/keepalived/scripts/lvs_real_server.sh << 'EOF'
#!/bin/bash

# VIP 地址
VIP=192.168.1.100

# 绑定 VIP 到 lo 接口
/sbin/ifconfig lo:0 $VIP netmask 255.255.255.255 broadcast $VIP up

# 配置 ARP 参数
/sbin/route add -host $VIP dev lo:0
echo "1" > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" > /proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" > /proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" > /proc/sys/net/ipv4/conf/all/arp_announce

# 持久化配置 (可选)
sysctl -p

echo "Real Server VIP configuration completed."
EOF

chmod +x /etc/keepalived/scripts/lvs_real_server.sh

3.3.2 配置开机启动

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
# 创建 systemd 服务
cat > /etc/systemd/system/lvs-real-server.service << 'EOF'
[Unit]
Description=LVS Real Server VIP Configuration
After=network.target

[Service]
Type=oneshot
ExecStart=/etc/keepalived/scripts/lvs_real_server.sh
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target
EOF

# 启用服务
systemctl daemon-reload
systemctl enable lvs-real-server
systemctl start lvs-real-server

# 验证配置
ip addr show lo:0

3.3.3 配置健康检查端点

在 Web 服务器上配置健康检查端点：

Nginx 示例：

1
2
3
4
5
6
7
8
9
10
11
# /etc/nginx/conf.d/health.conf
server {
    listen 80;
    server_name _;
    
    location /health {
        access_log off;
        return 200 "OK\n";
        add_header Content-Type text/plain;
    }
}

Tomcat 示例：

1
2
3
4
5
6
7
8
9
<!-- web.xml -->
<servlet>
    <servlet-name>HealthCheck</servlet-name>
    <servlet-class>com.example.HealthCheckServlet</servlet-class>
</servlet>
<servlet-mapping>
    <servlet-name>HealthCheck</servlet-name>
    <url-pattern>/health</url-pattern>
</servlet-mapping>

3.4 配置通知脚本

在 LVS 节点上创建通知脚本目录和脚本：

1
mkdir -p /etc/keepalived/scripts

3.4.1 健康检查脚本

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
cat > /etc/keepalived/scripts/check_lvs_health.sh << 'EOF'
#!/bin/bash

# 检查 LVS 服务状态
if ! systemctl is-active --quiet ipvsadm; then
    exit 1
fi

# 检查 VIP 是否存在
if ! ip addr show | grep -q "192.168.1.100"; then
    exit 1
fi

# 检查至少有一个 Real Server 可用
if ! ipvsadm -Ln | grep -q "192.168.2"; then
    exit 1
fi

exit 0
EOF

chmod +x /etc/keepalived/scripts/check_lvs_health.sh

3.4.2 状态切换通知脚本

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
# notify_master.sh - 成为 Master 时执行
cat > /etc/keepalived/scripts/notify_master.sh << 'EOF'
#!/bin/bash

logger "Keepalived: This node became MASTER"

# 发送通知邮件 (需要配置 mailx)
echo "Keepalived: $(hostname) became MASTER at $(date)" | \
    mail -s "Keepalived MASTER Notification" ops@example.com

# 可选：执行其他操作，如更新 DNS、通知监控系统等
EOF

# notify_backup.sh - 成为 Backup 时执行
cat > /etc/keepalived/scripts/notify_backup.sh << 'EOF'
#!/bin/bash

logger "Keepalived: This node became BACKUP"
EOF

# notify_fault.sh - 进入 Fault 状态时执行
cat > /etc/keepalived/scripts/notify_fault.sh << 'EOF'
#!/bin/bash

logger "Keepalived: This node entered FAULT state"

echo "Keepalived: $(hostname) entered FAULT state at $(date)" | \
    mail -s "Keepalived FAULT Alert" ops@example.com
EOF

chmod +x /etc/keepalived/scripts/*.sh

3.4.3 Real Server 状态变化通知

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
cat > /etc/keepalived/scripts/real_server_down.sh << 'EOF'
#!/bin/bash

RS_IP=$1

logger "Keepalived: Real Server $RS_IP is DOWN"

# 发送告警
echo "Real Server $RS_IP is DOWN at $(date)" | \
    mail -s "[ALERT] Real Server DOWN" ops@example.com

# 通知监控系统 (示例：调用 Webhook)
# curl -X POST -H "Content-Type: application/json" \
#      -d "{\"alert\": \"Real Server $RS_IP DOWN\", \"time\": \"$(date -Iseconds)\"}" \
#      https://monitoring.example.com/webhook
EOF

cat > /etc/keepalived/scripts/real_server_up.sh << 'EOF'
#!/bin/bash

RS_IP=$1

logger "Keepalived: Real Server $RS_IP is UP"

echo "Real Server $RS_IP recovered at $(date)" | \
    mail -s "[RECOVERED] Real Server UP" ops@example.com
EOF

chmod +x /etc/keepalived/scripts/real_server_*.sh

3.4.4 VIP 变化通知

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
cat > /etc/keepalived/scripts/notify_vip.sh << 'EOF'
#!/bin/bash

TYPE=$1
NAME=$2
STATE=$3
PRIORITY=$4
VIP=$5

logger "Keepalived: VIP $VIP $STATE on $NAME (priority: $PRIORITY)"

# 记录到日志文件
echo "$(date '+%Y-%m-%d %H:%M:%S') - VIP: $VIP, State: $STATE, Priority: $PRIORITY" >> /var/log/keepalived_vip.log
EOF

chmod +x /etc/keepalived/scripts/notify_vip.sh

3.5 启动服务

在 LVS 节点上执行：

1
2
3
4
5
6
7
8
9
10
11
12
13
# 启用 IP 转发
echo 1 > /proc/sys/net/ipv4/ip_forward
echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf

# 启动 keepalived
systemctl enable keepalived
systemctl start keepalived

# 检查状态
systemctl status keepalived

# 查看日志
journalctl -u keepalived -f

四、验证与测试

4.1 基础验证

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# 检查 VIP 是否绑定
ip addr show eth0 | grep 192.168.1.100

# 检查 IPVS 规则
ipvsadm -Ln

# 查看连接统计
ipvsadm -Lnc

# 查看 Keepalived 状态
ip addr show eth0

# 查看 VRRP 状态
journalctl -u keepalived | grep -i "state\|priority"

4.2 预期输出

Master 节点：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# ip addr show eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 ...
    inet 192.168.1.10/24 brd 192.168.1.255 scope global eth0
    inet 192.168.1.100/24 scope global eth0:0    # VIP 已绑定
    inet 192.168.1.101/24 scope global eth0:1

# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.1.100:80 wlc persistent 60
  -> 192.168.2.10:80              Route   1      0          0
  -> 192.168.2.11:80              Route   1      0          0
  -> 192.168.2.12:80              Route   1      0          0

4.3 故障切换测试

4.3.1 模拟 Master 故障

1
2
3
4
5
6
7
8
# 在 Master 上停止 keepalived
systemctl stop keepalived

# 在 Backup 上检查 VIP 是否接管
ip addr show eth0 | grep 192.168.1.100

# 查看 Backup 日志
journalctl -u keepalived | grep "Master"

4.3.2 恢复 Master

1
2
3
4
5
# 在 Master 上启动 keepalived
systemctl start keepalived

# 检查 VIP 是否回到 Master (抢占模式)
ip addr show eth0 | grep 192.168.1.100

4.4 负载测试

1
2
3
4
5
6
7
8
# 使用 ab 进行压力测试
ab -n 10000 -c 100 http://192.168.1.100/

# 查看连接分布
ipvsadm -Lnc

# 查看统计信息
ipvsadm -Ln --stats

4.5 Real Server 故障测试

1
2
3
4
5
6
7
8
9
# 停止一台 Real Server 的 Web 服务
systemctl stop nginx  # 或 httpd/tomcat

# 在 LVS 节点查看状态变化
ipvsadm -Ln
# 故障的 Real Server 应该被标记为不可用

# 查看日志
journalctl -u keepalived | grep "DOWN"

五、运维管理

5.1 常用命令

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# 查看 IPVS 连接表
ipvsadm -Lnc

# 查看 IPVS 统计信息
ipvsadm -Ln --stats
ipvsadm -Ln --rate

# 清空连接表
ipvsadm -C

# 查看 Keepalived 进程
ps aux | grep keepalived

# 重新加载配置
kill -HUP $(pidof keepalived)

# 强制切换到 Backup
systemctl stop keepalived

# 强制切换回 Master
systemctl start keepalived

5.2 日志管理

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# 配置独立日志
cat > /etc/rsyslog.d/keepalived.conf << 'EOF'
local0.*    /var/log/keepalived.log
EOF

# 重启 rsyslog
systemctl restart rsyslog

# 日志轮转配置
cat > /etc/logrotate.d/keepalived << 'EOF'
/var/log/keepalived.log {
    daily
    rotate 30
    compress
    delaycompress
    missingok
    notifempty
    create 640 root root
    postrotate
        systemctl reload rsyslog
    endscript
}
EOF

5.3 监控配置

5.3.1 Prometheus Exporter

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# 安装 keepalived_exporter
wget https://github.com/prometheus-community/keepalived_exporter/releases/download/v0.4.0/keepalived_exporter-0.4.0.linux-amd64.tar.gz
tar xzf keepalived_exporter-0.4.0.linux-amd64.tar.gz
mv keepalived_exporter /usr/local/bin/

# 创建 systemd 服务
cat > /etc/systemd/system/keepalived_exporter.service << 'EOF'
[Unit]
Description=Keepalived Prometheus Exporter
After=network.target

[Service]
Type=simple
ExecStart=/usr/local/bin/keepalived_exporter
Restart=always

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl enable keepalived_exporter
systemctl start keepalived_exporter

5.3.2 Grafana 告警规则

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
# prometheus-rules.yaml
groups:
  - name: keepalived
    rules:
      - alert: KeepalivedMasterDown
        expr: keepalived_state != 1
        for: 1m
        labels:
          severity: critical
        annotations:
          summary: "Keepalived Master 节点故障"
          description: "实例 {{ $labels.instance }} 不再是 Master 状态"

      - alert: KeepalivedNoBackends
        expr: keepalived_virtual_server_real_server_up == 0
        for: 1m
        labels:
          severity: critical
        annotations:
          summary: "LVS 后端服务器全部不可用"
          description: "虚拟服务器 {{ $labels.vserver }} 没有可用的后端"

      - alert: KeepalivedVipMissing
        expr: keepalived_vrrp_instance_vip == 0
        for: 1m
        labels:
          severity: warning
        annotations:
          summary: "VIP 未绑定"
          description: "VRRP 实例 {{ $labels.instance }} 的 VIP 未绑定"

5.4 性能调优

5.4.1 内核参数优化

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
cat >> /etc/sysctl.conf << 'EOF'
# IPVS 连接跟踪优化
net.ipv4.ip_conntrack_max = 1000000
net.netfilter.nf_conntrack_max = 1000000
net.netfilter.nf_conntrack_tcp_timeout_established = 1800
net.netfilter.nf_conntrack_tcp_timeout_time_wait = 120
net.netfilter.nf_conntrack_tcp_timeout_close_wait = 60
net.netfilter.nf_conntrack_tcp_timeout_fin_wait = 120

# 增加本地端口范围
net.ipv4.ip_local_port_range = 1024 65535

# TCP 优化
net.ipv4.tcp_max_syn_backlog = 8192
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_fin_timeout = 30
net.ipv4.tcp_keepalive_time = 1200
net.ipv4.tcp_keepalive_probes = 5
net.ipv4.tcp_keepalive_intvl = 15

# 增加文件描述符限制
fs.file-max = 2097152
fs.nr_open = 2097152
EOF

sysctl -p

5.4.2 调度算法选择

算法 适用场景
wlc (加权最少连接) 推荐，后端服务器性能不均时
rr (轮询) 后端服务器性能相同，请求处理时间相近
sh (源地址哈希) 需要会话保持，如购物车、登录状态
dh (目标地址哈希) 缓存服务器集群

5.4.3 持久连接配置

1
2
3
4
5
6
7
8
9
10
11
# 根据业务类型调整 persistence_timeout
# HTTP 短连接：30-60 秒
# HTTPS: 300-600 秒
# 数据库连接：3600 秒或更长

# 在 keepalived.conf 中：
persistence_timeout 60

# 基于源地址的持久化 (更精确)
persistence_engine sip_hash
persistence_timeout 60

5.5 故障排查

5.5.1 常见问题

问题 可能原因 解决方案
VIP 无法绑定 网络接口配置错误 检查 interface 配置，确保接口名称正确
Real Server 无法访问 路由/防火墙问题 检查路由表，关闭防火墙或添加规则
脑裂 (Split-Brain) 网络分区 检查网络连通性，配置多播/单播 VRRP
切换频繁 健康检查过于敏感 调整 fall/rise 参数，增加检测间隔
连接丢失 持久化时间过短 增加 persistence_timeout

5.5.2 调试技巧

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
# 启用详细日志
# 在 keepalived.conf 中添加:
global_defs {
  enable_script_security
}

# 查看 VRRP 包
tcpdump -i eth0 -n vrrp

# 查看 IPVS 包统计
ipvsadm -Ln --stats

# 检查连接跟踪
conntrack -L | grep 192.168.1.100

# 测试 Real Server 健康检查
curl -v http://192.168.2.10/health

六、高可用架构扩展

6.1 双主双备架构

对于更高可用性要求，可以部署双主双备架构：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
                    ┌─────────────────┐
                    │   用户请求       │
                    └────────┬────────┘
                             │
           ┌─────────────────┼─────────────────┐
           │                 │                 │
           ▼                 ▼                 ▼
    ┌─────────────┐   ┌─────────────┐   ┌─────────────┐
    │ VIP1: .100  │   │ VIP2: .101  │   │ VIP3: .102  │
    │ (HTTP)      │   │ (HTTPS)     │   │ (TCP)       │
    └──────┬──────┘   └──────┬──────┘   └──────┬──────┘
           │                 │                 │
    ┌──────┴──────┐   ┌──────┴──────┐   ┌──────┴──────┐
    │             │   │             │   │             │
    ▼             ▼   ▼             ▼   ▼             ▼
┌────────┐   ┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐
│Master 1│   │Master 2│ │Backup 1│ │Backup 2│ │Backup 1│ │Backup 2│
│(VIP1)  │   │(VIP2)  │ │(VIP1)  │ │(VIP2)  │ │(VIP3)  │ │(VIP3)  │
└────────┘   └────────┘ └────────┘ └────────┘ └────────┘ └────────┘

6.2 跨机房部署

使用 TUN 模式实现跨机房负载均衡：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# LVS 节点配置 (TUN 模式)
virtual_server 192.168.1.100 80 {
    lb_algo wlc
    lb_kind TUN          # 改为 TUN 模式
    persistence_timeout 60
    protocol TCP
    
    real_server 10.0.1.10 80 {  # 机房 A
        weight 1
        HTTP_GET { ... }
    }
    
    real_server 10.0.2.10 80 {  # 机房 B
        weight 1
        HTTP_GET { ... }
    }
}

# Real Server 配置 TUN 接口
ip tunnel add tunl0 mode ipip local <real_server_ip> remote 0.0.0.0
ifconfig tunl0 <vip> netmask 255.255.255.255 up

七、总结

Keepalived+LVS 是成熟稳定的四层负载均衡高可用解决方案，适用于大流量、高性能要求的生产环境。

核心要点回顾：

  1. 架构设计：Master-Backup 高可用 + DR 模式高性能
  2. 配置要点：VRRP 组 ID 一致、优先级区分、健康检查配置
  3. Real Server：VIP 绑定到 lo、ARP 参数配置
  4. 运维管理：日志监控、告警配置、性能调优
  5. 故障处理：脑裂预防、切换测试、问题排查

最佳实践：

  • 生产环境至少部署 2 个 LVS 节点（1 主 1 备）
  • 定期演练故障切换，确保高可用有效
  • 配置完善的监控告警，及时发现问题
  • 根据业务特点选择合适的调度算法和持久化策略
  • 做好容量规划，预留足够的性能余量

通过本文的实践指南，您可以快速搭建生产级的 Keepalived+LVS 高可用负载均衡集群，为业务系统提供稳定可靠的流量入口。