集群搭建和应用

3主3从，部署在同一台机器上

创建配置

进入redis根目录

touch master-8001.conf

echo "port  8001
bind  127.0.0.1
daemonize    yes
pidfile  /var/run/master-8001.pid
cluster-enabled  yes
cluster-config-file  master-8001.conf
cluster-node-timeout  15000
appendonly  yes
cluster-require-full-coverage no" >> master-8001.conf

创建 cluster-conf 目录，用来存放集群配置文件

mkdir cluster-conf
mkdir cluster-config-file

创建 master-8001.conf

手动修改以下信息：

port  8001
bind  127.0.0.1
daemonize    yes
pidfile  /var/run/master-8001.pid
cluster-enabled  yes
cluster-config-file  cluster-config-file/master-8001.conf
cluster-node-timeout  15000
appendonly  yes
cluster-require-full-coverage no

批量创建配置文件

cp master-8001.conf master-8002.conf
cp master-8001.conf master-8003.conf

sed -i "s/8001/8002/g" master-8002.conf
sed -i "s/8001/8003/g" master-8003.conf

cp master-8001.conf slave-8004.conf
sed -i "s/master/slave/g" slave-8004.conf
cp slave-8004.conf slave-8005.conf 
cp slave-8004.conf slave-8006.conf 
sed -i "s/8004/8005/g" slave-8005.conf
sed -i "s/8004/8006/g" slave-8006.conf

启动redis

./src/redis-server cluster-conf/master-8001.conf 
 ./src/redis-server cluster-conf/master-8002.conf 
 ./src/redis-server cluster-conf/master-8003.conf 

./src/redis-server cluster-conf/slave-8004.conf 
./src/redis-server cluster-conf/slave-8005.conf 
 ./src/redis-server cluster-conf/slave-8006.conf

启动Cluster

 ./src/redis-trib.rb create --replicas 1 127.0.0.1:8001 127.0.0.1:8002 127.0.0.1:8003 127.0.0.1:8004 127.0.0.1:8005 127.0.0.1:8006

报错

/usr/bin/env: “ruby”: 没有那个文件或目录

** 如果是5.0之前要安装ruby，5.0后不用。5.0后是通过redis-cli启动cluster的。**

 liangjf@blue:~/app/redis-5.0.5$ ./src/redis-trib.rb create --replicas 1 127.0.0.1:8001 127.0.0.1:8002 127.0.0.1:8003 127.0.0.1:8004 127.0.0.1:8005 127.0.0.1:8006
WARNING: redis-trib.rb is not longer available!
You should use redis-cli instead.

All commands and features belonging to redis-trib.rb have been moved
to redis-cli.
In order to use them you should call redis-cli with the --cluster
option followed by the subcommand name, arguments and options.

Use the following syntax:
redis-cli --cluster SUBCOMMAND [ARGUMENTS] [OPTIONS]

Example:
**redis-cli --cluster create 127.0.0.1:8001 127.0.0.1:8002 127.0.0.1:8003 127.0.0.1:8004 127.0.0.1:8005 127.0.0.1:8006 --cluster-replicas 1**

To get help about all subcommands, type:
redis-cli --cluster help

报错2

 liangjf@blue:~/app/redis-5.0.5$ ./src/redis-cli --cluster create 127.0.0.1:8001 127.0.0.1:8002 127.0.0.1:8003 127.0.0.1:8004 127.0.0.1:8005 127.0.0.1:8006 --cluster-replicas 1
[ERR] Node 127.0.0.1:8001 NOAUTH Authentication required.

暂时把 requirepass 123456 注释掉，因为测试环境就不配置认证了。

启动成功

进行选主等步骤

正确启动

 liangjf@blue:~/app/redis-5.0.5$ ./src/redis-cli --cluster create 127.0.0.1:8001 127.0.0.1:8002 127.0.0.1:8003 127.0.0.1:8004 127.0.0.1:8005 127.0.0.1:8006 --cluster-replicas 1
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 127.0.0.1:8005 to 127.0.0.1:8001
Adding replica 127.0.0.1:8006 to 127.0.0.1:8002
Adding replica 127.0.0.1:8004 to 127.0.0.1:8003
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: 59232d02f280a3dc02b563938ebe5aa869cd8853 127.0.0.1:8001
   slots:[0-5460] (5461 slots) master
M: 79f540c00247f52e0350baa81d9d170707b608a0 127.0.0.1:8002
   slots:[5461-10922] (5462 slots) master
M: db86f7b8724dc37fc07a8869bd11279a83be4b77 127.0.0.1:8003
   slots:[10923-16383] (5461 slots) master
S: c5c0b3ea0b2e809f99974ee89fb1322b3b83f15c 127.0.0.1:8004
   replicates 59232d02f280a3dc02b563938ebe5aa869cd8853
S: f0be8fe7cf5d22480236ce538c055481646c7a70 127.0.0.1:8005
   replicates 79f540c00247f52e0350baa81d9d170707b608a0
S: dd30ca40199a199739cb6871351ebc4da98b7fa5 127.0.0.1:8006
   replicates db86f7b8724dc37fc07a8869bd11279a83be4b77
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
......
>>> Performing Cluster Check (using node 127.0.0.1:8001)
M: 59232d02f280a3dc02b563938ebe5aa869cd8853 127.0.0.1:8001
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
M: 79f540c00247f52e0350baa81d9d170707b608a0 127.0.0.1:8002
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: dd30ca40199a199739cb6871351ebc4da98b7fa5 127.0.0.1:8006
   slots: (0 slots) slave
   replicates db86f7b8724dc37fc07a8869bd11279a83be4b77
S: c5c0b3ea0b2e809f99974ee89fb1322b3b83f15c 127.0.0.1:8004
   slots: (0 slots) slave
   replicates 59232d02f280a3dc02b563938ebe5aa869cd8853
S: f0be8fe7cf5d22480236ce538c055481646c7a70 127.0.0.1:8005
   slots: (0 slots) slave
   replicates 79f540c00247f52e0350baa81d9d170707b608a0
M: db86f7b8724dc37fc07a8869bd11279a83be4b77 127.0.0.1:8003
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

检测集群连接

随机连接集群一个redis。

liangjf@blue:~/app/redis-5.0.5$ ./src/redis-cli -c -p 8001
127.0.0.1:8001> cluster nodes

59232d02f280a3dc02b563938ebe5aa869cd8853 127.0.0.1:8001@18001 myself,master - 0 1563540210000 1 connected 0-5460 - 79f540c00247f52e0350baa81d9d170707b608a0 127.0.0.1:8002@18002 master - 0 1563540211998 2 connected 5461-10922
dd30ca40199a199739cb6871351ebc4da98b7fa5 127.0.0.1:8006@18006 slave db86f7b8724dc37fc07a8869bd11279a83be4b77 0 1563540213001 6 connected
c5c0b3ea0b2e809f99974ee89fb1322b3b83f15c 127.0.0.1:8004@18004 slave 59232d02f280a3dc02b563938ebe5aa869cd8853 0 1563540214006 4 connected
f0be8fe7cf5d22480236ce538c055481646c7a70 127.0.0.1:8005@18005 slave 79f540c00247f52e0350baa81d9d170707b608a0 0 1563540213000 5 connected
db86f7b8724dc37fc07a8869bd11279a83be4b77 127.0.0.1:8003@18003 master - 0 1563540212000 3 connected 10923-16383

测试集群功能

 127.0.0.1:8001> set name tony
-> Redirected to slot [5798] located at 127.0.0.1:8002
OK
127.0.0.1:8002> exit
liangjf@blue:~/app/redis-5.0.5$ ./src/redis-cli -c -p 8002
127.0.0.1:8002> get name
"tony"
127.0.0.1:8002> 

高可用测试

liangjf@blue:~/app/redis-5.0.5$ ps -aux | grep redis
liangjf  15279  0.2  0.0  86492  5668 ?        Ssl  20:38   0:01 ./src/redis-server 0.0.0.0:8001 [cluster]
liangjf  15294  0.2  0.0  88540  6132 ?        Ssl  20:38   0:01 ./src/redis-server 0.0.0.0:8002 [cluster]
liangjf  15309  0.2  0.0  82396  5508 ?        Ssl  20:38   0:01 ./src/redis-server 0.0.0.0:8003 [cluster]
liangjf  15328  0.2  0.0  87004  5260 ?        Ssl  20:38   0:01 ./src/redis-server 0.0.0.0:8004 [cluster]
liangjf  15343  0.2  0.0  87004  5244 ?        Ssl  20:38   0:01 ./src/redis-server 0.0.0.0:8005 [cluster]
liangjf  15359  0.2  0.0  87004  5260 ?        Ssl  20:38   0:01 ./src/redis-server 0.0.0.0:8006 [cluster]
liangjf  15472  0.0  0.0  40048  1088 pts/1    S+   20:47   0:00 grep --color=auto redis
liangjf@blue:~/app/redis-5.0.5$ kill -9 15279
liangjf@blue:~/app/redis-5.0.5$ ps -aux | grep redis
liangjf  15294  0.2  0.0  88540  6132 ?        Ssl  20:38   0:01 ./src/redis-server 0.0.0.0:8002 [cluster]
liangjf  15309  0.2  0.0  82396  5508 ?        Ssl  20:38   0:01 ./src/redis-server 0.0.0.0:8003 [cluster]
liangjf  15328  0.2  0.0  87004  5260 ?        Ssl  20:38   0:01 ./src/redis-server 0.0.0.0:8004 [cluster]
liangjf  15343  0.2  0.0  87004  5244 ?        Ssl  20:38   0:01 ./src/redis-server 0.0.0.0:8005 [cluster]
liangjf  15359  0.2  0.0  87004  5260 ?        Ssl  20:38   0:01 ./src/redis-server 0.0.0.0:8006 [cluster]
liangjf  15496  0.0  0.0  40048  1004 pts/1    S+   20:47   0:00 grep --color=auto redis
liangjf@blue:~/app/redis-5.0.5$ ./src/redis-cli -c -p 8002
127.0.0.1:8002> set name qwe
OK
127.0.0.1:8002> exit
liangjf@blue:~/app/redis-5.0.5$ ./src/redis-cli -c -p 8004
127.0.0.1:8004> get name
-> Redirected to slot [5798] located at 127.0.0.1:8002
"qwe"

重起被kill的机器8001

会看到8001重启后，重新加入集群

liangjf@blue:~/app/redis-5.0.5$ ./src/redis-server cluster-conf/master-8001.conf 
15546:C 19 Jul 2019 20:49:58.879 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
15546:C 19 Jul 2019 20:49:58.879 # Redis version=5.0.5, bits=64, commit=00000000, modified=0, pid=15546, just started
15546:C 19 Jul 2019 20:49:58.879 # Configuration loaded
liangjf@blue:~/app/redis-5.0.5$ 
liangjf@blue:~/app/redis-5.0.5$ ps -aux | grep redis
liangjf  15294  0.3  0.0  92636  6628 ?        Ssl  20:38   0:02 ./src/redis-server 0.0.0.0:8002 [cluster]
liangjf  15309  0.2  0.0  82396  5512 ?        Ssl  20:38   0:02 ./src/redis-server 0.0.0.0:8003 [cluster]
liangjf  15328  0.3  0.0  94684  6112 ?        Ssl  20:38   0:02 ./src/redis-server 0.0.0.0:8004 [cluster]
liangjf  15343  0.3  0.0  87004  5244 ?        Ssl  20:38   0:02 ./src/redis-server 0.0.0.0:8005 [cluster]
liangjf  15359  0.3  0.0  87004  5260 ?        Ssl  20:38   0:02 ./src/redis-server 0.0.0.0:8006 [cluster]
liangjf  15547  0.1  0.0  87004  5220 ?        Ssl  20:49   0:00 ./src/redis-server 0.0.0.0:8001 [cluster]
liangjf  15568  0.0  0.0  40048   996 pts/1    S+   20:50   0:00 grep --color=auto redis
liangjf@blue:~/app/redis-5.0.5$ ./src/redis-cli -c -p 8004
127.0.0.1:8004> cluster nodes
59232d02f280a3dc02b563938ebe5aa869cd8853 127.0.0.1:8001@18001 slave c5c0b3ea0b2e809f99974ee89fb1322b3b83f15c 0 1563540637069 7 connected
f0be8fe7cf5d22480236ce538c055481646c7a70 127.0.0.1:8005@18005 slave 79f540c00247f52e0350baa81d9d170707b608a0 0 1563540635061 5 connected
dd30ca40199a199739cb6871351ebc4da98b7fa5 127.0.0.1:8006@18006 slave db86f7b8724dc37fc07a8869bd11279a83be4b77 0 1563540636066 6 connected
79f540c00247f52e0350baa81d9d170707b608a0 127.0.0.1:8002@18002 master - 0 1563540635000 2 connected 5461-10922
db86f7b8724dc37fc07a8869bd11279a83be4b77 127.0.0.1:8003@18003 master - 0 1563540634059 3 connected 10923-16383
c5c0b3ea0b2e809f99974ee89fb1322b3b83f15c 127.0.0.1:8004@18004 myself,master - 0 1563540634000 7 connected 0-5460

启动脚本

start-cluster.sh

#!/bin/bash

./src/redis-server cluster-conf/master-8001.conf
./src/redis-server cluster-conf/master-8002.conf
./src/redis-server cluster-conf/master-8003.conf
./src/redis-server cluster-conf/slave-8004.conf
./src/redis-server cluster-conf/slave-8005.conf
./src/redis-server cluster-conf/slave-8006.conf

./src/redis-cli --cluster create 127.0.0.1:8001 127.0.0.1:8002 127.0.0.1:8003 127.0.0.1:8004 127.0.0.1:8005 127.0.0.1:8006 --cluster-replicas 1

kill-all.sh

#!/bin/bash

ps -ef | grep redis-server | grep -v grep | cut -c 9-15 | xargs kill -9

rm appendonly.aof  dump.rdb
rm cluster-config-file/*

requirepass 123456

密码控制也会影响到从库复制，从库必须在配置文件里使用 masterauth 指令配置相应的密码才可以进行复制操作。masterauth 123456

redis

redis集群搭建和应用