Redis on RedHat 8

Three nodes with following specifications:

  • 4 vCPU
  • 32 GiB RAM
  • 64 GB HDD (1 GB /boot, 16 GB swap, rest /)

Install and register RedHat 8

IP address and instances:

Host Namebpjs-poc01bpjs-poc02bpjs-poc03
IP Address172.20.1.220172.20.1.221172.20.1.222
Redis Instance
(Port Number)		master-1
(7000)		master-2
(7000)		master-3
(7000)
slave-3
(7001)		slave-1
(7001)		slave-2
(7001)

Login as root

Upgrade node to latest available updates

Modify /etc/sysctl.conf file

vi /etc/sysctl.conf

Add following lines

vm.overcommit_memory=1
net.core.somaxconn=511

Install required packages

dnf install -y vim wget tcl make gcc python3 systemd-devel.x86_64

Install redis

wget https://download.redis.io/redis-stable.tar.gz
tar zxvf redis-stable.tar.gz
cd redis-stable
make
make test
make install

Check installation

[root@node-1 ~]# whereis redis-server
redis-server: /usr/local/bin/redis-server
[root@node-1 ~]# redis-server --version
Redis server v=7.0.5 sha=00000000:0 malloc=jemalloc-5.2.1 bits=64 build=a49a56f97af1c0b0

Prepare required directories

On all nodes

mkdir -p /etc/redis

On bpjs-poc01

mkdir -p /var/redis/m1
mkdir -p /var/redis/s3

On bpjs-poc02

mkdir -p /var/redis/m2
mkdir -p /var/redis/s1

On bpjs-poc03

mkdir -p /var/redis/m3
mkdir -p /var/redis/s2

Prepare configuration files

On bpjs-poc01

cp ~/redis-stable/redis.conf /etc/redis/m1.conf
cp ~/redis-stable/redis.conf /etc/redis/s3.conf

On bpjs-poc02

cp ~/redis-stable/redis.conf /etc/redis/m2.conf
cp ~/redis-stable/redis.conf /etc/redis/s1.conf

On bpjs-poc03

cp ~/redis-stable/redis.conf /etc/redis/m3.conf
cp ~/redis-stable/redis.conf /etc/redis/s2.conf

Configuring redis-server master-1 on node bpjs-poc01

vi /etc/redis/m1.conf

Change following lines

bind 127.0.0.1 172.20.1.220
pidfile /var/run/redis_m1.pid
logfile "/var/log/redis_m1.log"
dir /var/redis/m1
cluster-config-file nodes-m1.conf

protected-mode no
port 7000
daemonize no
supervised systemd
loglevel notice
save 900 1 300 10 60 10000
stop-writes-on-bgsave-error yes
rdbchecksum yes
dbfilename dump.rdb
replica-read-only yes
maxmemory 64mb
maxmemory-policy volatile-ttl
appendonly yes
cluster-enabled yes
cluster-node-timeout 5000
cluster-replica-validity-factor 1
slowlog-log-slower-than 10000
slowlog-max-len 64
latency-monitor-threshold 100

Create systemd service file

vi /etc/systemd/system/redis-m1.service

Add following lines

[Unit]
Description=Redis data structure server - redis-m1
Documentation=https://redis.io/documentation
AssertPathExists=/etc/redis/m1.conf
AssertPathExists=/var/redis/m1

[Service]
ExecStart=/usr/local/bin/redis-server /etc/redis/m1.conf
LimitNOFILE=10032
NoNewPrivileges=yes
Type=notify
TimeoutStartSec=infinity
TimeoutStopSec=infinity
UMask=0077
User=root
Group=root
WorkingDirectory=/var/redis/m1
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

Reload service list

systemctl daemon-reload

Start redis-m1 service

systemctl start redis-m1.service

Check status of redis-m1.service

[root@bpjs-poc01 ~]# systemctl status redis-m1.service 
● redis-m1.service - Redis data structure server - redis-server-m1
   Loaded: loaded (/etc/systemd/system/redis-m1.service; enabled; vendor preset: disabled)
   Active: active (running) since Tue 2022-12-20 14:03:16 WIB; 1h 26min ago
     Docs: https://redis.io/documentation
 Main PID: 2826 (redis-server)
   Status: "Ready to accept connections"
    Tasks: 5 (limit: 204130)
   Memory: 11.5M
   CGroup: /system.slice/redis-m1.service
           └─2826 /usr/local/bin/redis-server 127.0.0.1:7000 [cluster]

Dec 20 14:03:16 bpjs-poc01.lab systemd[1]: Starting Redis data structure server - redis-server-m1...
Dec 20 14:03:16 bpjs-poc01.lab systemd[1]: Started Redis data structure server - redis-server-m1.

If everything is in order, make redis-m1 service to start at system boot

systemctl enable redis-m1

Configuring redis-server slave-3 on node bpjs-poc01

vi /etc/redis/s3.conf

Change following lines

bind 127.0.0.1 172.20.1.220
pidfile /var/run/redis_s3.pid
logfile "/var/log/redis_s3.log"
dir /var/redis/s3
cluster-config-file nodes-s3.conf

protected-mode no
port 7001
daemonize no
supervised systemd
loglevel notice
save 900 1 300 10 60 10000
stop-writes-on-bgsave-error yes
rdbchecksum yes
dbfilename dump.rdb
replica-read-only yes
maxmemory 64mb
maxmemory-policy volatile-ttl
appendonly yes
cluster-enabled yes
cluster-node-timeout 5000
cluster-replica-validity-factor 1
slowlog-log-slower-than 10000
slowlog-max-len 64
latency-monitor-threshold 100

Reload service list

systemctl daemon-reload

Start redis-s3 service

systemctl start redis-s3.service

Check status of redis-s3.service

[root@bpjs-poc01 ~]# systemctl status redis-s3.service 
● redis-s3.service - Redis data structure server - redis-server-s3
   Loaded: loaded (/etc/systemd/system/redis-s3.service; disabled; vendor preset: disabled)
   Active: active (running) since Tue 2022-12-20 15:24:25 WIB; 5min ago
     Docs: https://redis.io/documentation
 Main PID: 4181 (redis-server)
   Status: "Ready to accept connections"
    Tasks: 5 (limit: 204130)
   Memory: 9.6M
   CGroup: /system.slice/redis-s3.service
           └─4181 /usr/local/bin/redis-server 127.0.0.1:7001 [cluster]

Dec 20 15:24:25 bpjs-poc01.lab systemd[1]: Starting Redis data structure server - redis-server-s3...
Dec 20 15:24:25 bpjs-poc01.lab systemd[1]: Started Redis data structure server - redis-server-s3.

If everything is in order, make redis-s3 service to start at system boot

systemctl enable redis-m1

Repeat the procedure for other redis-server nodes on host bpjs-poc02 and bpjs-poc03

Configure firewall

For all hosts

firewall-cmd --permanent --new-zone=redis
firewall-cmd --permanent --zone=redis --add-port=7000/tcp
firewall-cmd --permanent --zone=redis --add-port=7001/tcp
firewall-cmd --permanent --zone=redis --add-port=17000/tcp
firewall-cmd --permanent --zone=redis --add-port=17001/tcp

For host bpjs-poc01

firewall-cmd --permanent --zone=redis --add-source=172.20.1.221/32
firewall-cmd --permanent --zone=redis --add-source=172.20.1.222/32

For host bpjs-poc02

firewall-cmd --permanent --zone=redis --add-source=172.20.1.220/32
firewall-cmd --permanent --zone=redis --add-source=172.20.1.222/32

For host bpjs-poc03

firewall-cmd --permanent --zone=redis --add-source=172.20.1.220/32
firewall-cmd --permanent --zone=redis --add-source=172.20.1.221/32

And finally on all hosts again

# additional source for redis clients:
# per-host:
# firewall-cmd --permanent --zone=redis --add-source=a.b.c.d/32
# per-subnet:
# firewall-cmd --permanent --zone=redis --add-source=a.b.c.d/24
firewall-cmd --reload

Test connectivity between redis-server nodes, do this test from all hosts

[root@bpjs-poc01 ~]# redis-cli -h 172.20.1.220 -p 7000 ping
PONG
[root@bpjs-poc01 ~]# redis-cli -h 172.20.1.221 -p 7000 ping
PONG
[root@bpjs-poc01 ~]# redis-cli -h 172.20.1.222 -p 7000 ping
PONG
[root@bpjs-poc01 ~]# redis-cli -h 172.20.1.220 -p 7001 ping
PONG
[root@bpjs-poc01 ~]# redis-cli -h 172.20.1.221 -p 7001 ping
PONG
[root@bpjs-poc01 ~]# redis-cli -h 172.20.1.222 -p 7001 ping
PONG

Configure slots allocation

  • Slot 0 – 5400 will be on master-1
  • Slot 5401 – 10800 will be on master-2
  • Slot 10801 – 16383 will be on master-3
for i in {0..5400}; do redis-cli -h 172.20.1.220 -p 7000 cluster addslots $i; done
for i in {5401..10800}; do redis-cli -h 172.20.1.221 -p 7000 cluster addslots $i; done
for i in {10801..16383}; do redis-cli -h 172.20.1.222 -p 7000 cluster addslots $i; done

Check slots allocation

redis-cli -h 172.20.1.220 -p 7000 cluster nodes
redis-cli -h 172.20.1.221 -p 7000 cluster nodes
redis-cli -h 172.20.1.222 -p 7000 cluster nodes

Expected result

[root@bpjs-poc01 ~]# redis-cli -h 172.20.1.220 -p 7000 cluster nodes
1bc48f7819cc086c9f7f3f6036ac07fb6cfdab32 :7000@17000 myself,master - 0 0 0 connected 0-5400
[root@bpjs-poc01 ~]# redis-cli -h 172.20.1.221 -p 7000 cluster nodes
56e2821e3f5c8090b313f5e1b675d1c8f7d6f1c2 :7000@17000 myself,master - 0 0 0 connected 5401-10800
[root@bpjs-poc01 ~]# redis-cli -h 172.20.1.222 -p 7000 cluster nodes
9d9377a36874520492e5f7d5972e49e303b59805 :7000@17000 myself,master - 0 0 0 connected 10801-16383

Configure cluster

redis-cli -h 172.20.1.220 -p 7000 cluster meet 172.20.1.221 7000
redis-cli -h 172.20.1.220 -p 7000 cluster meet 172.20.1.222 7000
redis-cli -h 172.20.1.220 -p 7000 cluster meet 172.20.1.220 7001
redis-cli -h 172.20.1.220 -p 7000 cluster meet 172.20.1.221 7001
redis-cli -h 172.20.1.220 -p 7000 cluster meet 172.20.1.222 7001

Check cluster

redis-cli -h 172.20.1.220 -p 7000 cluster nodes
redis-cli -h 172.20.1.221 -p 7000 cluster nodes
redis-cli -h 172.20.1.222 -p 7000 cluster nodes
redis-cli -h 172.20.1.220 -p 7001 cluster nodes
redis-cli -h 172.20.1.221 -p 7001 cluster nodes
redis-cli -h 172.20.1.222 -p 7001 cluster nodes

Sample output

[root@bpjs-poc01 ~]# redis-cli -h 172.20.1.220 -p 7000 cluster nodes
9d9377a36874520492e5f7d5972e49e303b59805 172.20.1.222:7000@17000 master - 0 1671527157596 2 connected 10801-16383
1bc48f7819cc086c9f7f3f6036ac07fb6cfdab32 172.20.1.220:7000@17000 myself,master - 0 1671527155000 1 connected 0-5400
56e2821e3f5c8090b313f5e1b675d1c8f7d6f1c2 172.20.1.221:7000@17000 master - 0 1671527157000 3 connected 5401-10800
a9b44086f805e90ac469fedff291e036e81cd83d 172.20.1.221:7001@17001 master - 0 1671527156000 0 connected
52ee97f631dbfb93adb2227e843c2be827f70cf6 172.20.1.222:7001@17001 master - 0 1671527156593 4 connected
267ec5816469ba6ab8d50a0660e75086920d0865 172.20.1.220:7001@17001 master - 0 1671527157596 5 connected

Summarize redis-server ID from above output

NodeIP AddressPortID
master-1172.20.1.22070001bc48f7819cc086c9f7f3f6036ac07fb6cfdab32
slave-1172.20.1.2217001a9b44086f805e90ac469fedff291e036e81cd83d
master-2172.20.1.221700056e2821e3f5c8090b313f5e1b675d1c8f7d6f1c2
slave-2172.20.1.222700152ee97f631dbfb93adb2227e843c2be827f70cf6
master-3172.20.1.22270009d9377a36874520492e5f7d5972e49e303b59805
slave-3172.20.1.2207001267ec5816469ba6ab8d50a0660e75086920d0865

Relate slave-1 with master-1 for replication

redis-cli -h 172.20.1.221 -p 7001 cluster replicate 1bc48f7819cc086c9f7f3f6036ac07fb6cfdab32

Relate slave-2 with master-2 for replication

redis-cli -h 172.20.1.222 -p 7001 cluster replicate 56e2821e3f5c8090b313f5e1b675d1c8f7d6f1c2

Relate slave-3 with master-3 for replication

redis-cli -h 172.20.1.220 -p 7001 cluster replicate 9d9377a36874520492e5f7d5972e49e303b59805

To check master – slave replication relation

redis-cli -h <node-ip-address> -p <node-port> cluster nodes

Sample output

[root@bpjs-poc01 ~]# redis-cli -h 172.20.1.221 -p 7001 cluster nodes
1bc48f7819cc086c9f7f3f6036ac07fb6cfdab32 172.20.1.220:7000@17000 master - 0 1671528172504 1 connected 0-5400
9d9377a36874520492e5f7d5972e49e303b59805 172.20.1.222:7000@17000 master - 0 1671528172000 2 connected 10801-16383
56e2821e3f5c8090b313f5e1b675d1c8f7d6f1c2 172.20.1.221:7000@17000 master - 0 1671528172403 3 connected 5401-10800
267ec5816469ba6ab8d50a0660e75086920d0865 172.20.1.220:7001@17001 slave 9d9377a36874520492e5f7d5972e49e303b59805 0 1671528172000 2 connected
52ee97f631dbfb93adb2227e843c2be827f70cf6 172.20.1.222:7001@17001 slave 56e2821e3f5c8090b313f5e1b675d1c8f7d6f1c2 0 1671528171400 3 connected
a9b44086f805e90ac469fedff291e036e81cd83d 172.20.1.221:7001@17001 myself,slave 1bc48f7819cc086c9f7f3f6036ac07fb6cfdab32 0 1671528171000 1 connected

To check master – slave replication status

redis-cli -h <node-ip-address> -p <node-port> info replication

Sample output

[root@bpjs-poc01 ~]# redis-cli -h 172.20.1.220 -p 7000 info replication
# Replication
role:master
connected_slaves:1
slave0:ip=172.20.1.221,port=7001,state=online,offset=476,lag=1
master_failover_state:no-failover
master_replid:9d936aa441797056fffbb0fdea0274e4087b8047
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:476
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:476
[root@bpjs-poc01 ~]# redis-cli -h 172.20.1.221 -p 7000 info replication
# Replication
role:master
connected_slaves:1
slave0:ip=172.20.1.222,port=7001,state=online,offset=504,lag=1
master_failover_state:no-failover
master_replid:0e57c188b71960afe0cb29eff030d9139e7ce6b8
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:504
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:504
[root@bpjs-poc01 ~]# redis-cli -h 172.20.1.222 -p 7000 info replication
# Replication
role:master
connected_slaves:1
slave0:ip=172.20.1.220,port=7001,state=online,offset=504,lag=0
master_failover_state:no-failover
master_replid:a34048f8341b69e94285a7f1a11664e08cffe98c
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:504
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:504

Benchmark test

redis-benchmark -q -h <node-ip-address> -p <node-port> --cluster

Sample output

[root@bpjs-poc01 ~]# redis-benchmark -q -h 172.20.1.220 -p 7000 --cluster
Cluster has 3 master nodes:

Master 0: 9d9377a36874520492e5f7d5972e49e303b59805 172.20.1.222:7000
Master 1: 1bc48f7819cc086c9f7f3f6036ac07fb6cfdab32 172.20.1.220:7000
Master 2: 56e2821e3f5c8090b313f5e1b675d1c8f7d6f1c2 172.20.1.221:7000

PING_INLINE: 133155.80 requests per second, p50=0.167 msec                    
PING_MBULK: 132978.73 requests per second, p50=0.159 msec                    
SET: 133155.80 requests per second, p50=0.215 msec                    
GET: 132978.73 requests per second, p50=0.199 msec                    
INCR: 133155.80 requests per second, p50=0.199 msec                    
LPUSH: 133155.80 requests per second, p50=0.199 msec                    
RPUSH: 132978.73 requests per second, p50=0.199 msec                    
LPOP: 133155.80 requests per second, p50=0.191 msec                    
RPOP: 132978.73 requests per second, p50=0.175 msec                    
SADD: 133333.33 requests per second, p50=0.143 msec                    
HSET: 132978.73 requests per second, p50=0.175 msec                    
SPOP: 133155.80 requests per second, p50=0.151 msec                    
ZADD: 133155.80 requests per second, p50=0.159 msec                    
ZPOPMIN: 132978.73 requests per second, p50=0.159 msec                    
LPUSH (needed to benchmark LRANGE): 133155.80 requests per second, p50=0.183 msec                    
LRANGE_100 (first 100 elements): 99900.09 requests per second, p50=0.223 msec                     
LRANGE_300 (first 300 elements): 49875.31 requests per second, p50=0.471 msec                   
LRANGE_500 (first 500 elements): 33244.68 requests per second, p50=0.655 msec                   
LRANGE_600 (first 600 elements): 30721.96 requests per second, p50=0.775 msec                   
MSET (10 keys): 132978.73 requests per second, p50=0.231 msec

Leave a Reply

Your email address will not be published. Required fields are marked *