「云原生」Redis on K8s 編排部署講解與實(shí)戰(zhàn)操作

作者：大數(shù)據(jù)老司機(jī) 2022-11-06 21:31:11

云計(jì)算

云原生 Redis有三種集群模式：主從模式，Sentinel（哨兵）模式，Cluster模式，這三種模式環(huán)境編排部署都會(huì)在本文章介紹與實(shí)戰(zhàn)操作。

成都創(chuàng)新互聯(lián)主要從事網(wǎng)站設(shè)計(jì)制作、網(wǎng)站設(shè)計(jì)、網(wǎng)頁(yè)設(shè)計(jì)、企業(yè)做網(wǎng)站、公司建網(wǎng)站等業(yè)務(wù)。立足成都服務(wù)撫順縣,10余年網(wǎng)站建設(shè)經(jīng)驗(yàn),價(jià)格優(yōu)惠、服務(wù)專(zhuān)業(yè),歡迎來(lái)電咨詢(xún)建站服務(wù):028-86922220

一、概述

REmote DIctionary Server(Redis) 是一個(gè)由 Salvatore Sanfilippo 寫(xiě)的 key-value 存儲(chǔ)系統(tǒng)，是跨平臺(tái)的非關(guān)系型數(shù)據(jù)庫(kù)。

Redis有三種集群模式：主從模式，Sentinel（哨兵）模式，Cluster模式，這三種模式環(huán)境編排部署都會(huì)在本文章介紹與實(shí)戰(zhàn)操作。

二、redis 主從模式編排部署實(shí)戰(zhàn)操作

地址：https://artifacthub.io/packages/helm/bitnami/redis

1）下載chart 包

helm repo add bitnami https://charts.bitnami.com/bitnami

helm pull bitnami/redis --version 17.3.7

tar -xf redis-17.3.7.tgz

2）構(gòu)建鏡像

這里就不重新構(gòu)建鏡像了，只是把遠(yuǎn)程鏡像tag一下，推到本地harbor倉(cāng)庫(kù)加速下載鏡像。有不清楚怎么構(gòu)建鏡像的小伙伴，可以私信或者留言。

docker pull docker.io/bitnami/redis:7.0.5-debian-11-r7

# tag
docker tag docker.io/bitnami/redis:7.0.5-debian-11-r7 myharbor.com/bigdata/redis:7.0.5-debian-11-r7

# 推送鏡像到本地harbor倉(cāng)庫(kù)
docker push myharbor.com/bigdata/redis:7.0.5-debian-11-r7

3）修改yaml編排

• redis/templates/master/pv.yaml

新增pv.yaml文件，內(nèi)容如下：

{{- range .Values.master.persistence.local }}
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: {{ .name }}
  labels:
    name: {{ .name }}
spec:
  storageClassName: {{ $.Values.master.persistence.storageClass }}
  capacity:
    storage: {{ $.Values.master.persistence.size }}
  accessModes:
    - ReadWriteOnce
  local:
    path: {{ .path }}
  nodeAffinity:
    required:
      nodeSelectorTerms:
        - matchExpressions:
            - key: kubernetes.io/hostname
              operator: In
              values:
                - {{ .host }}
---
{{- end }}

• redis/templates/replicas/pv.yaml

新增pv.yaml文件，內(nèi)容如下：

{{- range .Values.replica.persistence.local }}
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: {{ .name }}
  labels:
    name: {{ .name }}
spec:
  storageClassName: {{ $.Values.replica.persistence.storageClass }}
  capacity:
    storage: {{ $.Values.replica.persistence.size }}
  accessModes:
    - ReadWriteOnce
  local:
    path: {{ .path }}
  nodeAffinity:
    required:
      nodeSelectorTerms:
        - matchExpressions:
            - key: kubernetes.io/hostname
              operator: In
              values:
                - {{ .host }}
---
{{- end }}

• redis/values.yaml

global:
  redis:
    password: "123456"

...

image:
  registry: myharbor.com
  repository: bigdata/redis
  tag: 7.0.5-debian-11-r7

master:
  count: 1
  persistence:
    enabled: true
    size: 8Gi
    storageClass: "local-redis-storage"
    local:
    - name: redis-0
      host: "local-168-182-110"
      path: "/opt/bigdata/servers/redis/data/data1"

replica:
  replicaCount: 2
  persistence:
    enabled: true
    size: 8Gi
    storageClass: "local-redis-storage"
    local:
    - name: redis-1
      host: "local-168-182-111"
      path: "/opt/bigdata/servers/redis/data/data1"
    - name: redis-2
      host: "local-168-182-112"
      path: "/opt/bigdata/servers/redis/data/data1"

4）開(kāi)始部署

# 創(chuàng)建存儲(chǔ)目錄
mkdir /opt/bigdata/servers/redis/data/data1

# 先檢查語(yǔ)法
helm lint ./redis

# 開(kāi)始安裝
helm install redis ./redis -n redis --create-namespace

NOTES

REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: redis
CHART VERSION: 17.3.7
APP VERSION: 7.0.5

** Please be patient while the chart is being deployed **

Redis? can be accessed on the following DNS names from within your cluster:

    redis-master.redis.svc.cluster.local for read/write operations (port 6379)
    redis-replicas.redis.svc.cluster.local for read-only operations (port 6379)



To get your password run:

    export REDIS_PASSWORD=$(kubectl get secret --namespace redis redis -o jsonpath="{.data.redis-password}" | base64 -d)

To connect to your Redis? server:

1. Run a Redis? pod that you can use as a client:

   kubectl run --namespace redis redis-client --restart='Never'  --env REDIS_PASSWORD=$REDIS_PASSWORD  --image myharbor.com/bigdata/redis:7.0.5-debian-11-r7 --command -- sleep infinity

   Use the following command to attach to the pod:

   kubectl exec --tty -i redis-client \
   --namespace redis -- bash

2. Connect using the Redis? CLI:
   REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h redis-master
   REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h redis-replicas

To connect to your database from outside the cluster execute the following commands:

    kubectl port-forward --namespace redis svc/redis-master 6379:6379 &
    REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h 127.0.0.1 -p 6379

5）測(cè)試驗(yàn)證

kubectl get pods,svc -n redis -owide

# 登錄master，可讀可寫(xiě)
kubectl exec -it redis-master-0 -n redis -- redis-cli -h redis-master -a $(kubectl get secret --namespace redis redis -o jsonpath="{.data.redis-password}" | base64 -d)

# 登錄slave，只讀
kubectl exec -it redis-master-0 -n redis -- redis-cli -h redis-replicas -a $(kubectl get secret --namespace redis redis -o jsonpath="{.data.redis-password}" | base64 -d)

6）卸載

helm uninstall redis-sentinel -n redis-sentinel
# delete ns 
kubectl delete ns redis-sentinel --force
# delete pv
kubectl delete pv `kubectl get pv|grep ^redis-|awk '{print $1}'` --force

rm -fr /opt/bigdata/servers/redis/data/data1/*

三、redis 哨兵模式編排部署實(shí)戰(zhàn)操作

主從模式的弊端就是不具備高可用性，當(dāng)master掛掉以后，Redis將不能再對(duì)外提供寫(xiě)入操作，因此sentinel應(yīng)運(yùn)而生。

1）構(gòu)建鏡像

這里也重新構(gòu)建鏡像了，有不懂構(gòu)建鏡像的小伙伴可以在評(píng)論下方留言。這里也只是把遠(yuǎn)程的鏡像推送到本地harbor。

docker pull docker.io/bitnami/redis-sentinel:7.0.5-debian-11-r6
# tag
docker tag docker.io/bitnami/redis-sentinel:7.0.5-debian-11-r6 myharbor.com/bigdata/redis-sentinel:7.0.5-debian-11-r6
# push
docker push  myharbor.com/bigdata/redis-sentinel:7.0.5-debian-11-r6

2）修改yaml編排

• redis-sentinel/values.yaml

replica:
  # replica.replicaCount與sentinel.quorum值一樣
  replicaCount: 3
  storageClass: "local-redis-storage"
    local:
    - name: redis-0
      host: "local-168-182-110"
      path: "/opt/bigdata/servers/redis/data/data1"
    - name: redis-1
      host: "local-168-182-111"
      path: "/opt/bigdata/servers/redis/data/data1"
    - name: redis-2
      host: "local-168-182-112"
      path: "/opt/bigdata/servers/redis/data/data1"

sentinel:
  enabled: true
  image:
    registry: myharbor.com
    repository: bigdata/redis-sentinel
    tag: 7.0.5-debian-11-r6
  quorum: 3

• redis-sentinel/templates/replicas/pv.yaml

新增pv.yaml文件，內(nèi)容如下：

{{- range .Values.sentinel.persistence.local }}
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: {{ .name }}
  labels:
    name: {{ .name }}
spec:
  storageClassName: {{ $.Values.sentinel.persistence.storageClass }}
  capacity:
    storage: {{ $.Values.sentinel.persistence.size }}
  accessModes:
    - ReadWriteOnce
  local:
    path: {{ .path }}
  nodeAffinity:
    required:
      nodeSelectorTerms:
        - matchExpressions:
            - key: kubernetes.io/hostname
              operator: In
              values:
                - {{ .host }}
---
{{- end }}

3）開(kāi)始部署

# 創(chuàng)建存儲(chǔ)目錄
mkdir -p /opt/bigdata/servers/redis/data/data1

helm install redis-sentinel ./redis-sentinel -n redis-sentinel --create-namespace 

NOTES

NAME: redis-sentinel
LAST DEPLOYED: Fri Nov  4 22:42:52 2022
NAMESPACE: redis-sentinel
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: redis
CHART VERSION: 17.3.7
APP VERSION: 7.0.5

** Please be patient while the chart is being deployed **

Redis? can be accessed via port 6379 on the following DNS name from within your cluster:

    redis-sentinel.redis-sentinel.svc.cluster.local for read only operations

For read/write operations, first access the Redis? Sentinel cluster, which is available in port 26379 using the same domain name above.



To get your password run:

    export REDIS_PASSWORD=$(kubectl get secret --namespace redis-sentinel redis-sentinel -o jsonpath="{.data.redis-password}" | base64 -d)

To connect to your Redis? server:

1. Run a Redis? pod that you can use as a client:

   kubectl run --namespace redis-sentinel redis-client --restart='Never'  --env REDIS_PASSWORD=$REDIS_PASSWORD  --image myharbor.com/bigdata/redis:7.0.5-debian-11-r7 --command -- sleep infinity

   Use the following command to attach to the pod:

   kubectl exec --tty -i redis-client \
   --namespace redis-sentinel -- bash

2. Connect using the Redis? CLI:
   REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h redis-sentinel -p 6379 # Read only operations
   REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h redis-sentinel -p 26379 # Sentinel access

To connect to your database from outside the cluster execute the following commands:

    kubectl port-forward --namespace redis-sentinel svc/redis-sentinel 6379:6379 &
    REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h 127.0.0.1 -p 6379

查看

kubectl get pods,svc -n redis-sentinel -owide

4）模擬故障測(cè)試

# 查看
kubectl exec -it redis-sentinel-node-0 -n redis-sentinel -- redis-cli -h redis-sentinel -a $(kubectl get secret --namespace redis-sentinel redis-sentinel -o jsonpath="{.data.redis-password}" | base64 -d) info replication

模擬故障，kill master pod

kubectl delete pod redis-sentinel-node-0 -n redis-sentinel

再次查看master所在節(jié)點(diǎn)，master節(jié)點(diǎn)已經(jīng)切換到其它節(jié)點(diǎn)了。

再測(cè)試讀寫(xiě)

# 登錄master節(jié)點(diǎn)
kubectl exec -it redis-sentinel-node-0 -n redis-sentinel -- redis-cli -h redis-sentinel-node-2.redis-sentinel-headless -a $(kubectl get secret --namespace redis-sentinel redis-sentinel -o jsonpath="{.data.redis-password}" | base64 -d)

# 登錄slave節(jié)點(diǎn)
kubectl exec -it redis-sentinel-node-0 -n redis-sentinel -- redis-cli -h redis-sentinel-node-0.redis-sentinel-headless -a $(kubectl get secret --namespace redis-sentinel redis-sentinel -o jsonpath="{.data.redis-password}" | base64 -d)

5）卸載

helm uninstall redis-sentinel -n redis
# delete ns 
kubectl delete ns redis --force
# delete pv
kubectl delete pv `kubectl get pv|grep ^redis-|awk '{print $1}'` --force

rm -fr /opt/bigdata/servers/redis/data/data1/*

四、redis 集群模式編排部署實(shí)戰(zhàn)操作

集群模式可以說(shuō)是sentinel+主從模式的結(jié)合體，通過(guò)cluster可以實(shí)現(xiàn)主從和master重選功能，所以如果配置兩個(gè)副本三個(gè)分片的話，就需要六個(gè)Redis實(shí)例。因?yàn)镽edis的數(shù)據(jù)是根據(jù)一定規(guī)則分配到cluster的不同機(jī)器的，當(dāng)數(shù)據(jù)量過(guò)大時(shí)，可以新增機(jī)器進(jìn)行擴(kuò)容。

1）下載chart 包

helm repo add bitnami https://charts.bitnami.com/bitnami

helm pull bitnami/redis-cluster --version 8.2.7

tar -xf redis-cluster-8.2.7.tgz

2）構(gòu)建鏡像

這里就不重新構(gòu)建鏡像了，只是把遠(yuǎn)程鏡像tag一下，推到本地harbor倉(cāng)庫(kù)加速下載鏡像。有不清楚怎么構(gòu)建鏡像的小伙伴，可以私信或者留言。

docker pull docker.io/bitnami/redis-cluster:7.0.5-debian-11-r9

# tag
docker tag docker.io/bitnami/redis-cluster:7.0.5-debian-11-r9 myharbor.com/bigdata/redis-cluster:7.0.5-debian-11-r9

# 推送鏡像到本地harbor倉(cāng)庫(kù)
docker push myharbor.com/bigdata/redis-cluster:7.0.5-debian-11-r9

3）修改yaml編排

• redis-cluster/templates/pv.yaml

新增pv.yaml文件，內(nèi)容如下：

{{- range .Values.persistence.local }}
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: {{ .name }}
  labels:
    name: {{ .name }}
spec:
  storageClassName: {{ $.Values.persistence.storageClass }}
  capacity:
    storage: {{ $.Values.persistence.size }}
  accessModes:
    - ReadWriteOnce
  local:
    path: {{ .path }}
  nodeAffinity:
    required:
      nodeSelectorTerms:
        - matchExpressions:
            - key: kubernetes.io/hostname
              operator: In
              values:
                - {{ .host }}
---
{{- end }}

password: "123456"

...

image:
  registry: myharbor.com
  repository: bigdata/redis-cluster
  tag: 7.0.5-debian-11-r9

...

persistence:
  storageClass: "local-redis-cluster-storage"
  local:
    - name: redis-cluster-0
      host: "local-168-182-110"
      path: "/opt/bigdata/servers/redis-cluster/data/data1"
    - name: redis-cluster-1
      host: "local-168-182-110"
      path: "/opt/bigdata/servers/redis-cluster/data/data2"
    - name: redis-cluster-2
      host: "local-168-182-110"
      path: "/opt/bigdata/servers/redis-cluster/data/data3"
    - name: redis-cluster-3
      host: "local-168-182-111"
      path: "/opt/bigdata/servers/redis-cluster/data/data1"
    - name: redis-cluster-4
      host: "local-168-182-111"
      path: "/opt/bigdata/servers/redis-cluster/data/data2"
    - name: redis-cluster-5
      host: "local-168-182-111"
      path: "/opt/bigdata/servers/redis-cluster/data/data3"
    - name: redis-cluster-6
      host: "local-168-182-112"
      path: "/opt/bigdata/servers/redis-cluster/data/data1"
    - name: redis-cluster-7
      host: "local-168-182-112"
      path: "/opt/bigdata/servers/redis-cluster/data/data2"
    - name: redis-cluster-8
      host: "local-168-182-112"
      path: "/opt/bigdata/servers/redis-cluster/data/data3"
  
cluster:
  init: true
  # 一主兩從（三組）
  nodes: 9
  replicas: 2

4）開(kāi)始部署

# 創(chuàng)建存儲(chǔ)目錄
mkdir -p /opt/bigdata/servers/redis-cluster/data/data{1..3}

helm install redis-cluster ./redis-cluster -n redis-cluster --create-namespace

NOTES

NOTES:
CHART NAME: redis-cluster
CHART VERSION: 8.2.7
APP VERSION: 7.0.5** Please be patient while the chart is being deployed **


To get your password run:
    export REDIS_PASSWORD=$(kubectl get secret --namespace "redis-cluster" redis-cluster -o jsonpath="{.data.redis-password}" | base64 -d)

You have deployed a Redis? Cluster accessible only from within you Kubernetes Cluster.INFO: The Job to create the cluster will be created.To connect to your Redis? cluster:

1. Run a Redis? pod that you can use as a client:
kubectl run --namespace redis-cluster redis-cluster-client --rm --tty -i --restart='Never' \
 --env REDIS_PASSWORD=$REDIS_PASSWORD \
--image myharbor.com/bigdata/redis-cluster:7.0.5-debian-11-r9 -- bash

2. Connect using the Redis? CLI:

redis-cli -c -h redis-cluster -a $REDIS_PASSWORD

查看

kubectl get pods,svc -n redis-cluster -owide

5）故障模擬測(cè)試

kubectl exec -it redis-cluster-0 -n redis-cluster -- redis-cli -c -h redis-cluster -a $(kubectl get secret --namespace "redis-cluster" redis-cluster -o jsnotallow="{.data.redis-password}" | base64 -d) CLUSTER INFO

kubectl exec -it redis-cluster-0 -n redis-cluster -- redis-cli -c -h redis-cluster -a $(kubectl get secret --namespace "redis-cluster" redis-cluster -o jsonpath="{.data.redis-password}" | base64 -d) CLUSTER NODES

刪除其中一個(gè)master節(jié)點(diǎn)

kubectl delete pod redis-cluster-1 -n redis-cluster

# 再查看節(jié)點(diǎn)情況
kubectl exec -it redis-cluster-0 -n redis-cluster -- redis-cli -c -h redis-cluster -a $(kubectl get secret --namespace "redis-cluster" redis-cluster -o jsonpath="{.data.redis-password}" | base64 -d) CLUSTER NODES

6）卸載

helm uninstall redis-cluster -n redis-cluster
# delete ns 
kubectl delete ns redis-cluster --force
# delete pv
kubectl delete pv `kubectl get pv|grep ^redis-cluster-|awk '{print $1}'` --force

rm -fr /opt/bigdata/servers/redis-cluster/data/data{1..3}/*

git地址：https://gitee.com/hadoop-bigdata/redis-on-k8s?

網(wǎng)站名稱(chēng)：「云原生」RedisonK8s編排部署講解與實(shí)戰(zhàn)操作
網(wǎng)站URL：http://www.dlmjj.cn/article/codecji.html