eck使用

发表于 分类于
使用 ECK Operator 在 Kubernetes 上部署 Elasticsearch，含关闭 TLS、节点分离与端口暴露等配置。

官方文档 https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-overview.html

安装 eck operator

1
2
3
4
kubectl create -f https://download.elastic.co/downloads/eck/2.8.0/crds.yaml
kubectl apply -f https://download.elastic.co/downloads/eck/2.8.0/operator.yaml

kubectl -n elastic-system logs -f statefulset.apps/elastic-operator

安装 es 参考 https://www.bladewan.com/2020/09/25/logging_1/

https://blog.csdn.net/cr7258/article/details/126613064

关闭 xpack

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: imwl
  namespace: elastic-system
spec:
  version: 7.17.9
  http:
    tls:
      selfSignedCertificate:
        disabled: true # 关闭tls
  nodeSets:
  - name: es-nodes
    count: 3
    podTemplate:
      spec:
        containers:
        - name: elasticsearch
          env:
       #   - name: ES_JAVA_OPTS
       #     value: -Xms2g -Xmx2g
       #   - name: READINESS_PROBE_PROTOCOL
       #     value: http

          resources:
            requests:
              memory: 4Gi
              cpu: 0.5
        #    limits:
        #      memory: 4Gi
        #      cpu: 2
          image: registry.example.com/test/elasticsearch-with-ik:7.17.9
        initContainers:
        - name: increase-vm-max-map
          image: registry.example.com/test/busybox:latest
          command: ["sysctl", "-w", "vm.max_map_count=262144"]
          securityContext:
            privileged: true
        - name: increase-fd-ulimit
          image: registry.example.com/test/busybox:latest
          command: ["sh", "-c", "ulimit -n 65536"]

    config:
      xpack.security.enabled: false
    volumeClaimTemplates:
    - metadata:
        name: elasticsearch-data
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 10Gi
        storageClassName: local-path

不关闭 xpack 设置自定义密码

只需要有这两个文件就行

1
2
3
4
5
6
7
[root@172-20-19-20 ~]# mkdir filerealm
[root@172-20-19-20 ~]# touch filerealm/users filerealm/users_roles
[root@172-20-19-20 ~]# docker run -v $(pwd)/filerealm:/usr/share/elasticsearch/config registry.example.com/test/elasticsearch-with-ik:7.17.9 bin/elasticsearch-users useradd imwl -p password -r superuser
[root@172-20-19-20 filerealm]# cat filerealm/users
imwl:<YOUR_PASSWORD_HASH>
[root@172-20-19-20 filerealm]# cat filerealm/users_roles
superuser:imwl

创建 secret

1
kubectl create secret generic imwl-elasticsearch-realm-secret --from-file filerealm -n elastic-system

修改

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: imwl
  namespace: elastic-system
spec:
  version: 7.17.9
  auth:
    fileRealm:
    - secretName: imwl-elasticsearch-realm-secret
  nodeSets:
  - name: es-nodes
    count: 3
    podTemplate:
      spec:
        containers:
        - name: elasticsearch
          env:
       #   - name: ES_JAVA_OPTS
       #     value: -Xms2g -Xmx2g
       #   - name: READINESS_PROBE_PROTOCOL
       #     value: http

          resources:
            requests:
              memory: 4Gi
              cpu: 0.5
        #    limits:
        #      memory: 4Gi
        #      cpu: 2
          image: registry.example.com/test/elasticsearch-with-ik:7.17.9
        initContainers:
        - name: increase-vm-max-map
          image: registry.example.com/test/busybox:latest
          command: ["sysctl", "-w", "vm.max_map_count=262144"]
          securityContext:
            privileged: true
        - name: increase-fd-ulimit
          image: registry.example.com/test/busybox:latest
          command: ["sh", "-c", "ulimit -n 65536"]
    volumeClaimTemplates:
    - metadata:
        name: elasticsearch-data
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 10Gi
        storageClassName: local-path

暴露端口

暴露端口

1
2
3
kubectl patch svc -n elastic-system imwl-es-http -p '{"spec": {"type": "NodePort"}}'

kubectl patch svc -n elastic-system imwl-es-http --type='json' -p='[{"op": "replace", "path": "/spec/ports/0/nodePort", "value": 31920}]'

访问

1
2
curl --insecure  -u "imwl:password" -k  https://192.168.2.100:31920/_cat/health?v
curl --insecure  -u "imwl:password" -k  https://192.168.2.100:31920/_cat/nodes?v

分离节点

可以粗略划分 [“master”, “data”, “ingest”, “ml”] # [] 表示 coordinating

按实际情况，可以分很多节点，也可以合并节点

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: test-es
  namespace: elastic-system
spec:
  version: 7.17.9
  auth:
    fileRealm:
    - secretName: imwl-elasticsearch-realm-secret
  nodeSets:
  - name: master
    count: 3
    config:
      node.roles: ["master"]
      node.store.allow_mmap: false
    volumeClaimTemplates:
    - metadata:
        name: elasticsearch-data
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 10Gi
        storageClassName: local-path
    podTemplate:
      spec:
        containers:
        - name: elasticsearch
          env:
          - name: ES_JAVA_OPTS
            value: -Xms2g -Xmx2g
       #   - name: READINESS_PROBE_PROTOCOL
       #     value: http

          resources:
            requests:
              memory: 4Gi
              cpu: 0.5
            limits:
              memory: 4Gi
              cpu: 2
          image: registry.example.com/test/elasticsearch-with-ik:7.17.9
        initContainers:
        - name: increase-vm-max-map
          image: registry.example.com/test/busybox:latest
          command: ["sysctl", "-w", "vm.max_map_count=262144"]
          securityContext:
            privileged: true
            runAsUser: 0
        - name: increase-fd-ulimit
          image: registry.example.com/test/busybox:latest
          command: ["sh", "-c", "ulimit -n 65536"]
          securityContext:
            privileged: true
            runAsUser: 0

  - name: data
    count: 3
    config:
      node.roles: ["data"]
      node.store.allow_mmap: false
    volumeClaimTemplates:
    - metadata:
        name: elasticsearch-data
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 10Gi
        storageClassName: local-path
    podTemplate:
      spec:
        containers:
        - name: elasticsearch
          env:
          - name: ES_JAVA_OPTS
            value: -Xms2g -Xmx2g
       #   - name: READINESS_PROBE_PROTOCOL
       #     value: http

          resources:
            requests:
              memory: 4Gi
              cpu: 0.5
            limits:
              memory: 4Gi
              cpu: 2
          image: registry.example.com/test/elasticsearch-with-ik:7.17.9
        initContainers:
        - name: increase-vm-max-map
          image: registry.example.com/test/busybox:latest
          command: ["sysctl", "-w", "vm.max_map_count=262144"]
          securityContext:
            privileged: true
            runAsUser: 0
        - name: increase-fd-ulimit
          image: registry.example.com/test/busybox:latest
          command: ["sh", "-c", "ulimit -n 65536"]
          securityContext:
            privileged: true
            runAsUser: 0
  - name: ingest
    count: 3
    config:
      node.roles: ["ingest"]
      node.store.allow_mmap: false
    volumeClaimTemplates:
    - metadata:
        name: elasticsearch-data
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 10Gi
        storageClassName: local-path
    podTemplate:
      spec:
        containers:
        - name: elasticsearch
          env:
          - name: ES_JAVA_OPTS
            value: -Xms2g -Xmx2g
       #   - name: READINESS_PROBE_PROTOCOL
       #     value: http

          resources:
            requests:
              memory: 4Gi
              cpu: 0.5
            limits:
              memory: 4Gi
              cpu: 2
          image: registry.example.com/test/elasticsearch-with-ik:7.17.9
        initContainers:
        - name: increase-vm-max-map
          image: registry.example.com/test/busybox:latest
          command: ["sysctl", "-w", "vm.max_map_count=262144"]
          securityContext:
            privileged: true
            runAsUser: 0
        - name: increase-fd-ulimit
          image: registry.example.com/test/busybox:latest
          command: ["sh", "-c", "ulimit -n 65536"]
          securityContext:
            privileged: true
            runAsUser: 0
  - name: coordinating
    count: 3
    config:
      node.roles: []
      node.store.allow_mmap: false
    volumeClaimTemplates:
    - metadata:
        name: elasticsearch-data
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 10Gi
        storageClassName: local-path
    podTemplate:
      spec:
        containers:
        - name: elasticsearch
          env:
          - name: ES_JAVA_OPTS
            value: -Xms2g -Xmx2g
       #   - name: READINESS_PROBE_PROTOCOL
       #     value: http

          resources:
            requests:
              memory: 4Gi
              cpu: 0.5
            limits:
              memory: 4Gi
              cpu: 2
          image: registry.example.com/test/elasticsearch-with-ik:7.17.9
        initContainers:
        - name: increase-vm-max-map
          image: registry.example.com/test/busybox:latest
          command: ["sysctl", "-w", "vm.max_map_count=262144"]
          securityContext:
            privileged: true
            runAsUser: 0
        - name: increase-fd-ulimit
          image: registry.example.com/test/busybox:latest
          command: ["sh", "-c", "ulimit -n 65536"]
          securityContext:
            privileged: true
            runAsUser: 0

开启 kibana

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
apiVersion: kibana.k8s.elastic.co/v1
kind: Kibana
metadata:
  name: test-kibana
  namespace: elastic-system
spec:
  version: 7.17.9
  count: 1
  elasticsearchRef:  # 获取上面的 es 信息
    name: test-es
    namespace: elastic-system
  config:
    server.host: "::"
    server.port: 5601
    server.rewriteBasePath: true   # 自定义 url prefix
    server.basePath: "/test/kibana"
    server.publicBaseUrl: "https://localhost:5601/test/kibana"
    i18n.locale: zh-CN
  podTemplate:
    spec:
      containers:
      - name: kibana
        image: registry.example.com/test/kibana:7.17.9
        env:
          - name: NODE_OPTIONS
            value: "--max-old-space-size=2048"
        resources:
          requests:
            memory: 1Gi
            cpu: 0.5
          limits:
            memory: 2.5Gi
            cpu: 2

        readinessProbe:   # 需要修改 健康检查 url
          failureThreshold: 3
          httpGet:
            path: /test/kibana/login
            port: 5601
            scheme: HTTPS
          initialDelaySeconds: 10
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 5

其他插件类似

fluent-bit 放在同一命名空间 

1
2
3
4
5
6
7
8
9
10
11
[OUTPUT]
    Name es
    Match *
    Host test-es-http   # test-es-http.elastic-system.svc.cluster.local
    Port 9200
    tls On
    tls.verify Off
    HTTP_User test
    HTTP_Passwd password
    Logstash_Format On
    Retry_Limit False