{"componentChunkName":"component---src-templates-post-template-jsx","path":"/works/posts/2022-08-23--001","result":{"data":{"site":{"siteMetadata":{"title":"Blog by Eunyoung","subtitle":"작업 기록 블로그","copyright":"© All rights reserved.","author":{"name":"EunYoung","twitter":"#"},"disqusShortname":"","url":"https://ssongey.github.io"}},"markdownRemark":{"id":"61f1c961-c4c1-552e-8f0e-14f1255862d8","html":"

이번에 인프라 이전 작업이 잦아질 듯 싶다.
\n작업 기록을 정리하고 스크립트를 만들 수 있는 곳을 찾아봐야겠다.

\n

✔️ 깃헙 레포 미러링

\n\n

✔️ 클러스터 생성

\n\n

✔️ 클러스터 세팅

\n

1. ingress controller 설치

\n

1.1 도커파일 세팅

\n\n
$ docker pull {official docker image}\n\n# 이미지명 변경\n$ docker image tag {이미지ID} {변경할 이름}:{태그}\n또는\n$ docker image tag {변경전 이름}:{태그} {변경할 이름}:{태그}\n\n$ docker push {변경된 이름}:{태그}
\n\n
echo {패스워드/CREDENTIAL_SECRET} | docker login {registry 주소} -u {아이디/CREDENTIAL_ID} --password-stdin
\n

1.2. 설치

\n\n

1.3. 배포 확인

\n\n
\n

2. secret 생성

\n

✔️ docker-registry

\n\n

✔️ tls 인증서

\n\n
\n

3. proxy daemonset 설치

\n\n
\n

✔️ EFK 세팅

\n

1. ES 설치는 인프라팀에 요청

\n

2. Kibana 설치

\n\n
\n

3. Log Aggregator 서버 설치

\n\n

3.1. vm 생성 후 apt 업데이트

\n
$ sudo apt -y update
\n

3.2. td-agent 설치

\n\n
$ curl -fsSL https://toolbelt.treasuredata.com/sh/install-ubuntu-bionic-td-agent4.sh | sh\n\n# 설치 후 fluentd 버전 확인\n$ /opt/td-agent/bin/fluentd --version
\n\n

3.3. /etc/td-agent/td-agent.conf 파일 수정

\n\n

3.4. td-agent 실행

\n\n
\n

4. fluentd 데몬셋 세팅

\n

4.1. RBAC 설정

\n
apiVersion: v1\nkind: ServiceAccount\nmetadata:\n  name: fluentd-service-account\n  namespace: fluentd\n\n---\napiVersion: rbac.authorization.k8s.io/v1\nkind: ClusterRole\nmetadata:\n  name: fluentd-cluster-role\n  namespace: fluentd\nrules:\n  - apiGroups:\n      - \"\"\n    resources:\n      - pods\n      - namespaces\n    verbs:\n      - get\n      - list\n      - watch\n---\nkind: ClusterRoleBinding\napiVersion: rbac.authorization.k8s.io/v1\nmetadata:\n  name: fluentd-cluster-role-binding\n  namespace: fluentd\nroleRef:\n  kind: ClusterRole\n  name: fluentd-cluster-role\n  apiGroup: rbac.authorization.k8s.io\nsubjects:\n  - kind: ServiceAccount\n    name: fluentd-service-account\n    namespace : fluentd
\n

4.2. input, filter, output 에 대한 기본 설정 (configmap.yaml)

\n
apiVersion: v1\nkind: ConfigMap\nmetadata:\n  name: fluentd-configmap\n  namespace: fluentd\n  labels:\n    k8s-app: fluentd-logging\n    kubernetes.io/cluster-service: \"true\"\n    version: v1\ndata:\n  fluent.conf: |\n    <system>\n      workers 1\n      @log_level info\n    </system>\n    @include input-kubernetes.conf\n    @include filter-kubernetes.conf\n\n    <match **>\n      @type copy\n      <store>\n        @type                         forward\n        @id                           out_ai_forward\n        expire_dns_cache              60s\n        <server>\n          host                        {로그어그리게이터 서버주소}\n        </server>\n        <buffer>\n          @type                       memory\n          flush_interval              5s\n          flush_at_shutdown           true\n        </buffer>\n      </store>\n    </match>\n\n  input-kubernetes.conf: |\n    <match fluent.**>\n      @type                           null\n    </match>\n    <source>\n      @type                           tail\n      @id                             in_tail_container_logs\n      path                            /var/log/containers/*.log\n      pos_file                        /var/log/fluentd-containers.log.pos\n      exclude_path                    [\"/var/log/containers/fluentd*.log\"]\n      tag                             kubernetes.*\n      read_from_head                  \"#{ENV['FLUENTD_INPUT_READ_FROM_HEAD'] || 'false'}\"\n      limit_recently_modified         \"#{ENV['LIMIT_RECENTLY_MODIFIED'] || '120m'}\"\n      rotate_wait                     0\n      enable_stat_watcher             false\n      <parse>\n        @type                         regexp\n        expression                    /^(?<time>[^ ]+) (?<stream>stdout|stderr) (?<flags>[^ ]+) (?<message>.*)$/\n      </parse>\n    </source>\n\n  filter-kubernetes.conf: |\n    <filter kubernetes.**>\n      @type                           kubernetes_metadata\n      @id                             filter_kube_metadata\n    </filter>\n    \n    \n    <filter kubernetes.**>\n      @type                           record_modifier\n      tag                             ${record.dig(\"kubernetes\", \"container_name\").gsub('-', '.')}\n      <record>\n        @timestamp                    ${(Time.at(time) + (60*60*9)).strftime('%Y-%m-%dT%H:%M:%S.%L+09:00')}\n        cluster_name                  \"cluster name\"\n        hostname                      \"#{ENV['K8S_NODE_NAME']}\"\n        container_image               ${record.dig(\"kubernetes\", \"container_image\")}\n        pod_name                      ${record.dig(\"kubernetes\", \"pod_name\")}\n      </record>\n    </filter>\n    \n    <filter kubernetes.**>\n      @type                           parser\n      key_name                        message\n      reserve_data                    true\n      remove_key_name_field           true\n      <parse>\n        @type                         json\n      </parse>\n    </filter>\n    \n    <filter kubernetes.**>\n      @type                           record_transformer\n      remove_keys                     $.kubernetes\n    </filter>
\n

4.3. 데몬셋 설정

\n\n
apiVersion: apps/v1\nkind: DaemonSet\nmetadata:\n  name: fluentd-daemonset\n  namespace: fluentd\n  labels:\n    k8s-app: fluentd-logging\n    version: v1\n    kubernetes.io/cluster-service: \"true\"\nspec:\n  selector:\n    matchLabels:\n      k8s-app: fluentd-logging\n      kubernetes.io/cluster-service: \"true\"\n      version: v1\n  updateStrategy:\n    type: RollingUpdate\n  template:\n    metadata:\n      labels:\n        k8s-app: fluentd-logging\n        version: v1\n        kubernetes.io/cluster-service: \"true\"\n    spec:\n      hostNetwork: true\n      dnsPolicy: ClusterFirstWithHostNet\n      serviceAccount: fluentd-service-account\n      serviceAccountName: fluentd-service-account\n      tolerations:\n        - key: node-role.kubernetes.io/master\n          effect: NoExecute\n      containers:\n        - name: fluentd\n          image: {fluentd 이미지}\n          imagePullPolicy: IfNotPresent\n          env:\n            - name: K8S_NODE_NAME\n              valueFrom:\n                fieldRef:\n                  fieldPath: spec.nodeName\n          resources:\n            limits:\n              memory: 400Mi\n            requests:\n              cpu: 100m\n              memory: 200Mi\n          volumeMounts:\n            - name: varlog\n              mountPath: /var/log\n            - name: varlibdockercontainers\n              mountPath: /var/lib/docker/containers\n              readOnly: true\n            - name: resolvconf\n              mountPath: /etc/resolv.conf\n            - name: tz-config\n              mountPath: /etc/localtime\n              readOnly: true\n            - name: fluentd-config\n              mountPath: /fluentd/etc/\n      terminationGracePeriodSeconds: 30\n      volumes:\n        - name: varlog\n          hostPath:\n            path: /var/log\n        - name: varlibdockercontainers\n          hostPath:\n            path: /var/lib/docker/containers\n        - name: resolvconf\n          hostPath:\n            path: /etc/resolv.conf\n        - name: tz-config\n          hostPath:\n            path: /etc/localtime\n        - name: fluentd-config\n          configMap:\n            name: fluentd-configmap
\n\n
\n

5. Kibana에서 index 패턴 추가

\n\n

\n \n \n \n