본문 바로가기
카테고리 없음

40. 클라우드 네이티브 Go 실습 2편

philipjung 2022. 1. 11. 19:04

적어도 향후 몇년 동안은 Go, Mongo에 집중해야 겠다는 생각이 들어서, 블로그를 통해서 내용을 정리하고 공유하고 있다. 데이터와 프론트엔드에 집중하다 보니, 백엔드와 객체지향적인 설계에 소홀한 것 같다. 요즘 객체지향, 포인터, 클래스 등을 다시 공부하면서 즐거움을 느끼고 있다. 보면 볼수록 Go는 여러가지로 매력적인 언어다.

 

  • gRPC는 구글이 최초로 개발한 오픈 소스 원격 프로시저 호출 시스템이다. 전송을 위해 HTTP/2를, 인터페이스 정의 언어로 프로토콜 버퍼를 사용하며 인증, 양방향 스트리밍 및 흐름 제어, 차단 및 비차단 바인딩, 취소 및 타임아웃 등의 기능을 제공
  • Apache Beam은 배치와 실시간 데이터 파이프라인을 개발하고, 다양한 런타임 환경을 지원
  • Go-kit는 마이크로서비스 프레임워크이고 다양한 마이크로서비스 디자인 패턴을 지원
  • Mongo는 가장 유명한 NoSQL이고 JSON의 관리 및 집계에 유용
  • Redis는 가장 유명한 캐시 저장소
  • Tracing는 관찰가능성 중 분산추적을 위한 프레임워크를 제공
  • Echo는 웹 어플리케이션 프레임워크이고 Air와 함께 auto reload를 지원
  • MySQL은 오픈소스 RDBMS
  • GORM은 Go언어에서 사용 가능한 ORM(Object Relation Mapping) 라이브러리
  • Operator는 사용자 정의 리소스의 컨트롤러 역할을 수행하며, 쿠버네티스 자원에 대한 멱등성을 구현
  • Exporter는 다양한 시스템으로부터 메트릭을 수집해서 프로메테우스에게 제공
  • RealWorld은 Echo, GORM 등을 사용한 레퍼런스 데모를 제공

2편에서는 Mongo, Redis, Tracing을 데모하고 설명한다.

4. Mongo

[root@localhost stringsvc1]# vi /etc/yum.repos.d/mongo.repo
[mongodb-org-4.2]
name=MongoDB Repository
baseurl=https://repo.mongodb.org/yum/redhat/$releasever/mongodb-org/4.2/x86_64/
gpgcheck=1
enabled=1
gpgkey=https://www.mongodb.org/static/pgp/server-4.2.asc

 

[root@localhost stringsvc1]# yum install -y mongodb-org

 

[root@localhost stringsvc1]# systemctl start mongod

 

[root@localhost stringsvc1]# mongo
MongoDB shell version v4.2.18
connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("856135a0-9d8b-4b19-a994-d0e2bed35249") }
MongoDB server version: 4.2.18
Welcome to the MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
	https://docs.mongodb.com/
Questions? Try the MongoDB Developer Community Forums
	https://community.mongodb.com
Server has startup warnings: 
2022-01-09T21:26:42.341-0500 I  CONTROL  [initandlisten] 
2022-01-09T21:26:42.341-0500 I  CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2022-01-09T21:26:42.341-0500 I  CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2022-01-09T21:26:42.341-0500 I  CONTROL  [initandlisten] 
2022-01-09T21:26:42.342-0500 I  CONTROL  [initandlisten] 
2022-01-09T21:26:42.342-0500 I  CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2022-01-09T21:26:42.342-0500 I  CONTROL  [initandlisten] **        We suggest setting it to 'never'
2022-01-09T21:26:42.342-0500 I  CONTROL  [initandlisten] 
2022-01-09T21:26:42.342-0500 I  CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2022-01-09T21:26:42.342-0500 I  CONTROL  [initandlisten] **        We suggest setting it to 'never'
2022-01-09T21:26:42.342-0500 I  CONTROL  [initandlisten] 
---
Enable MongoDB's free cloud-based monitoring service, which will then receive and display
metrics about your deployment (disk utilization, CPU, operation statistics, etc).

The monitoring data will be available on a MongoDB website with a unique URL accessible to you
and anyone you share the URL with. MongoDB may use this information to make product
improvements and to suggest MongoDB products and deployment options to you.

To enable free monitoring, run the following command: db.enableFreeMonitoring()
To permanently disable this reminder, run the following command: db.disableFreeMonitoring()
---

>

 

[root@localhost stringsvc1]# go get go.mongodb.org/mongo-driver/mongo
go: downloading go.mongodb.org/mongo-driver v1.8.2
go: downloading github.com/pkg/errors v0.9.1
go: downloading github.com/youmark/pkcs8 v0.0.0-20181117223130-1be2e3e5546d
go: downloading github.com/go-stack/stack v1.8.0
go: downloading github.com/golang/snappy v0.0.1
go: downloading github.com/klauspost/compress v1.13.6
go: downloading github.com/xdg-go/scram v1.0.2
go: downloading github.com/xdg-go/stringprep v1.0.2
go: downloading golang.org/x/crypto v0.0.0-20210314154223-e6e6c4f2bb5b
go: downloading golang.org/x/sync v0.0.0-20201207232520-09787c993a3a
go: downloading github.com/xdg-go/pbkdf2 v1.0.0
go: downloading golang.org/x/text v0.3.5
go get: added go.mongodb.org/mongo-driver v1.8.2

 

[root@localhost mongo]# git clone https://github.com/tfogo/mongodb-go-tutorial.git

 

[root@localhost mongodb-go-tutorial]# go mod init mongo
go: creating new go.mod: module mongo
go: copying requirements from Gopkg.lock
go: to add module requirements and sums:
	go mod tidy

 

[root@localhost mongodb-go-tutorial]# go mod tidy
go: downloading go.mongodb.org/mongo-driver v1.0.0
go: downloading github.com/xdg/scram v0.0.0-20180814205039-7eeb5667e42c
go: downloading github.com/xdg/stringprep v0.0.0-20180714160509-73f8eece6fdc
go: downloading golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6
go: downloading golang.org/x/crypto v0.0.0-20190313024323-a1f597ede03a
go: downloading github.com/golang/snappy v0.0.0-20190218232222-2a8bb927dd31
go: downloading golang.org/x/text v0.3.0
go: finding module for package github.com/stretchr/testify/require
go: finding module for package github.com/google/go-cmp/cmp
go: downloading github.com/stretchr/testify v1.7.0
go: downloading github.com/google/go-cmp v0.5.6
go: finding module for package github.com/tidwall/pretty
go: finding module for package github.com/stretchr/testify/assert
go: downloading github.com/tidwall/pretty v1.2.0
go: found github.com/google/go-cmp/cmp in github.com/google/go-cmp v0.5.6
go: found github.com/stretchr/testify/require in github.com/stretchr/testify v1.7.0
go: found github.com/tidwall/pretty in github.com/tidwall/pretty v1.2.0
go: found github.com/stretchr/testify/assert in github.com/stretchr/testify v1.7.0
go: downloading github.com/davecgh/go-spew v1.1.0
go: downloading github.com/pmezard/go-difflib v1.0.0
go: downloading gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c

 

[root@localhost mongodb-go-tutorial]# go build
[root@localhost mongodb-go-tutorial]# ll
total 9576
-rw-r--r--. 1 root root     644 Jan  9 21:57 go.mod
-rw-r--r--. 1 root root    3119 Jan  9 21:50 Gopkg.lock
-rw-r--r--. 1 root root     731 Jan  9 21:50 Gopkg.toml
-rw-r--r--. 1 root root    3057 Jan  9 21:57 go.sum
-rw-r--r--. 1 root root    2967 Jan  9 21:50 main.go
-rwxr-xr-x. 1 root root 9771029 Jan  9 21:57 mongo
-rw-r--r--. 1 root root   11662 Jan  9 21:50 README.md

 

[root@localhost mongodb-go-tutorial]# ./mongo
Connected to MongoDB!
Inserted a single document:  ObjectID("61dba108ecb2ec34514eb85f")
Inserted multiple documents:  [ObjectID("61dba108ecb2ec34514eb860") ObjectID("61dba108ecb2ec34514eb861")]
Matched 1 documents and updated 1 documents.
Found a single document: {Name:Ash Age:11 City:Pallet Town}
Found multiple documents (array of pointers): [0xc00006f170 0xc00006f1a0]
Deleted 3 documents in the trainers collection
Connection to MongoDB closed.
[root@localhost mongodb-go-tutorial]#

 

package main

import (
	"context"
	"fmt"
	"log"

	"go.mongodb.org/mongo-driver/bson"
	"go.mongodb.org/mongo-driver/mongo"
	"go.mongodb.org/mongo-driver/mongo/options"
)

type Trainer struct {
	Name string
	Age  int
	City string
}

func main() {

	// Set client options
	clientOptions := options.Client().ApplyURI("mongodb://localhost:27017")

	// Connect to MongoDB
	client, err := mongo.Connect(context.TODO(), clientOptions)
	if err != nil {
		log.Fatal(err)
	}

	// Check the connection
	err = client.Ping(context.TODO(), nil)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println("Connected to MongoDB!")

	// Get a handle for your collection
	collection := client.Database("test").Collection("trainers")

	// Some dummy data to add to the Database
	ash := Trainer{"Ash", 10, "Pallet Town"}
	misty := Trainer{"Misty", 10, "Cerulean City"}
	brock := Trainer{"Brock", 15, "Pewter City"}

	// Insert a single document
	insertResult, err := collection.InsertOne(context.TODO(), ash)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println("Inserted a single document: ", insertResult.InsertedID)

	// Insert multiple documents
	trainers := []interface{}{misty, brock}

	insertManyResult, err := collection.InsertMany(context.TODO(), trainers)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println("Inserted multiple documents: ", insertManyResult.InsertedIDs)

	// Update a document
	filter := bson.D{{"name", "Ash"}}

	update := bson.D{
		{"$inc", bson.D{
			{"age", 1},
		}},
	}

	updateResult, err := collection.UpdateOne(context.TODO(), filter, update)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Printf("Matched %v documents and updated %v documents.\n", updateResult.MatchedCount, updateResult.ModifiedCount)

	// Find a single document
	var result Trainer

	err = collection.FindOne(context.TODO(), filter).Decode(&result)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Printf("Found a single document: %+v\n", result)

	findOptions := options.Find()
	findOptions.SetLimit(2)

	var results []*Trainer

	// Finding multiple documents returns a cursor
	cur, err := collection.Find(context.TODO(), bson.D{{}}, findOptions)
	if err != nil {
		log.Fatal(err)
	}

	// Iterate through the cursor
	for cur.Next(context.TODO()) {
		var elem Trainer
		err := cur.Decode(&elem)
		if err != nil {
			log.Fatal(err)
		}

		results = append(results, &elem)
	}

	if err := cur.Err(); err != nil {
		log.Fatal(err)
	}

	// Close the cursor once finished
	cur.Close(context.TODO())

	fmt.Printf("Found multiple documents (array of pointers): %+v\n", results)

	// Delete all the documents in the collection
	deleteResult, err := collection.DeleteMany(context.TODO(), bson.D{{}})
	if err != nil {
		log.Fatal(err)
	}

	fmt.Printf("Deleted %v documents in the trainers collection\n", deleteResult.DeletedCount)

	// Close the connection once no longer needed
	err = client.Disconnect(context.TODO())

	if err != nil {
		log.Fatal(err)
	} else {
		fmt.Println("Connection to MongoDB closed.")
	}

}

5. Redis

[root@localhost redis]# go mod init github.com/my/repo
go: /root/redis/redis/go.mod already exists

 

[root@localhost redis]# go get github.com/go-redis/redis/v8
go: downloading github.com/cespare/xxhash/v2 v2.1.2
go: downloading github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f

 

[root@localhost redis]# yum install gcc

 

[root@localhost redis1]# yum install docker-ce docker-ce-cli containerd.io -y

 

[root@localhost redis1]# systemctl start docker
[root@localhost redis1]# docker pull redis
Using default tag: latest
latest: Pulling from library/redis
a2abf6c4d29d: Pull complete 
c7a4e4382001: Pull complete 
4044b9ba67c9: Pull complete 
c8388a79482f: Pull complete 
413c8bb60be2: Pull complete 
1abfd3011519: Pull complete 
Digest: sha256:db485f2e245b5b3329fdc7eff4eb00f913e09d8feb9ca720788059fdc2ed8339
Status: Downloaded newer image for redis:latest
docker.io/library/redis:latest
[root@localhost redis1]# docker run --name redis-test-instance -p 6379:6379 -d redis
5488bc12b617bf9d8bcca4eb59a011a68d4e16936e28452be32263fc02b861f3
[root@localhost redis1]#

 

package main

import (
        "fmt"

        "encoding/json"
        "github.com/go-redis/redis"
)

type Author struct {
        Name string `json:"name"`
        Age int `json:"age"`
}

func main() {
    client := redis.NewClient(&redis.Options{
                Addr: "localhost:6379",
                Password: "",
                DB: 0,
    })

    json, err := json.Marshal(Author{Name: "Elliot", Age: 25})
    if err != nil {
        fmt.Println(err)
    }

    err = client.Set("id1234", json, 0).Err()
    if err != nil {
        fmt.Println(err)
    }
    val, err := client.Get("id1234").Result()
    if err != nil {
        fmt.Println(err)
    }
    fmt.Println(val)
}

 

[root@localhost redis1]# go get github.com/go-redis/redis

 

[root@localhost redis1]# go env -w GO111MODULE=auto

 

[root@localhost redis1]# go mod init redis

 

[root@localhost redis1]# go mod tidy

 

[root@localhost redis1]# go build

 

[root@localhost redis1]# ./redis
{"name":"Elliot","age":25}

6. Tracing

[root@localhost opentracing-tutorial]# docker run -d -p 6831:6831/udp -p 16686:16686 jaegertracing/all-in-one:latest

 

[root@localhost opentracing-tutorial]# go get github.com/opentracing/opentracing-go

 

[root@localhost opentracing-tutorial]# go get github.com/uber/jaeger-client-go

 

[root@localhost src]# git clone https://github.com/alex-leonhardt/go-trace-example.git
[root@localhost app1]# echo $GOPATH
/root/go

 

[root@localhost app1]# cd $GOPATH/src/go-trace-example/app1
[root@localhost app1]# go run main.go 
2022/01/10 01:03:14 starting...

 

package main

import (
	"context"
	"fmt"
	"io/ioutil"
	"log"
	"math/rand"
	"net/http"
	"time"

	"github.com/opentracing/opentracing-go"
	"github.com/opentracing/opentracing-go/ext"
	"github.com/uber/jaeger-client-go"
	jaegercfg "github.com/uber/jaeger-client-go/config"
	jaegerlog "github.com/uber/jaeger-client-go/log"
	"github.com/uber/jaeger-lib/metrics"
)

func traceF1(tracer opentracing.Tracer) func(w http.ResponseWriter, r *http.Request) {
	return func(w http.ResponseWriter, r *http.Request) {

		spanCtx, _ := tracer.Extract(opentracing.HTTPHeaders, opentracing.HTTPHeadersCarrier(r.Header))
		span := tracer.StartSpan("start", ext.RPCServerOption(spanCtx))
		defer span.Finish()

		span.Context().ForeachBaggageItem(func(k, v string) bool {
			fmt.Println(span, "baggage:", k, v)
			span.LogKV(k, v)
			return true
		})

		ctx, cancel := context.WithTimeout(context.Background(), time.Minute)
		defer cancel()

		ctx = opentracing.ContextWithSpan(ctx, span)

		s, return1, return2 := f1(ctx, tracer)

		w.WriteHeader(http.StatusOK)
		fmt.Fprintf(w, s+return1+return2)
		return
	}
}

func f1(ctx context.Context, tracer opentracing.Tracer) (string, string, string) {

	// span := root.Tracer().StartSpan("f1", opentracing.ChildOf(root.Context()))
	span, ctx := opentracing.StartSpanFromContext(ctx, "f1")
	defer span.Finish()

	sleept := time.Duration(rand.Intn(1120)) * time.Millisecond
	time.Sleep(sleept)

	return1 := f2(ctx)
	return2 := f3(ctx, tracer)

	s := "<html><body>f1 done:"

	return s, return1, return2

}

func f2(ctx context.Context) string {
	span, ctx := opentracing.StartSpanFromContext(ctx, "f2")
	defer span.Finish()

	sleept := time.Duration(rand.Intn(1920)) * time.Millisecond
	span.LogKV("sleep", sleept)
	time.Sleep(sleept)

	return "f2 done:"
}

func f3(ctx context.Context, tracer opentracing.Tracer) string {
	span, ctx := opentracing.StartSpanFromContext(ctx, "f3")
	defer span.Finish()

	url := "http://localhost:8181/"
	req, err := http.NewRequestWithContext(ctx, "GET", url, nil)
	if err != nil {
		ext.Error.Set(span, true)
		span.LogKV("error", err)
		return err.Error()
	}

	span.LogKV("request", req.URL)

	ext.SpanKindRPCClient.Set(span)
	ext.PeerAddress.Set(span, url)
	ext.PeerService.Set(span, "app2")

	ext.HTTPUrl.Set(span, url)
	ext.HTTPMethod.Set(span, "GET")

	client := &http.Client{Timeout: time.Second * 10}
	tracer.Inject(span.Context(), opentracing.HTTPHeaders, opentracing.HTTPHeadersCarrier(req.Header))

	response, err := client.Do(req)
	if err != nil {
		span.LogKV("error", err)
		ext.Error.Set(span, true)
		log.Println(err)
		return err.Error()
	}
	defer response.Body.Close()

	out, err := ioutil.ReadAll(response.Body)
	if err != nil {
		ext.Error.Set(span, true)
		span.LogKV("error", err)
		log.Println(err)
		return err.Error()
	}

	return "f3 done: <br>&nbsp;&nbsp; => " + string(out) + "</body></html>"
}

// ----------

func main() {

	log.Println("starting...")

	cfg := jaegercfg.Configuration{
		ServiceName: "app1",
		Sampler: &jaegercfg.SamplerConfig{
			Type:  jaeger.SamplerTypeConst,
			Param: 1, // trace every call
		},
		Reporter: &jaegercfg.ReporterConfig{
			LogSpans: false,
		},
	}

	jLogger := jaegerlog.StdLogger
	jMetricsFactory := metrics.NullFactory

	tracer, closer, err := cfg.NewTracer(
		jaegercfg.Logger(jLogger),
		jaegercfg.Metrics(jMetricsFactory),
	)

	if err != nil {
		log.Fatalf("could not initialize jaeger tracer: %s", err.Error())
	}
	defer closer.Close()

	opentracing.SetGlobalTracer(tracer)

	f1 := traceF1(tracer)
	http.HandleFunc("/favicon.ico", func(w http.ResponseWriter, r *http.Request) {
		w.WriteHeader(http.StatusOK)
		fmt.Fprintf(w, "")
		return
	})
	http.HandleFunc("/", f1)

	log.Fatalln(http.ListenAndServe(":8080", nil))
}

 

[root@localhost app1]# cd $GOPATH/src/go-trace-example/app2
[root@localhost app2]# go run main.go 
2022/01/10 01:03:31 starting...
2022/01/10 01:03:31 Initializing logging reporter
package main

import (
	"context"
	"fmt"
	"log"
	"math/rand"
	"net/http"
	"time"

	"github.com/opentracing/opentracing-go"
	"github.com/opentracing/opentracing-go/ext"
	"github.com/uber/jaeger-client-go"
	jaegercfg "github.com/uber/jaeger-client-go/config"
	jaegerlog "github.com/uber/jaeger-client-go/log"
	"github.com/uber/jaeger-lib/metrics"
)

func traceF1(tracer opentracing.Tracer) func(w http.ResponseWriter, r *http.Request) {
	return func(w http.ResponseWriter, r *http.Request) {

		spanCtx, err := tracer.Extract(opentracing.HTTPHeaders, opentracing.HTTPHeadersCarrier(r.Header))
		if err != nil {
			log.Println(err)
		}
		span := tracer.StartSpan("start", ext.RPCServerOption(spanCtx))
		defer span.Finish()

		ctx, cancel := context.WithTimeout(context.Background(), time.Minute)
		defer cancel()

		ctx = opentracing.ContextWithSpan(ctx, span)
		s, return1 := f1(ctx)

		w.WriteHeader(http.StatusOK)
		fmt.Fprintf(w, s+return1)
		return
	}
}

func f1(ctx context.Context) (string, string) {

	span, ctx := opentracing.StartSpanFromContext(ctx, "f1")
	defer span.Finish()

	sleept := time.Duration(rand.Intn(1120)) * time.Millisecond
	span.LogKV("sleep", sleept)

	time.Sleep(sleept)

	return1 := f2(ctx)

	s := "f1 done:"

	return s, return1

}

func f2(ctx context.Context) string {
	span, ctx := opentracing.StartSpanFromContext(ctx, "f2")
	defer span.Finish()

	sleept := time.Duration(rand.Intn(1920)) * time.Millisecond
	span.LogKV("sleep", sleept)

	time.Sleep(sleept)

	return "f2 done:"
}

// ----------

func main() {

	log.Println("starting...")

	cfg := jaegercfg.Configuration{
		ServiceName: "app2",
		Sampler: &jaegercfg.SamplerConfig{
			Type:  jaeger.SamplerTypeConst,
			Param: 1,
		},
		Reporter: &jaegercfg.ReporterConfig{
			LogSpans: true,
		},
	}

	jLogger := jaegerlog.StdLogger
	jMetricsFactory := metrics.NullFactory

	tracer, closer, err := cfg.NewTracer(
		jaegercfg.Logger(jLogger),
		jaegercfg.Metrics(jMetricsFactory),
	)

	if err != nil {
		log.Fatalf("could not initialize jaeger tracer: %s", err.Error())
	}
	defer closer.Close()

	opentracing.SetGlobalTracer(tracer)

	f1 := traceF1(tracer)
	http.HandleFunc("/favicon.ico", func(w http.ResponseWriter, r *http.Request) {
		w.WriteHeader(http.StatusOK)
		fmt.Fprintf(w, "")
		return
	})
	http.HandleFunc("/", f1)

	log.Fatalln(http.ListenAndServe(":8181", nil))
}

 

[root@localhost app2]# curl -H "jaeger-debug-id: some-correlation-id" -H "jaeger-baggage: key=value,key2=value2" http://localhost:8080
<html><body>f1 done:f2 done:f3 done: <br>&nbsp;&nbsp; => f1 done:f2 done:</body></html>[root@localhost app2]#

 

[root@localhost app2]# 2022/01/10 01:03:31 starting...
2022/01/10 01:03:31 Initializing logging reporter
117cabdd8745b9d5:117cabdd8745b9d5:0000000000000000:3 baggage: key value
117cabdd8745b9d5:117cabdd8745b9d5:0000000000000000:3 baggage: key2 value2
2022/01/10 01:04:15 Reporting span 117cabdd8745b9d5:62fb9121cdaff8d0:21de0eef92769469:3
2022/01/10 01:04:15 Reporting span 117cabdd8745b9d5:21de0eef92769469:30f888cbbb0678a9:3
2022/01/10 01:04:15 Reporting span 117cabdd8745b9d5:30f888cbbb0678a9:4445c6fdbc8fba47:3

 

 

 

 

 

 

 

 

 

 

 

 

티스토리툴바