SRE:3节点部署TiKV用于测试功能

源起

闲置几台屌丝版腾讯云服务器(2 core CPU 4GB Memory 40GB Disk),难得清闲,用其中的3台部署一套TiKV用于测试功能和代码研究。

节点分布

使用Docker进行部署,官方镜像pingcap/pd:v2.0.6、pingcap/tikv:v2.0.6

|    Name   |   Host IP   |  Services  |  Docker Volume  |  Data Path  |
| Node1(TB) | 172.22.0.6  |    PD1     |     pd-data     |    /data    |
| Node2(TD) | 172.22.0.10 |    PD2     |     pd-data     |    /data    |
| Node3(TE) | 172.22.0.15 |    PD3     |     pd-data     |    /data    |
| Node1(TB) | 172.22.0.6  |   TiKV1    |     tikv-data   |    /data    |
| Node2(TD) | 172.22.0.10 |   TiKV2    |     tikv-data   |    /data    |
| Node3(TE) | 172.22.0.15 |   TiKV3    |     tikv-data   |    /data    |

Go modules

§Definition

A module is a collection of related go packages. Modules are the unit of source code interchange and versionning.

§Quick history

  • Go before 1.5: populating GOPATH with go get.
  • Go 1.5 and after: dependency vendoring is introduced.
  • vgo is proposed as a prototype for Go modules support.
  • Go 1.11 (beta): vgo is being merged and refined as go mod (experimental).

§Terminology

This article refers to recurrent expressions. Let’s clarify them:

  • “Module root”: the directory containing the file named go.mod.
  • “Module path”: the import path prefix corresponding to the module root.
  • “Main module”: the module containing the directory where the go command is run.

§Module structure

A module is a tree of Go source files to which is added a file named go.mod. It contains the module import name, and the declaration of dependency requirements, exclusions and replacements. Its content would look like this:

module my/thing
  
require (
        one/thing v1.3.2
        other/thing v2.5.0 // indirect
        ...
)

exclude (
        bad/thing v0.7.3
)

replace (
        src/thing 1.0.2 => dst/thing v1.1.0
)

Guide: gorilla/mux

The name mux stands for “HTTP request multiplexer”. Like the standard http.ServeMux, mux.Router matches incoming requests against a list of registered routes and calls a handler for the route that matches the URL or other conditions. The main features are:

* Requests can be matched based on URL host, path, path prefix, schemes,
  header and query values, HTTP methods or using custom matchers.
* URL hosts, paths and query values can have variables with an optional
  regular expression.
* Registered URLs can be built, or "reversed", which helps maintaining
  references to resources.
* Routes can be used as subrouters: nested routes are only tested if the
  parent route matches. This is useful to define groups of routes that
  share common conditions like a host, a path prefix or other repeated
  attributes. As a bonus, this optimizes request matching.
* It implements the http.Handler interface so it is compatible with the
  standard http.ServeMux.

Let’s start registering a couple of URL paths and handlers:

func main() {
  r := mux.NewRouter()
  r.HandleFunc("/", HomeHandler)
  r.HandleFunc("/products", ProductsHandler)
  r.HandleFunc("/articles", ArticlesHandler)
  http.Handle("/", r)
  log.Fatal(http.ListenAndServe(":12345", nil))
}

Docker: 对Docker Remote API进行认证

  • 建立证书授权中心
$ sudo mkdir /etc/docker
$ cd /etc/docker
$ echo 01 | sudo tee ca.csl
$ sudo openssl genrsa -des3 -out ca-key.pem
$ sudo openssl req -new -x509 -days 365 -key ca-key.pem -out ca.pem
  • 创建服务器的证书签名请求和密钥
$ sudo openssl genrsa -des3 -out server-key.pem
$ sudo openssl req -new -key server-key.pem -out server.csr
$ sudo openssl x509 -req -days 365 -in server.csr -CA ca.pem -CAkey ca-key.pem -out server-cert.pem
$ sudo openssl rsa -in server-key.pem -out server-key.pem
$ sudo chmod 0600 /etc/docker/server-key.pem /etc/docker/server-cert.pem /etc/docker/ca-key.pem /etc/docker/ca.pem
  • 配置Docker守护进程 (/etc/docker/daemon.json on Linux systems, or C:\ProgramData\docker\config\daemon.json on Windows.)
{
  "debug": true,
  "tls": true,
  "tlscacert": "/etc/docker/ca.pem",
  "tlscert": "/etc/docker/server-cert.pem",
  "tlskey": "/etc/docker/server-key.pem",
  "hosts": ["tcp://<config of CN>:2376"]
}

Gogs: PR-5322

Add new Dockerfile.docker-ce for docker-ce(>=v17.06) to build Gogs’s docker image

Docker-CE can be given to a new build stage by adding AS name to theFROM instruction sine release version of v17.06. The Dockerfile’s FROM instruction like below:

FROM

FROM <image> [AS <name>]

Or

FROM <image>[:<tag>] [AS <name>]

Or

FROM <image>[@<digest>] [AS <name>]
  • Optionally a name can be given to a new build stage by adding AS name to the FROM instruction. The name can be used in subsequent FROM and COPY --from=<name|index> instructions to refer to the image built in this stage.

Find Docker-ce official document here.

Gogs: PR-5262

Fix make build failure when enviroment of GOPATH have multiple items

[alimy@rover gogs]$ pwd
/home/alimy/art/arg/src/github.com/gogs/gogs
[alimy@rover gogs]$ echo $GOPATH
/home/alimy/art/ago:/home/alimy/art/arg
[alimy@rover gogs]$ make
go install "-v" -ldflags '-X "github.com/gogs/gogs/pkg/setting.BuildTime=2018-06-04 06:17:19 UTC" -X "github.com/gogs/gogs/pkg/setting.BuildGitHash=c08aab90ec696b7fcc56b8da0a468e74d266b89e"' -tags '""'
cp '/home/alimy/art/ago:/home/alimy/art/arg/bin/gogs' .
cp: cannot stat '/home/alimy/art/ago:/home/alimy/art/arg/bin/gogs': No such file or directory
Makefile:36: recipe for target 'build' failed
make: *** [build] Error 1

In this scene GOPATH have two item (/home/alimy/art/ago and /home/alimy/art/arg) and gogs source is not in first GOPATH items, when excecute go install ... will install to path that contain the source of gogs’s GOPATH items. when cp gogs file back will occur error like above. this patch fixed this problem.

[alimy@rover gogs]$ echo $GOPATH
/home/alimy/art/ago:/home/alimy/art/arg
[alimy@rover gogs]$ pwd
/home/alimy/art/arg/src/github.com/gogs/gogs
[alimy@rover gogs]$ echo ${PWD%%src*}
/home/alimy/art/arg/

NewSQL: 分布式数据库TiDB、CockroachDB

TiDB

TiDB 开源分布式 NewSQL 关系型数据库 TiDB 是新一代开源分布式 NewSQL 数据库,模型受 Google Spanner / F1 论文的启发,实现了自动的水平伸缩,强一致性的分布式事务,基于 Raft 算法的多副本复制等重要 NewSQL 特性。TiDB 结合了 RDBMS 和 NoSQL 的优点,部署简单,在线弹性扩容和异步表结构变更不影响业务, 真正的异地多活及自动故障恢复保障数据安全,同时兼容 MySQL 协议,使迁移使用成本降到极低。

CockroachDB (蟑螂DB/小强DB)

CockroachDB(中文名蟑螂DB,所以又可以称为小强DB),是构建于事务处理及强一致性KV存储上的分布式SQL数据库,支持水平扩展、自动容错处理、强一致性事务,并且提供SQL接口用于数据处理,是Google Spanner/F1的开源实现。 CockroachDB适用于应用对数据要求精确、可靠、完全正确的场景,支持自动复制、均匀分布、基于极小配置的数据恢复,可用于分布式的、可复制的联机事务处理(OLTP),多数据中心的部署,私有云的基础构建,它不适用于读少写多的场景,可以用内存数据库来代替,也不适用于复杂的join查询,重量级的数据分析及联机分析处理(OLAP)。


Archive