Go Kit: A Microservice Toolkit for Go
Go Kit is a powerful microservice toolkit for Go that provides a comprehensive set of tools and patterns for building distributed systems. With over 27,561 stars on GitHub, it has become the de facto standard for Go microservices, offering solutions for service discovery, load balancing, pluggable transports, request tracking, and more. This library is essential for developers building scalable, maintainable microservices because it addresses the complexity of distributed systems through well-designed abstractions and patterns.
Go Kit solves real-world problems that every microservice developer faces: how to structure services for testability, how to handle communication between services reliably, how to implement observability, and how to manage cross-cutting concerns like logging, metrics, and authentication. Whether you’re building a small internal service or a large-scale distributed system, Go Kit provides the foundation you need to create robust, production-ready microservices.
Key Features
Service Interface Design: Go Kit enforces a clean separation between your business logic and the transport layer through its endpoint and service patterns. This makes your services inherently testable and allows you to switch between different transports (HTTP, gRPC, etc.) without changing your core logic.
Transport Abstraction: The library provides pluggable transport layers that allow you to expose your services via HTTP, gRPC, or other protocols. This abstraction means you can start with HTTP for simplicity and later add gRPC for performance without rewriting your business logic.
Middleware Ecosystem: Go Kit’s middleware system lets you add cross-cutting concerns like logging, metrics, circuit breaking, and authentication as stackable layers around your endpoints. This follows the decorator pattern and keeps your business logic clean.
Service Discovery and Load Balancing: Built-in support for service discovery (Consul, etcd) and client-side load balancing ensures your services can find and communicate with each other reliably in dynamic environments.
Request Tracing: Integration with OpenTracing and OpenTelemetry provides end-to-end request tracing across service boundaries, crucial for debugging distributed systems.
Circuit Breaking: Implementation of the circuit breaker pattern prevents cascading failures and allows your system to gracefully handle downstream service issues.
Context Propagation: Proper context handling throughout the stack enables request-scoped values, cancellation, and deadlines to flow naturally through your service calls.
Installation and Setup
To install Go Kit, use the following command:
1go get github.com/go-kit/kit/...
Go Kit requires Go 1.16 or later. The library is organized into multiple subpackages, so the ... ensures you get all available components. No additional configuration is needed after installation.
To verify the installation, create a simple test file:
1package main
2
3import (
4 "github.com/go-kit/kit/endpoint"
5 "fmt"
6)
7
8func main() {
9 var e endpoint.Endpoint
10 fmt.Println("Go Kit installed successfully!")
11}
Run go run main.go to confirm everything is working correctly.
Basic Usage
Let’s start with a simple “Hello World” example that demonstrates the core patterns of Go Kit. This example shows how to create a service, define an endpoint, and expose it over HTTP.
1package main
2
3import (
4 "context"
5 "encoding/json"
6 "fmt"
7 "log"
8 "net/http"
9 "os"
10 "os/signal"
11 "syscall"
12
13 "github.com/go-kit/kit/endpoint"
14 httptransport "github.com/go-kit/kit/transport/http"
15)
16
17// Service defines the business logic interface
18type Service interface {
19 SayHello(ctx context.Context, name string) (string, error)
20}
21
22// service implements the Service interface
23type service struct{}
24
25func (s service) SayHello(ctx context.Context, name string) (string, error) {
26 if name == "" {
27 return "", fmt.Errorf("name is required")
28 }
29 return "Hello, " + name + "!", nil
30}
31
32// request and response structs for the endpoint
33type helloRequest struct {
34 Name string `json:"name"`
35}
36
37type helloResponse struct {
38 Greeting string `json:"greeting"`
39 Err string `json:"err,omitempty"`
40}
41
42// makeHelloEndpoint creates an endpoint for the SayHello method
43func makeHelloEndpoint(svc Service) endpoint.Endpoint {
44 return func(ctx context.Context, request interface{}) (interface{}, error) {
45 req := request.(helloRequest)
46 greeting, err := svc.SayHello(ctx, req.Name)
47 if err != nil {
48 return helloResponse{Greeting: ""}, err
49 }
50 return helloResponse{Greeting: greeting}, nil
51 }
52}
53
54func main() {
55 // Create the service
56 svc := service{}
57
58 // Create the endpoint
59 helloHandler := httptransport.NewServer(
60 makeHelloEndpoint(svc),
61 func(_ context.Context, r *http.Request) (interface{}, error) {
62 var req helloRequest
63 if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
64 return helloRequest{}, err
65 }
66 return req, nil
67 },
68 func(_ context.Context, w http.ResponseWriter, response interface{}) error {
69 return json.NewEncoder(w).Encode(response)
70 },
71 )
72
73 // Setup HTTP server
74 errChan := make(chan error)
75 go func() {
76 c := make(chan os.Signal, 1)
77 signal.Notify(c, syscall.SIGINT, syscall.SIGTERM)
78 errChan <- fmt.Errorf("%s", <-c)
79 }()
80
81 go func() {
82 fmt.Println("Starting server at :8080")
83 errChan <- http.ListenAndServe(":8080", helloHandler)
84 }()
85
86 log.Fatalln(<-errChan)
87}
Expected output when running the server and making a request:
1$ go run main.go
2Starting server at :8080
3
4$ curl -X POST http://localhost:8080 \
5 -H "Content-Type: application/json" \
6 -d '{"name":"World"}'
7{"greeting":"Hello, World!"}
Real-World Examples
Example 1: Complete User Service with Middleware
This example demonstrates a production-ready user service with multiple endpoints, middleware for logging and authentication, and proper error handling.
1package main
2
3import (
4 "context"
5 "encoding/json"
6 "fmt"
7 "log"
8 "net/http"
9 "os"
10 "os/signal"
11 "syscall"
12 "time"
13
14 "github.com/go-kit/kit/auth/jwt"
15 "github.com/go-kit/kit/endpoint"
16 httptransport "github.com/go-kit/kit/transport/http"
17 "github.com/go-kit/kit/log"
18 "github.com/go-kit/kit/log/level"
19 "github.com/go-kit/kit/transport/http/jsonrpc"
20)
21
22// User represents a user in the system
23type User struct {
24 ID string `json:"id"`
25 Email string `json:"email"`
26 Password string `json:"-"`
27}
28
29// UserService defines the business logic interface
30type UserService interface {
31 CreateUser(ctx context.Context, email, password string) (*User, error)
32 GetUser(ctx context.Context, id string) (*User, error)
33 UpdateUser(ctx context.Context, id, email string) (*User, error)
34 DeleteUser(ctx context.Context, id string) error
35}
36
37// userService implements the UserService interface
38type userService struct {
39 users map[string]User
40 log log.Logger
41}
42
43func (s *userService) CreateUser(ctx context.Context, email, password string) (*User, error) {
44 level.Info(s.log).Log("method", "CreateUser", "email", email)
45
46 if email == "" || password == "" {
47 return nil, fmt.Errorf("email and password are required")
48 }
49
50 user := &User{
51 ID: fmt.Sprintf("%d", time.Now().UnixNano()),
52 Email: email,
53 Password: password,
54 }
55 s.users[user.ID] = *user
56 return user, nil
57}
58
59func (s *userService) GetUser(ctx context.Context, id string) (*User, error) {
60 level.Info(s.log).Log("method", "GetUser", "id", id)
61
62 if id == "" {
63 return nil, fmt.Errorf("user ID is required")
64 }
65
66 user, exists := s.users[id]
67 if !exists {
68 return nil, fmt.Errorf("user not found")
69 }
70 return &user, nil
71}
72
73func (s *userService) UpdateUser(ctx context.Context, id, email string) (*User, error) {
74 level.Info(s.log).Log("method", "UpdateUser", "id", id, "email", email)
75
76 if id == "" || email == "" {
77 return nil, fmt.Errorf("user ID and email are required")
78 }
79
80 user, exists := s.users[id]
81 if !exists {
82 return nil, fmt.Errorf("user not found")
83 }
84
85 user.Email = email
86 s.users[id] = user
87 return &user, nil
88}
89
90func (s *userService) DeleteUser(ctx context.Context, id string) error {
91 level.Info(s.log).Log("method", "DeleteUser", "id", id)
92
93 if id == "" {
94 return fmt.Errorf("user ID is required")
95 }
96
97 delete(s.users, id)
98 return nil
99}
100
101// request and response structs
102type createUserRequest struct {
103 Email string `json:"email"`
104 Password string `json:"password"`
105}
106
107type createUserResponse struct {
108 User *User `json:"user"`
109 Err string `json:"err,omitempty"`
110}
111
112type getUserRequest struct {
113 ID string `json:"id"`
114}
115
116type getUserResponse struct {
117 User *User `json:"user"`
118 Err string `json:"err,omitempty"`
119}
120
121type updateUserRequest struct {
122 ID string `json:"id"`
123 Email string `json:"email"`
124}
125
126type updateUserResponse struct {
127 User *User `json:"user"`
128 Err string `json:"err,omitempty"`
129}
130
131type deleteUserRequest struct {
132 ID string `json:"id"`
133}
134
135type deleteUserResponse struct {
136 Err string `json:"err,omitempty"`
137}
138
139// Endpoint creation functions
140func makeCreateUserEndpoint(svc UserService) endpoint.Endpoint {
141 return func(ctx context.Context, request interface{}) (interface{}, error) {
142 req := request.(createUserRequest)
143 user, err := svc.CreateUser(ctx, req.Email, req.Password)
144 if err != nil {
145 return createUserResponse{User: nil, Err: err.Error()}, nil
146 }
147 return createUserResponse{User: user, Err: ""}, nil
148 }
149}
150
151func makeGetUserEndpoint(svc UserService) endpoint.Endpoint {
152 return func(ctx context.Context, request interface{}) (interface{}, error) {
153 req := request.(getUserRequest)
154 user, err := svc.GetUser(ctx, req.ID)
155 if err != nil {
156 return getUserResponse{User: nil, Err: err.Error()}, nil
157 }
158 return getUserResponse{User: user, Err: ""}, nil
159 }
160}
161
162func makeUpdateUserEndpoint(svc UserService) endpoint.Endpoint {
163 return func(ctx context.Context, request interface{}) (interface{}, error) {
164 req := request.(updateUserRequest)
165 user, err := svc.UpdateUser(ctx, req.ID, req.Email)
166 if err != nil {
167 return updateUserResponse{User: nil, Err: err.Error()}, nil
168 }
169 return updateUserResponse{User: user, Err: ""}, nil
170 }
171}
172
173func makeDeleteUserEndpoint(svc UserService) endpoint.Endpoint {
174 return func(ctx context.Context, request interface{}) (interface{}, error) {
175 req := request.(deleteUserRequest)
176 err := svc.DeleteUser(ctx, req.ID)
177 if err != nil {
178 return deleteUserResponse{Err: err.Error()}, nil
179 }
180 return deleteUserResponse{Err: ""}, nil
181 }
182}
183
184// Authentication middleware
185func authMiddleware(jwtKey string) endpoint.Middleware {
186 return func(next endpoint.Endpoint) endpoint.Endpoint {
187 return func(ctx context.Context, request interface{}) (interface{}, error) {
188 token := jwt.FromContext(ctx)
189 if token == "" {
190 return nil, fmt.Errorf("missing authentication token")
191 }
192
193 // Validate JWT token (simplified)
194 if token != jwtKey {
195 return nil, fmt.Errorf("invalid authentication token")
196 }
197
198 return next(ctx, request)
199 }
200 }
201}
202
203// Logging middleware
204func loggingMiddleware(logger log.Logger) endpoint.Middleware {
205 return func(next endpoint.Endpoint) endpoint.Endpoint {
206 return func(ctx context.Context, request interface{}) (interface{}, error) {
207 level.Info(logger).Log("msg", "request received")
208 defer level.Info(logger).Log("msg", "response sent")
209 return next(ctx, request)
210 }
211 }
212}
213
214func main() {
215 // Setup logging
216 logger := log.NewLogfmtLogger(log.NewSyncWriter(os.Stderr))
217 logger = level.NewFilter(logger, level.AllowInfo())
218 logger = log.With(logger, "ts", log.DefaultTimestampUTC, "caller", log.DefaultCaller)
219
220 // Create service
221 svc := &userService{
222 users: make(map[string]User),
223 log: logger,
224 }
225
226 // Create endpoints
227 createEndpoint := makeCreateUserEndpoint(svc)
228 getEndpoint := makeGetUserEndpoint(svc)
229 updateEndpoint := makeUpdateUserEndpoint(svc)
230 deleteEndpoint := makeDeleteUserEndpoint(svc)
231
232 // Apply middleware
233 jwtKey := "supersecretkey"
234 createEndpoint = loggingMiddleware(logger)(createEndpoint)
235 getEndpoint = authMiddleware(jwtKey)(loggingMiddleware(logger)(getEndpoint))
236 updateEndpoint = authMiddleware(jwtKey)(loggingMiddleware(logger)(updateEndpoint))
237 deleteEndpoint = authMiddleware(jwtKey)(loggingMiddleware(logger)(deleteEndpoint))
238
239 // Create HTTP handlers
240 createHandler := httptransport.NewServer(
241 createEndpoint,
242 func(_ context.Context, r *http.Request) (interface{}, error) {
243 var req createUserRequest
244 if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
245 return nil, err
246 }
247 return req, nil
248 },
249 func(_ context.Context, w http.ResponseWriter, response interface{}) error {
250 w.Header().Set("Content-Type", "application/json; charset=utf-8")
251 return json.NewEncoder(w).Encode(response)
252 },
253 )
254
255 getHandler := httptransport.NewServer(
256 getEndpoint,
257 func(_ context.Context, r *http.Request) (interface{}, error) {
258 var req getUserRequest
259 req.ID = r.URL.Query().Get("id")
260 return req, nil
261 },
262 func(_ context.Context, w http.ResponseWriter, response interface{}) error {
263 w.Header().Set("Content-Type", "application/json; charset=utf-8")
264 return json.NewEncoder(w).Encode(response)
265 },
266 )
267
268 updateHandler := httptransport.NewServer(
269 updateEndpoint,
270 func(_ context.Context, r *http.Request) (interface{}, error) {
271 var req updateUserRequest
272 if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
273 return nil, err
274 }
275 return req, nil
276 },
277 func(_ context.Context, w http.ResponseWriter, response interface{}) error {
278 w.Header().Set("Content-Type", "application/json; charset=utf-8")
279 return json.NewEncoder(w).Encode(response)
280 },
281 )
282
283 deleteHandler := httptransport.NewServer(
284 deleteEndpoint,
285 func(_ context.Context, r *http.Request) (interface{}, error) {
286 var req deleteUserRequest
287 req.ID = r.URL.Query().Get("id")
288 return req, nil
289 },
290 func(_ context.Context, w http.ResponseWriter, response interface{}) error {
291 w.Header().Set("Content-Type", "application/json; charset=utf-8")
292 return json.NewEncoder(w).Encode(response)
293 },
294 )
295
296 // Setup HTTP server
297 errChan := make(chan error)
298 go func() {
299 c := make(chan os.Signal, 1)
300 signal.Notify(c, syscall.SIGINT, syscall.SIGTERM)
301 errChan <- fmt.Errorf("%s", <-c)
302 }()
303
304 go func() {
305 mux := http.NewServeMux()
306 mux.Handle("/create", createHandler)
307 mux.Handle("/get", getHandler)
308 mux.Handle("/update", updateHandler)
309 mux.Handle("/delete", deleteHandler)
310
311 fmt.Println("Starting server at :8080")
312 errChan <- http.ListenAndServe(":8080", mux)
313 }()
314
315 log.Fatalln(<-errChan)
316}
Expected usage:
1# Create user (no auth required)
2curl -X POST http://localhost:8080/create \
3 -H "Content-Type: application/json" \
4 -d '{"email":"[email protected]","password":"secret"}'
5
6# Get user (requires auth)
7curl -X GET http://localhost:8080/get?id=1234567890 \
8 -H "Authorization: Bearer supersecretkey"
9
10# Update user (requires auth)
11curl -X POST http://localhost:8080/update \
12 -H "Content-Type: application/json" \
13 -H "Authorization: Bearer supersecretkey" \
14 -d '{"id":"1234567890","email":"[email protected]"}'
15
16# Delete user (requires auth)
17curl -X GET http://localhost:8080/delete?id=1234567890 \
18 -H "Authorization: Bearer supersecretkey"
Example 2: Service with Circuit Breaker and Rate Limiting
This example shows how to integrate resilience patterns using Go Kit’s middleware ecosystem.
1package main
2
3import (
4 "context"
5 "encoding/json"
6 "fmt"
7 "log"
8 "net/http"
9 "os"
10 "os/signal"
11 "syscall"
12 "time"
13
14 "github.com/go-kit/kit/circuitbreaker"
15 "github.com/go-kit/kit/endpoint"
16 "github.com/go-kit/kit/ratelimit"
17 "github.com/go-kit/kit/transport/http"
18 "github.com/sony/gobreaker"
19 httptransport "github.com/go-kit/kit/transport/http"
20)
21
22// PaymentService defines a payment processing service
23type PaymentService interface {
24 ProcessPayment(ctx context.Context, amount float64, currency string) (string, error)
25 GetPaymentStatus(ctx context.Context, paymentID string) (string, error)
26}
27
28// paymentService implements the PaymentService interface
29type paymentService struct {
30 log log.Logger
31}
32
33func (s *paymentService) ProcessPayment(ctx context.Context, amount float64, currency string) (string, error) {
34 s.log.Log("method", "ProcessPayment", "amount", amount, "currency", currency)
35
36 // Simulate payment processing
37 if amount <= 0 {
38 return "", fmt.Errorf("amount must be positive")
39 }
40
41 // Simulate external service call
42 time.Sleep(100 * time.Millisecond)
43
44 return fmt.Sprintf("payment-%d", time.Now().UnixNano()), nil
45}
46
47func (s *paymentService) GetPaymentStatus(ctx context.Context, paymentID string) (string, error) {
48 s.log.Log("method", "GetPaymentStatus", "paymentID", paymentID)
49
50 if paymentID == "" {
51 return "", fmt.Errorf("payment ID is required")
52 }
53
54 // Simulate status check
55 time.Sleep(50 * time.Millisecond)
56
57 return "completed", nil
58}
59
60// Request and response structs
61type processPaymentRequest struct {
62 Amount float64 `json:"amount