Getting started with Envoy, SPIFFE, and Kubernetes

This guide will get you started with SPIRE and Envoy SDS by walking through the deployment and configuration of an edge Envoy proxy and an Envoy sidecar in front of a simple app, configured to communicate with each other using SPIRE for mTLS.

A quick intro

SPIFFE, the Secure Production Identity Framework for Everyone, defines a set of standards to provide secure identities to individual workloads and can be leveraged to create a zero trust security model within your microservice architecture. The SPIFFE runtime environment, SPIRE, is available to implement it.

The setup

Prerequisites

To follow this guide, have the following:

  1. Clone the repo for this guide:

Step 1: Install SPIRE

Note on the SPIRE Kubernetes Workload Registrar

For this deployment, we will use the Kubernetes Workload Registrar service provided by SPIRE.

Configuration

There are many complex internals for the server and agent configurations that I won’t go into detail on in this post — but the SPIRE project provides docs, examples, and information on each element. See here for full server configuration docs, and here for the agent.

  1. pod_label: configured for the registrar service and determines if a SPIFFE identity is created for a new pod and (if so) the second piece of the format
spiffe://quickstart.spire.io/example-service

Install

With the above configurations noted, install the server:

kubectl apply -f spire/server_template.yaml
kubectl apply -f spire/agent_template.yaml

Step 2: Install services

Configuration

For this example, we will create a deployment running an edge Envoy proxy for ingress, and a deployment running a hello world service with a sidecar. To create these proxy images, I followed this guide from Mark Vinzce and pushed the docker images used in the deployment files.

- name: spire_agent
connect_timeout: 0.25s
http2_protocol_options: {}
load_assignment:
cluster_name: spire_agent
endpoints:
- lb_endpoints:
- endpoint:
address:
pipe:
path: /run/spire/socket/agent.sock

Edge proxy

The full edge Envoy proxy configuration can be found here. The important piece to talk through for this guide is the SDS configuration.

spiffe://quickstart.spire.io/edge-proxy
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
common_tls_context:
tls_certificate_sds_secret_configs:
- name: spiffe://quickstart.spire.io/edge-proxy
sds_config:
api_config_source:
api_type: gRPC
grpc_services:
envoy_grpc:
cluster_name: spire_agent
combined_validation_context:
default_validation_context:
match_subject_alt_names:
exact: spiffe://quickstart.spire.io/helloworld
validation_context_sds_secret_config:
name: spiffe://quickstart.spire.io
sds_config:
api_config_source:
api_type: gRPC
grpc_services:
envoy_grpc:
cluster_name: spire_agent
tls_params:
ecdh_curves:
- X25519:P-256:P-521:P-384

Sidecar

The second Envoy proxy runs in the same pod as a simple helloworld app. All communication to the app will go through the proxy. The full configuration for this proxy can be found here.

spiffe://quickstart.spire.io/helloworld
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
common_tls_context:
tls_certificate_sds_secret_configs:
- name: spiffe://quickstart.spire.io/helloworld
sds_config:
api_config_source:
api_type: gRPC
grpc_services:
envoy_grpc:
cluster_name: spire_agent
combined_validation_context:
default_validation_context:
match_subject_alt_names:
exact: spiffe://quickstart.spire.io/edge-proxy
validation_context_sds_secret_config:
name: spiffe://quickstart.spire.io
sds_config:
api_config_source:
api_type: gRPC
grpc_services:
envoy_grpc:
cluster_name: spire_agent
tls_params:
ecdh_curves:
- X25519:P-256:P-521:P-384

Install

Install both deployments:

kubectl apply -f services/edge-deployment.yamlkubectl apply -f services/backend-deployment.yaml

Testing

Grab the external IP address for your ingress load balancer from kubectl get svc edge-proxy .

Hello, world!
Version: 1.0.0
Hostname: helloworld-56b5668bc5-tpgkr
  • curl localhost:8001/certs to see the certificates for the proxy — they will be the SPIFFE certificates with that proxies identity
  • curl localhost:8001/stats to see statistics — grep for ssl for security specific stats

Final Thoughts

If you’ve made it this far, thanks for following along! Hopefully someone may have found this helpful. Feel free to reach out with thoughts/feedback — I can be found in the Envoy slack.

References

Developer at @GreymatterIo