Simplifying Ingress Management for Kubernetes: Deploying a Traefik Cluster with Automatic TLS
Fri, Jan 10, 2025Managing ingress traffic in a Kubernetes cluster is a critical aspect of ensuring the accessibility and security of your applications. This guide will walk you through the process of deploying a Traefik cluster on Kuberntes, including the setup of automatic TLS using Let's Encrypt.
Prerequisites
Before proceeding, ensure you have the following:
-
Ensure you have a Kubernetes cluster set up.
-
Install Helm for package management.
-
Have a domain name that resolves to the public IP of your Kubernetes cluster.
Setting Up the Kubernetes Cluster
If you don't already have a Kubernetes cluster, you can set one up using your preferred method (e.g., using Minikube, kind, or any other Kubernetes distribution).
Installing Helm
To manage packages in your Kubernetes cluster, you need Helm. Here’s how to install it:
curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3
chmod 700 get_helm.sh
./get_helm.sh
Installing Traefik via Helm
To install Traefik using Helm, you need to configure the traefik-values.yaml file. Here is an example configuration that includes Let's Encrypt for automatic TLS:
# traefik-values.yaml
logs:
general:
level: DEBUG
service:
type: LoadBalancer
persistence:
enabled: true
certificatesResolvers:
letsencrypt:
acme:
email: "[email protected]"
storage: "traefik-acme.json"
keyType: "RSA4096"
tlsChallenge: {}
You can override the acme.email field directly in the helm install command if needed:
helm repo add traefik https://helm.traefik.io/traefik
helm repo update
helm install traefik traefik/traefik --set [email protected]
Configuring DNS for Traefik
After installing Traefik, you need to set up a DNS name for the public IP of the Traefik controller.
# Get the public IP of the Traefik service
PUBLIC_IP=$(kubectl get svc traefik -n kube-system -o jsonpath='{.status.loadBalancer.ingress.hostname}')
# Update the DNS name for the public IP
DNSNAME=$(az network public-ip show --ids $(kubectl get svc traefik -n kube-system -o jsonpath='{.status.loadBalancer.ingress.hostname}' | cut -d '.' -f 1) --query dnsSettings.fqdn -o tsv)
echo "DNSNAME: $DNSNAME"
Deploying a Sample Application
To demonstrate the functionality of Traefik, you can deploy a sample application. Here is an example of how to deploy the azure-vote-app:
# azure-vote-app.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: azure-vote-back
spec:
replicas: 1
selector:
matchLabels:
app: azure-vote-back
template:
metadata:
labels:
app: azure-vote-back
spec:
containers:
- name: azure-vote-back
image: mcr.microsoft.com/oss/nginx/nginx:1.15.5-alpine
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: azure-vote-back
spec:
selector:
app: azure-vote-back
ports:
- name: http
port: 80
targetPort: 80
type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: azure-vote-front
spec:
replicas: 1
selector:
matchLabels:
app: azure-vote-front
template:
metadata:
labels:
app: azure-vote-front
spec:
containers:
- name: azure-vote-front
image: mcr.microsoft.com/oss/nginx/nginx:1.15.5-alpine
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: azure-vote-front
spec:
selector:
app: azure-vote-front
ports:
- name: http
port: 80
targetPort: 80
type: ClusterIP
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: azure-vote-ingress
annotations:
traefik.ingress.kubernetes.io/router.tls.certresolver: letsencrypt
traefik.ingress.kubernetes.io/router.entrypoints: websecure
spec:
rules:
- host: ${DNSNAME}
http:
paths:
- path: /
pathType: Exact
backend:
service:
name: azure-vote-front
port:
number: 80
Update the host field in the Ingress resource to match your Traefik public IP FQDN:
sed -i "s/host: <DNSNAME>.<LOCATION>.cloudapp.azure.com/host: ${DNSNAME}/g" azure-vote-app.yaml
Then, apply the configuration:
kubectl create ns azure-vote
kubectl apply -f azure-vote-app.yaml -n azure-vote
Using IngressRoute CRD
Traefik also supports the IngressRoute CRD for more advanced routing configurations. Here is an example of how to use it:
# azure-vote-ingressroute.yaml
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
name: azure-vote-ingressroute
spec:
entryPoints:
- websecure
routes:
- match: Host(`${DNSNAME}`)
kind: Rule
services:
- name: azure-vote-front
port: 80
tls:
certResolver: letsencrypt
Apply the IngressRoute configuration:
kubectl apply -f azure-vote-ingressroute.yaml -n azure-vote
Middleware Configuration
You can also configure middleware using Traefik's CRDs. Here is an example of how to set up a middleware to add security headers:
# azure-vote-middleware.yaml
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: test-header
spec:
headers:
frameDeny: true
browserXssFilter: true
Apply the middleware configuration:
kubectl apply -f azure-vote-middleware.yaml -n azure-vote
Then, reference the middleware in your IngressRoute:
# Updated azure-vote-ingressroute.yaml
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
name: azure-vote-ingressroute
spec:
entryPoints:
- websecure
routes:
- match: Host(`${DNSNAME}`)
kind: Rule
middlewares:
- name: test-header
services:
- name: azure-vote-front
port: 80
tls:
certResolver: letsencrypt
When considering high availability in a setup involving multiple instances of Traefik with Let's Encrypt, it is important to note that Let's Encrypt itself does not inherently provide high availability solutions. However, Let's Encrypt does offer robust security features for obtaining and managing TLS certificates.
Limitations with Traefik and Let's Encrypt
Using multiple instances of Traefik with Let's Encrypt can be challenging due to the nature of the ACME challenge. Each Traefik instance may attempt to renew the certificate independently, leading to conflicts and overwriting of certificates.
Here are some recommended approaches to address this issue:
1. Centralized Storage with Shared File System
Use a shared file system (e.g., NFS, Ceph, or AWS EFS) that is accessible to all Traefik instances.
Configure Traefik to use the shared file system as the storage backend for ACME certificates by specifying the acme.json file location.
Example configuration in traefik.yml:
certificatesResolvers:
letsEncrypt:
acme:
email: "[email protected]"
storage: "/shared/acme.json"
httpChallenge:
entryPoint: "web"
2. Use a Distributed Key-Value Store
Traefik supports using distributed key-value stores like Consul, Etcd, or Redis to store ACME certificates.
This ensures that all instances have consistent access to certificate data and can avoid conflicts.
Example configuration for Consul:
certificatesResolvers:
letsEncrypt:
acme:
email: "[email protected]"
storage: "traefik/acme/account"
httpChallenge:
entryPoint: "web"
providers:
consul:
endpoints:
- "127.0.0.1:8500"
3. Avoid Simultaneous Renewal Attempts
Use a leader-election mechanism (e.g., Kubernetes leader-election or a similar process in other environments) to designate a single Traefik instance as the one responsible for certificate renewal.
Non-leader instances can still use the certificates but do not attempt renewal.
4. DNS Challenge for Cert Management
Consider using the DNS challenge for certificate validation, especially in a multi-instance setup. This approach is stateless and avoids potential conflicts during HTTP challenges.
Example DNS challenge configuration:
certificatesResolvers:
letsEncrypt:
acme:
email: "[email protected]"
storage: "/shared/acme.json"
dnsChallenge:
provider: "cloudflare"
By implementing one or more of these strategies, you can ensure smooth certificate management across multiple Traefik instances and avoid the challenges associated with Let's Encrypt's ACME protocol.
Conclusion
Deploying Traefik as an ingress controller on Kubernetes with automatic TLS using Let's Encrypt or Cert-Manager simplifies the management of ingress traffic and improves the security of your applications. By following the steps outlined in this guide, you can set up a robust and scalable ingress solution that meets the demands of your Kubernetes workloads.