Back to Home

Kubernetes in Production

Security

Hardening Kubernetes clusters with RBAC, Pod Security Standards, network policies, and security best practices.

Role-Based Access Control (RBAC)

RBAC is the standard mechanism for controlling access to Kubernetes resources based on roles assigned to users and service accounts.

Core RBAC Components

Role / ClusterRole

Define permissions (verbs) on resources

Scope: Role: namespace-scoped | ClusterRole: cluster-wide

Example: get, list, watch, create, update, patch, delete

RoleBinding / ClusterRoleBinding

Grant permissions to subjects (users, groups, service accounts)

Scope: Binding: links role to subjects in namespace or cluster

Example: Bind 'developer' role to 'john' user in 'production' namespace

ServiceAccount

Identity for pods to authenticate with API server

Scope: Every namespace has a 'default' service account

Example: Custom service account for CI/CD pipelines

RBAC Best Practices

  • Least privilege: Grant minimum permissions needed
  • Avoid wildcards: Be specific about resources and verbs
  • Use service accounts: Never use default service account for apps
  • Regular audits: Review and remove unused permissions
  • Namespace isolation: Use namespaces to separate teams/apps

Pod Security Standards

Pod Security Standards (PSS) replaced Pod Security Policies (PSP) in Kubernetes 1.25. They define three policies for securing pods: Privileged, Baseline, and Restricted. These standards are enforced via Pod Security Admission controller built into the API server.

Pod Security Admission (2025 Standard)

Pod Security Admission is now the recommended way to enforce pod security policies. Apply at namespace level using labels:

  • pod-security.kubernetes.io/enforce: restricted
  • pod-security.kubernetes.io/audit: restricted
  • pod-security.kubernetes.io/warn: restricted
🔓

Privileged

Unrestricted policy, allows known privilege escalations

Use Case:

Trusted system workloads, infrastructure components

Key Controls:

  • No restrictions
  • All capabilities allowed
  • HostPath volumes allowed
🔒

Baseline

Minimally restrictive, prevents known privilege escalations

Use Case:

Most applications, default security posture

Key Controls:

  • Host namespaces denied
  • Privileged containers blocked
  • Capabilities restricted
🔐

Restricted

Heavily restricted, follows pod hardening best practices

Use Case:

Security-critical applications, zero-trust environments

Key Controls:

  • Non-root required
  • Read-only root filesystem
  • All capabilities dropped

Implementation Modes

enforce

Policy violations reject pod creation

audit

Violations logged but allowed

warn

User warned but pod allowed

Network Security

Network Policy Patterns

Default Deny All

Block all ingress/egress by default

Benefit: Strongest security posture, explicitly allow only needed traffic

Namespace Isolation

Prevent cross-namespace communication

Benefit: Multi-tenant isolation, blast radius reduction

Egress Filtering

Control outbound traffic to specific destinations

Benefit: Prevent data exfiltration, limit attack surface

Pod Selector Rules

Allow traffic only from specific pods by label

Benefit: Microservices security, service-to-service authentication

Service Mesh Security

Service meshes like Istio, Linkerd provide advanced security features:

  • mTLS: Automatic mutual TLS between services
  • Authorization policies: Fine-grained access control
  • Traffic encryption: End-to-end encryption
  • Identity verification: Service-to-service authentication
  • Audit logging: Comprehensive request logging

Additional Security Hardening

Image Security

  • Scan images for vulnerabilities (Trivy, Grype, Snyk)
  • Use minimal base images (distroless, alpine, chainguard)
  • Sign images with Sigstore/cosign
  • Implement image admission policies (Kyverno, OPA)
  • Private registries with authentication

Secrets Management

  • Enable encryption at rest for etcd
  • Use external secret stores (Vault, AWS Secrets Manager)
  • External Secrets Operator for GitOps
  • Rotate secrets regularly with automation
  • Limit secret access with RBAC
  • Never log or expose secrets

Runtime Security

  • Use Falco for runtime threat detection
  • Implement admission controllers (Kyverno, OPA Gatekeeper)
  • Monitor system calls and file access
  • Set up comprehensive audit logging
  • Use seccomp and AppArmor/SELinux profiles

Cluster Hardening

  • Disable anonymous authentication
  • Enable audit logging with retention
  • Restrict API server access (private endpoints)
  • Use private cluster endpoints when possible
  • Regular security patches and updates
  • Implement CIS Kubernetes Benchmark

Modern Policy Engines (2025)

Kyverno (CNCF Graduated)

Kubernetes-native policy management with kubectl-style policies

Features: Validation, mutation, generation, image verification, cleanup policies

OPA Gatekeeper

Policy enforcement using Open Policy Agent with Rego language

Features: Constraint templates, audit violations, external data sources

Falco (CNCF Graduated)

Runtime security using eBPF for kernel-level monitoring

Features: Threat detection, compliance monitoring, real-time alerts, cloud-native detection

Common Security Mistakes

  • Running as root: Always run containers as non-root users
  • Privileged containers: Avoid unless absolutely necessary (monitoring agents)
  • Exposed dashboards: Secure Kubernetes Dashboard and other admin UIs
  • Default service accounts: Create dedicated service accounts with minimal permissions
  • No network policies: Default "allow all" is insecure in production
  • Unscanned images: Use admission controllers to enforce image scanning

Key Takeaways

  • RBAC provides fine-grained access control; apply least privilege principle
  • Pod Security Standards offer three levels; start with Baseline, target Restricted
  • Network policies are essential for zero-trust security; default deny all traffic
  • Never run containers as root or with privileged mode in production
  • Security is layered: combine RBAC, PSS, network policies, and runtime security