Kubernetes Limits in Production: A Guide with Verifiable Thresholds

A cluster that works is not a production-ready cluster#

There's a dangerous idea that keeps coming up on many teams: assuming a cluster is ready because it "works." It works in staging, it works with three services, it works on demo day. But a cluster that works is not the same as a cluster designed for production, and the difference almost always lies in an area few people review in time: Kubernetes limits.

Kubernetes is flexible, but it isn't infinite. Behind every component there are caps (in the API server, etcd, the kubelet and the naming rules) that rarely show up in an internal runbook and that surface exactly when the cluster grows. The typical result: deployments that fail for no obvious reason, rising costs, endless troubleshooting and architecture decisions that have to be redone under pressure.

This guide walks through the limits with the most impact on real-world operations, explains where each one comes from and how to keep an eye on it. Every value points to the official documentation; where a managed provider changes the number, that's called out.

Documentation: Kubernetes · Considerations for large clusters ↗

Why limits are not a technical footnote#

Kubernetes limits aren't interview trivia: they're constraints that shape the cluster's architecture from day one.

The best-known example: the official documentation states that Kubernetes is designed for configurations that meet all of the following at once: no more than 110 pods per node, no more than 5,000 nodes, no more than 150,000 total pods and no more than 300,000 total containers per cluster. These aren't arbitrary numbers: they mark the range in which the project guarantees predictable control plane behavior.

When a team approaches those caps or ignores them, it doesn't get a clear error saying "you went over." What it usually sees is growing API server latency, slower scheduling, add-ons restarting because they run out of memory and a cluster that's increasingly hard to operate. A limit doesn't go away because you don't know about it; it shows up as an incident.

The same goes for storage. Kubernetes keeps all cluster state in etcd, and etcd has its own limits: by default, the maximum request size is 1.5 MiB and the storage quota is 2 GiB, with 8 GiB as the suggested maximum for normal environments. If etcd hits its quota, it stops accepting writes: it's not "one more service" going down, the memory of the entire cluster freezes.

Documentation: Kubernetes · Considerations for large clusters ↗ · etcd · System limits ↗

Not every limit comes from the same place#

A common mistake is treating every limit as if it had the same origin. Understanding who enforces each one changes how you diagnose problems.

• API server. Individual Secrets are limited to 1 MiB to prevent huge objects from exhausting API server and kubelet memory; the data in a ConfigMap can't exceed 1 MiB either. In addition, the total size of an object's annotations (keys and values) can't exceed 256 KiB.
• etcd. The 1.5 MiB per-request cap belongs to etcd and can be configured with --max-request-bytes; the database quota is set with --quota-backend-bytes. On managed services you don't control these flags.
• Metadata and names. Label and annotation keys have a name of up to 63 characters and an optional prefix of up to 253; label values, up to 63. Most objects, such as Pods, use DNS subdomain names (up to 253 characters), while others, such as Services, require DNS labels of at most 63 characters.
• Kubelet and node. This is where operational values live, such as the termination grace period (30 seconds by default) and the default eviction thresholds on Linux: memory.available<100Mi, nodefs.available<10%, imagefs.available<15% and nodefs.inodesFree<5%. Once they're crossed, the kubelet starts evicting pods.

A real case that trips up many teams: applying a ConfigMap of, say, 400 KiB with client-side kubectl apply. The data is under 1 MiB, but kubectl stores a full copy of the manifest in the kubectl.kubernetes.io/last-applied-configuration annotation, and that annotation exceeds the 256 KiB limit. The error mentions annotations, not the ConfigMap. The fix is to use server-side apply, which doesn't rely on that annotation, or to shrink the object.

These look like minor details, but an overly long name generated by a Helm chart or an annotation that grows unchecked can break an operator or an entire automation, often intermittently, which is the worst way to fail.

Documentation: Kubernetes · Secrets (size limit) ↗ · Kubernetes · ConfigMaps ↗ · Kubernetes · Annotations ↗ · Kubernetes · Labels and selectors ↗ · Kubernetes · Object names and IDs ↗ · etcd · System limits ↗ · Kubernetes · Node-pressure eviction ↗ · Kubernetes · Pod lifecycle (termination) ↗ · Kubernetes · Declarative management with kubectl apply ↗ · Kubernetes · Server-Side Apply ↗

Quick reference table#

Upstream project defaults and the component that enforces each limit. Some change depending on the provider, version or network plugin, as the next section shows.

Documentation: Kubernetes · Considerations for large clusters ↗ · etcd · System limits ↗ · Kubernetes · Secrets (size limit) ↗ · Kubernetes · ConfigMaps ↗ · Kubernetes · Annotations ↗ · Kubernetes · Object names and IDs ↗ · Kubernetes · Service (NodePort) ↗ · Kubernetes · Ports and protocols ↗ · Kubernetes · Pod lifecycle (termination) ↗

Default values are not recommended values#

This is probably the most expensive misunderstanding. A default value or a maximum limit is a ceiling, not a recommendation for production.

The 1 MiB limit on Secrets and ConfigMaps illustrates it well. Technically you can get close to that cap, but the documentation itself points out that a ConfigMap isn't designed to hold large chunks of data and that many small Secrets can also exhaust memory. Keep these objects as small as possible and, if you need more, mount a volume or use an external service. The limit tells you what's possible; good practice tells you what's healthy.

The same applies to etcd: 2 GiB is the default quota, but running a large cluster requires planning history compaction, periodic defragmentation to reclaim space and continuous monitoring of the database size. Without that, the database grows until it hits the quota and etcd goes into read-only mode.

Documentation: Kubernetes · ConfigMaps ↗ · Kubernetes · Secrets (size limit) ↗ · etcd · Maintenance (compaction and defragmentation) ↗

Some limits change on EKS, GKE and AKS#

If your cluster runs on a managed service, several of these numbers don't apply the same way as on a self-managed cluster. The clearest case is pods per node:

• Amazon EKS: with the VPC CNI, each pod gets an IP address from the VPC, so the maximum depends on the network interfaces and IP addresses the instance type supports. On managed node groups without a custom AMI, EKS also caps pods at 110 on instances with fewer than 30 vCPUs and at 250 on larger ones.
• Google GKE: Standard clusters default to 110 and can be configured for up to 512 pods per node; in Autopilot, GKE picks a maximum between 8 and 256 based on the expected density.
• Azure AKS: the maximum is 250 pods per node, and the default varies by network plugin and deployment method (CLI, ARM template or portal).

The operational takeaway is simple: don't assume the "official" number is the one in your environment. Check your provider's documentation, the instance type and the network plugin you use, and verify the actual value on each node. Claiming that "Kubernetes works the same everywhere" is exactly the kind of assumption that causes incidents. On managed services you don't control etcd's flags either, so it pays to know the quotas your provider publishes.

# 1. Allowed pods per node (what your provider actually configured)
kubectl get nodes -o custom-columns=NODE:.metadata.name,MAX_PODS:.status.allocatable.pods

# 2. Scheduled pods per node, highest first
kubectl get pods -A -o jsonpath='{range .items[*]}{.spec.nodeName}{"\n"}{end}' \
  | sort | uniq -c | sort -rn | head

# 3. Largest ConfigMaps (approximate size of the serialized object)
kubectl get configmaps -A -o json \
  | jq -r '.items[] | "\(tojson | length)\t\(.metadata.namespace)/\(.metadata.name)"' \
  | sort -rn | head

# 4. Large objects: apply them with server-side apply to avoid creating
#    the last-applied-configuration annotation
kubectl apply --server-side -f big-configmap.yaml

Documentation: Amazon EKS · Choose an instance type and maxPods ↗ · Google Cloud · Maximum Pods per node in GKE ↗ · Microsoft Learn · AKS quotas and limits ↗ · Kubernetes · Server-Side Apply ↗

Best practices for DevOps and platform engineering teams#

Knowing the limits is the first step. The second is building an operation that doesn't depend on every engineer remembering them by heart.

• Document limits as an internal standard: pods per node for each pool, the maximum accepted size for Secrets and ConfigMaps, naming conventions and etcd behavior should live in a platform standard, not in one person's head.
• Validate in CI/CD and at admission, not in production: manifest validation in the pipeline and admission policies such as ValidatingAdmissionPolicy can stop oversized objects or invalid names before deployment.
• Use observability as an early warning: track etcd database size against its quota, pods per node and API server latency so you know when you're approaching a limit, not after you've crossed it.
• Treat capacity as an explicit design variable: decide pods per node and nodes per cluster based on real limits, including your provider's, not on accidental growth.
• Keep objects small: lightweight Secrets, ConfigMaps and annotations reduce the load on etcd and the API server and make the cluster more stable and cheaper to run.

groups:
  - name: kubernetes-limits
    rules:
      # Self-managed etcd: alert before reaching the database quota
      - alert: EtcdDatabaseNearQuota
        expr: etcd_mvcc_db_total_size_in_bytes / etcd_server_quota_backend_bytes > 0.8
        for: 15m
        labels:
          severity: warning
      # API server p99 latency per verb (excludes WATCH and CONNECT, which are long-running)
      - alert: KubeAPIServerHighLatency
        expr: |
          histogram_quantile(0.99,
            sum by (le, verb) (
              rate(apiserver_request_duration_seconds_bucket{verb!~"WATCH|CONNECT"}[5m])
            )
          ) > 1
        for: 10m
        labels:
          severity: warning

This is exactly what a mature platform engineering practice brings: turning scattered knowledge into standards, automation and guardrails that scale with the team.

Documentation: Kubernetes · Validating Admission Policy ↗ · Kubernetes · Metrics reference ↗ · etcd · Maintenance (compaction and defragmentation) ↗

Conclusion#

Maturity with Kubernetes isn't measured by how many workloads you deploy, but by how well you understand the cluster's behavior under load. The limits on pods, etcd, Secrets, names and the kubelet aren't obstacles: they're the information that lets you design to scale instead of improvising and patching.

A production-ready cluster is one where the limits are understood, checked against the provider, monitored and turned into operational standards. Everything else is a failure waiting for its moment.