EKS · Central VPC Infrastructure
Started from the desire to design a Kubernetes platform that could actually handle production-level traffic. Built an EKS-based application platform and Central VPC centralized operations network, then validated the architecture with 2,000 RPS and 120K total requests in QA.
Role: As team lead, coordinated the schedule and direction while driving the EKS-centric architecture design. My focus was on predicting 'where will it break first under load?' and preparing for it proactively.
Five environments — Prod / QA / Dev / DR / Central VPC. Prod and QA run CloudFront → ALB (Ingress) → EKS Pod across multiple AZs. Data layer uses Aurora + RDS Proxy. Central VPC consolidates GitLab, monitoring, and DNS security observability.
Design Rationale
I considered ECS but wanted fine-grained autoscaling control via open-source tools like KEDA and Karpenter, plus consistent Helm-based config management across environments — so EKS was the better fit. Central VPC wasn't in the original design; I added it after realizing that setting up monitoring separately per environment scattered alerts and made it impossible to see the whole picture. DR used Pilot Light because Active-Active was beyond budget — maintaining minimal resources in standby and scaling up on failure was the cost-recovery tradeoff.
Managing five environments manually via console would inevitably lead to configuration drift. DR especially needs to be code-defined — if you can't reproduce it reliably, it's useless when you actually need it.
I debated ECS, but KEDA for RPS-based scaling and Karpenter for automatic node provisioning required the Kubernetes ecosystem. I wanted a structure where 'both Pods and nodes scale together when traffic rises.'
HPA's CPU-based scaling had poor correlation with actual user traffic. Scaling by average RPS per Pod from Prometheus metrics responds to real load, and pre-scaling 45 Pods absorbed cold start issues.
Even if KEDA scales Pods, they go Pending without enough nodes. Cluster Autoscaler was too slow; Karpenter detects Pending Pods and provisions right-sized nodes quickly, keeping Pod and node scaling in sync.
Running kubectl apply manually makes it impossible to be sure the live environment matches Git. With five environments, Git as the single source of truth for deployment state and history was essential.
Attaching an IAM Role to a node gives every Pod on that node the same permissions. Mapping only needed permissions per ServiceAccount to each Pod minimizes the blast radius.
- Issue
- During load testing, Spring Boot Pods received traffic before boot completed, HPA scaling lagged behind traffic increases, and Pods went Pending when nodes were insufficient.
- Analysis
- Spring Boot takes 10–15 seconds to boot, but Pods were registered to the service immediately without a readiness probe. HPA used CPU metrics, so its reaction timing diverged from actual request volume. Pod scaling without node scaling created a compounding problem.
- Solution
- Separated startup/readiness/liveness probes to block traffic before boot completes. Switched to KEDA with average RPS per Pod scaling, pre-scaled 45 Pods for initial load absorption, and added Karpenter for automatic node provisioning on Pending — aligning Pod and node scaling timing.
- Result
- Sustained ~2,000 RPS for 60 seconds in QA, processing 120,000 requests with zero downtime.
- Issue
- Worker nodes failed to join the EKS API or fell into NotReady state.
- Analysis
- I initially assumed it was a node-level issue, but the actual cause was that Private Subnet routing tables weren't going through the NAT Gateway, so nodes couldn't reach the control plane. A network configuration error, not a compute one.
- Solution
- Fixed subnet routing and DNS settings, and also adjusted MaxPods limits per instance type. After this experience, I created an EKS network checklist to prevent the same mistake when adding new environments.
- Result
- Node join issues were resolved and Karpenter-based autoscaling operated stably.
The biggest takeaway was validating a designed architecture against real load. Architecture on paper and architecture under traffic are different things, and I felt that difference firsthand.
What stayed with me longer, though, was how to work with AI. Terraform modules, KEDA ScaledObjects, Karpenter NodePools — drafting all of it went noticeably faster with AI. But what got faster was implementation, not judgment. If I couldn't explain why KEDA instead of HPA, or Karpenter instead of Cluster Autoscaler, the generated config was just YAML that happened to run. I once set scaling values without properly reasoning through the policy and blew past the AWS budget during load testing. That wasn't the tool's failure; it was mine for directing it without criteria.
So I reversed the order: settle why this technology first, then bring AI in for the stretch that turns the decision into code. The same tool behaved completely differently after that. It stopped being 'something I could only manage because AI existed' and became 'more configurations I could validate in the same amount of time.' The clearer my reasons for a technology choice, the more AI was worth — that was this project's conclusion. Not getting Karpenter fully into the GitOps flow before the project wrapped still sits with me as unfinished.