A few months ago, I was debugging a strange issue in one of our Kubernetes clusters. A service that usually responded in milliseconds was suddenly taking seconds, and sometimes timing out altogether. Logs didn’t show much. Prometheus metrics told me something was wrong, but not why.
I felt blind.
That’s when I started looking into eBPF — and it felt like putting on night-vision goggles in a dark room. Suddenly, I could see what processes were running, what network calls they were making, and even which ones were failing — all without changing a single line of application code.
Why Monitoring in Kubernetes is Hard
If you’ve worked with Kubernetes long enough, you know this pain:
- Pods spin up and die within minutes.
- A single request can bounce across multiple services, sidecars, and proxies.
- Your “standard” monitoring tools tell you something’s slow… but not where or why.
In my case, I knew requests were stalling, but I didn’t know if it was the app, the node, the network, or something in between.
Enter eBPF
At its core, eBPF (extended Berkeley Packet Filter) lets you attach tiny programs directly into the Linux kernel. Don’t let “kernel” scare you — you don’t need to recompile anything. These programs run safely in a sandbox, and the kernel won’t let them crash your system.
The magic? eBPF can see every system call, every network packet, every file access — and tell you which process or container triggered it. In Kubernetes, that means you can tie kernel activity back to pods and services.
Think of it as asking the operating system: “Show me exactly what’s happening right now, at the lowest level, across all my containers.”
Installing the Basics
On most modern Linux distros, eBPF is already there (kernel 4.9+). What you need are some user-space tools. On Ubuntu or Debian:
sudo apt-get update
sudo apt-get install bpfcc-tools linux-headers-$(uname -r)
To check it’s working:
bpftool version
Watching Processes in Real Time
When I first installed eBPF tools, the simplest command I ran was:
sudo execsnoop-bpfcc
This little utility watches every process executed on the system and prints it out. My terminal lit up instantly:
PCOMM PID PPID RET ARGS
nginx 1245 1201 0 /usr/sbin/nginx -g daemon off;
curl 1267 1245 0 curl -s http://service:8080/
python3 1298 1250 0 python3 app.pyIn plain English: I could see nginx starting, curl making requests, and even my Python app firing up.
In Kubernetes, this is gold. If a pod is compromised or misconfigured, you’d see unexpected processes immediately. In my case, it reassured me that the right processes were starting — and no mysterious shells were being spawned.
Why This Matters
That day, eBPF showed me something my normal tools couldn’t: a pod was retrying a failing database connection so aggressively it was overwhelming the node. Traditional metrics just showed “high CPU.” eBPF let me pinpoint the exact system calls and processes causing it.
Since then, I’ve used eBPF for:
- Tracing network latency between pods.
- Profiling CPU usage without adding an agent inside containers.
- Catching noisy neighbors hogging disk I/O.
And this was just scratching the surface.
Kubernetes monitoring is hard because it hides the details you need most. eBPF flips that on its head by giving you kernel-level visibility into what your pods are really doing.
In this post, we covered:
- Why monitoring in Kubernetes is challenging.
- How eBPF gives you low-overhead visibility.
- How to install and run your first tool (execsnoop-bpfcc).
In Part 2, I’ll walk through more hands-on eBPF tools for monitoring: tracing network requests, profiling CPU, watching file I/O, and even spotting memory leaks.
If you’ve ever felt blind in a Kubernetes cluster, trust me — eBPF feels like turning the lights on.