Spark "Container Killed by YARN for Exceeding Memory Limits" — Explained and Fixed

July 3, 2026 · 5 min read

ExecutorLostFailure (executor 7 exited caused by one of the running tasks)
Reason: Container killed by YARN for exceeding memory limits.
14.2 GB of 14 GB physical memory used.
Consider boosting spark.yarn.executor.memoryOverhead.

The confusing part: you set spark.executor.memory=12g, your job barely uses the heap, and YARN still kills the container. The log even suggests a fix — but boosting overhead blindly often just moves the failure later in the job.

Why YARN kills executors with healthy heaps

YARN enforces a limit on the whole OS process, not the JVM heap. The container's budget is:

container limit = spark.executor.memory        (JVM heap)
                + spark.executor.memoryOverhead (everything else)
                  # default: max(384MB, 10% of executor memory)

"Everything else" is where jobs die: off-heap allocations from the shuffle machinery, netty network buffers, memory-mapped files, PySpark worker processes (each Python worker is a separate OS process!), and native libraries like Arrow. None of that appears in JVM heap metrics — which is exactly why the heap looks fine while the process exceeds its budget.

The four fixes, in the order to try them

1. Raise overhead — if the overrun is small and stable.If the kill happens at "14.2 GB of 14 GB", a modest bump absorbs it:

--conf spark.executor.memoryOverhead=2g

2. For PySpark: cap Python memory. Python workers are usually the real consumer. Limit them explicitly so pandas UDFs cannot eat the container:

--conf spark.executor.pyspark.memory=2g

3. Reduce parallel memory pressure per executor. Fewer cores per executor means fewer concurrent tasks sharing the same off-heap pools — often more effective than adding memory:

--conf spark.executor.cores=3   # instead of 5

4. Fix the skew. If one executor dies while others idle, a hot partition is concentrating data. Salting the key or enabling AQE skew handling (spark.sql.adaptive.enabled=true) fixes the cause rather than the symptom.

Reading the pattern from the log

The failure's distribution tells you which fix applies: all executors dying gradually points to overhead sizing (fix 1–3); one executor dying repeatedly on the same task points to skew (fix 4); deaths only during wide shuffles point to cores-per-executor (fix 3).

If you have a wall of executor logs and want the failure classified for you, paste it into the Hexabench Spark Error Decoder — it recognizes YARN kills, OOM variants, shuffle fetch failures, and serialization errors, and suggests the matching fix.

Try the Spark Error Decoder → · Back to Blog