Spark Logs
Decode Spark errors — OOM, shuffle failures, executor loss
Paste Apache Spark logs or stack traces and get the failure identified with concrete fixes: out-of-memory causes, shuffle fetch failures, executor losses, serialization problems, and more. Faster than digging through Stack Overflow.
Frequently Asked Questions
- What causes java.lang.OutOfMemoryError in Spark?
- Usually skewed partitions, oversized broadcasts, or collecting too much to the driver. Fixes include increasing executor memory, repartitioning skewed keys, and avoiding collect() on large datasets.
- What does "FetchFailedException" mean?
- A shuffle fetch failure — an executor could not retrieve shuffle data, typically because the serving executor died (often OOM) or network timeouts. The fix is usually addressing the underlying executor loss.
- Why does Spark say "Container killed by YARN for exceeding memory limits"?
- The executor JVM plus off-heap overhead exceeded the YARN container. Raise spark.executor.memoryOverhead, reduce spark.executor.memory pressure, or lower cores per executor.
- Is my log content kept private?
- Pattern-based decoding runs without storing your logs. Avoid pasting secrets regardless — logs sometimes embed connection strings.