What is the effect of using a broadcast join in a Spark job?

• A. It reduces data shuffle by sending the small DataFrame to all executors.
• B. It increases the amount of data shuffled across the cluster.
• C. It eliminates the need for a join condition.
• D. It requires both DataFrames to be partitioned identically.

Answer: (A)

Broadcast joins in Spark replicate the smaller dataset to every executor, so each worker can join its local partition of the larger dataset with the broadcasted data. This replication lets Spark avoid shuffling the large DataFrame across the cluster, cutting network I/O and speeding up the join when one side is small enough to fit in memory on each node. The join condition is still required; broadcasting does not eliminate it. You don’t need both DataFrames to be partitioned identically—the small side is copied to all nodes instead of being repartitioned to match the other side. A practical caveat is memory: if the small dataset doesn’t fit in memory on a worker, broadcasting can cause memory pressure and hurt performance.