Batch Normalization

Normalize along each dimension of batches

Layer Normalization

Normalize within each sample.

Comparison

• Simplest case