Parallelization of a program always introduces some extra work in addition to the work done by the sequential version of the program. The main sources of parallelization overhead are data communication (between processes) and synchronization (of processes and threads). Other sources are additional operations that are introduced at the algorithmic level (for example in global reduction operations) or at a lower software level (for example by address calculations). d

• To provide knowledge about overheads for communication and synchronization that are introduced by parallelization
• To provide knowledge about other sources of parallel inefficiency: load imbalances, hardware effects

m

• the various overheads, i.e. overheads for communication, synchronization
• the problems of load imbalances