Tutorial: Parallel Processing with Python

Introduction

Parallel processing refers to using multiple CPU cores to perform computations simultaneously, which can significantly reduce the overall execution time of tasks that can be divided into independent units of work. In Python, the multiprocessing module provides a powerful interface for parallel processing.

In this tutorial, we’ll cover:

1. Basics of worker functions
2. Serial processing example
3. Parallel processing example
4. Performance comparison with timers

1. Basics of Worker Functions

A worker function is a function that performs a specific task and can run independently. In parallel processing, multiple instances of this function run simultaneously on different CPU cores.

2. Serial Processing Example

Let’s start with a simple example where we calculate the squares of numbers from 0 to 9 using a single CPU core (serial processing).

import time

def square(n):
    """Function to calculate the square of a number"""
    return n * n
def serial_processing(numbers):
    results = []
    for number in numbers:
        result = square(number)
        results.append(result)
    return results
if __name__ == "__main__":
    numbers = list(range(10))
    start_time = time.time()
    results = serial_processing(numbers)
    end_time = time.time()
    print(f"Results: {results}")
    print(f"Serial processing time: {end_time - start_time} seconds")

3. Parallel Processing Example

Now, let’s use the multiprocessing module to perform the same task in parallel.

import time
import multiprocessing

def square(n):
    """Function to calculate the square of a number"""
    return n * n
def parallel_processing(numbers):
    with multiprocessing.Pool(processes=multiprocessing.cpu_count()) as pool:
        results = pool.map(square, numbers)
    return results
if __name__ == "__main__":
    numbers = list(range(10))
    start_time = time.time()
    results = parallel_processing(numbers)
    end_time = time.time()
    print(f"Results: {results}")
    print(f"Parallel processing time: {end_time - start_time} seconds")

Performance Comparison

Now that we have both serial and parallel processing examples, let’s execute them and compare the execution times.

Running the Serial Processing Code

When you run the serial processing code, you’ll see the time taken to calculate the squares of numbers from 0 to 9 using a single CPU core and if you run it we should get around 5 seconds give or take not bad at all

Running the Parallel Processing Code

When you run the parallel processing code, you’ll see the time taken to perform the same calculations using multiple CPU cores and if you did it should be about 1 second give or take Wooow a big improvement so whats happening in more detail?

Code Breakdown

1. Importing Modules

import time
import multiprocessing

• time: This module provides various time-related functions. In this script, it is used to measure the duration of the execution.
• multiprocessing: This module allows you to create processes, which can run concurrently, thus enabling parallel processing.

2. Defining the square Function

def square(n):
    """Function to calculate the square of a number"""
    return n * n

• square: A simple function that takes an integer n and returns its square, i.e., n * n.

3. Defining the parallel_processing Function

def parallel_processing(numbers):
    with multiprocessing.Pool(processes=multiprocessing.cpu_count()) as pool:
        results = pool.map(square, numbers)
    return results

• multiprocessing.Pool: Creates a pool of worker processes. The number of processes is determined dynamically using, multiprocessing.cpu_count()which returns the number of CPU cores available on the machine.
• pool.map: This method maps the square function to each element in the numbers list, distributing the work across multiple processes in the pool. Essentially, each process takes a chunk of the work and calculates the square of each number in its chunk.
• results: Collects the results from all the processes.

4. Main Block

if __name__ == "__main__":
    numbers = list(range(10))

    start_time = time.time()
    results = parallel_processing(numbers)
    end_time = time.time()
    print(f"Results: {results}")
    print(f"Parallel processing time: {end_time - start_time} seconds")

• if __name__ == "__main__":: Ensures that the code inside this block only runs if the script is executed directly (not if it is imported as a module in another script).
• numbers = list(range(10)): Creates a list of numbers from 0 to 9. This list will be processed.
• start_time = time.time(): Records the current time before starting the parallel processing. This will be used to calculate the total processing time.
• results = parallel_processing(numbers): Calls the parallel_processing function with the list of numbers. This function distributes the work across multiple processes and returns the list of squared numbers.
• end_time = time.time(): Records the current time after parallel processing is complete.
• print(f"Results: {results}"): Prints the list of squared numbers obtained from parallel processing.
• print(f"Parallel processing time: {end_time - start_time} seconds"): Prints the total time taken for parallel processing. The time is calculated by subtracting start_time from end_time.

Execution Flow

1. The script imports the necessary modules.
2. It defines the square function, which calculates the square of a number.
3. It defines the parallel_processing function, which uses a pool of worker processes to calculate the squares of a list of numbers in parallel.
4. In the main block:

• It creates a list of numbers from 0 to 9.
• It records the start time.
• It calls the parallel_processing function to calculate the squares of the numbers in parallel.
• It records the end time.
• It prints the results and the time taken for parallel processing.

Why Parallel Processing Helps

• Speed: By distributing the work across multiple CPU cores, tasks that can be performed independently (like calculating the square of different numbers) can be computed much faster than with a single core.
• Scalability: As the number of elements in the list increases, parallel processing becomes increasingly beneficial. Multiple cores can handle larger workloads more efficiently.

Full Example Code with Timer for Both Cases

Here’s the complete code with both serial and parallel processing along with timers:

import time
import multiprocessing

def square(n):
    """Function to calculate the square of a number"""
    return n * n
def serial_processing(numbers):
    results = []
    for number in numbers:
        result = square(number)
        results.append(result)
    return results
def parallel_processing(numbers):
    with multiprocessing.Pool(processes=multiprocessing.cpu_count()) as pool:
        results = pool.map(square, numbers)
    return results
if __name__ == "__main__":
    numbers = list(range(10))  # You can increase this range for a better demonstration of timing differences
    print("Serial Processing")
    start_time = time.time()
    serial_results = serial_processing(numbers)
    serial_end_time = time.time()
    serial_time = serial_end_time - start_time
    print(f"Results: {serial_results}")
    print(f"Serial processing time: {serial_time} seconds")
    print("\nParallel Processing")
    start_time = time.time()
    parallel_results = parallel_processing(numbers)
    parallel_end_time = time.time()
    parallel_time = parallel_end_time - start_time
    print(f"Results: {parallel_results}")
    print(f"Parallel processing time: {parallel_time} seconds")
    print(f"\nTime difference: {serial_time - parallel_time} seconds")

Summary

By using parallel processing with the multiprocessing module, we can significantly improve the performance of tasks that can be divided into independent units of work. This tutorial demonstrated the concept using a simple example of calculating squares of numbers, but the principles can be applied to more complex tasks in data processing, machine learning, and other fields.

Notes

• The time difference might not be significant for very small datasets (e.g., squaring numbers from 0 to 9), but with larger datasets, the benefits of parallel processing will become more apparent.
• Ensure that the task is suitable for parallelization; not all tasks can benefit from this approach, especially those with dependencies between units of work.

Feel free to experiment with larger datasets and different types of tasks to see how parallel processing can improve performance in your specific use case.