Answers and Solutions

Answer

Parallel Processing Systems are designed to speed up the execution of programs by dividing the program into multiple fragments and processing these fragments simultaneously. Such systems are multiprocessor systems also known as tightly coupled systems. Parallel systems deal with the simultaneous use of multiple computer resources that can include a single computer with multiple processors, a number of computers connected by a network to form a parallel processing cluster or a combination of both.

Parallel systems are more difficult to program than computers with a single processor because the architecture of parallel computers varies accordingly and the processes of multiple CPUs must be coordinated and synchronized. Several models for connecting processors and memory modules exist, and each topology requires a different programming model. The three models that are most commonly used in building parallel computers include synchronous processors each with its own memory, asynchronous processors each with its own memory and asynchronous processors with a common, shared memory. Flynn has classified the computer systems based on parallelism in the instructions and in the data streams. These are:

1.         Single instruction stream, single data stream (SISD).

2.         Single instruction stream, multiple data stream (SIMD).

3.         Multiple instruction streams, single data stream (MISD).

4.         Multiple instruction stream, multiple data stream (MIMD).

The above classification of parallel computing system is focused in terms of two independent factors: the number of data streams that can be simultaneously processed, and the number of instruction streams that can be simultaneously processed. Here 'instruction stream' we mean an algorithm that instructs the computer what to do whereas 'data stream' (i.e. input to an algorithm) we mean the data that are being operated upon.  

Even though Flynn has classified the computer 'systems into four types based on parallelism but only two of them are relevant to parallel computers. These are SIMD and MIMD computers.

Answer

There are many ways of classifying parallel processors based on their structures and behaviors. The major classification method considers the number of instruction and/or operand sets that can be processed simultaneously, the internal organization of the processors,the interprocessor connection structure or the methods used to control the flow of instructions and data through the system.

Flynn's Classification:

Single Instruction, Single Data (SISD):

• A serial (non-parallel) computer
• Single instruction: only one instruction stream is being acted on by the CPU during any one clock cycle
• Single data: only one data stream is being used as input during any one clock cycle
• Deterministic execution
• This is the oldest and even today, the most common type of computer
• Examples: older generation mainframes, minicomputers and workstations; most modern day PCs.

Single Instruction, Multiple Data (SIMD):

• A type of parallel computer
• Single instruction: All processing units execute the same instruction at any given clock cycle
• Multiple data: Each processing unit can operate on a different data element
• Best suited for specialized problems characterized by a high degree of regularity, such as graphics/image processing.
• Synchronous (lockstep) and deterministic execution
• Two varieties: Processor Arrays and Vector Pipelines

Multiple Instruction, Single Data (MISD):

• A single data stream is fed into multiple processing units.
• Each processing unit operates on the data independently via independent instruction streams.
• Few actual examples of this class of parallel computer have ever existed. One is the experimental Carnegie-Mellon C.mmp computer (1971).
• Some conceivable uses might be: Multiple frequency filters operating on a single signal stream. Multiple cryptography algorithms attempting to crack a single coded message.

Multiple Instruction, Multiple Data (MIMD):

• Currently, the most common type of parallel computer. Most modern computers fall into this category.
• Multiple Instruction: every processor may be executing a different instruction stream
• Multiple Data: every processor may be working with a different data stream
• Execution can be synchronous or asynchronous, deterministic or non-deterministic
• Examples: most current supercomputers, networked parallel computer clusters and "grids", multi-processor SMP computers, multi-core PCs.
• Note: many MIMD architectures also include SIMD execution sub-components

Answer

Single Instruction, Single Data (SISD):

• A serial (non-parallel) computer
• Single instruction: only one instruction stream is being acted on by the CPU during any one clock cycle
• Single data: only one data stream is being used as input during any one clock cycle
• Deterministic execution
• This is the oldest and even today, the most common type of computer
• Examples: older generation mainframes, minicomputers and workstations; most modern day PCs.

Single Instruction, Multiple Data (SIMD):

• A type of parallel computer
• Single instruction: All processing units execute the same instruction at any given clock cycle
• Multiple data: Each processing unit can operate on a different data element
• Best suited for specialized problems characterized by a high degree of regularity, such as graphics/image processing.
• Synchronous (lockstep) and deterministic execution
• Two varieties: Processor Arrays and Vector Pipelines
• Examples:
• Processor Arrays: Connection Machine CM-2, MasPar MP-1 & MP-2, ILLIAC IV
• Vector Pipelines: IBM 9000, Cray X-MP, Y-MP & C90, Fujitsu VP, NEC SX-2, Hitachi S820, ETA10
• Most modern computers, particularly those with graphics processor units (GPUs) employ SIMD instructions and execution units.

Multiple Instruction, Single Data (MISD):

• A single data stream is fed into multiple processing units.
• Each processing unit operates on the data independently via independent instruction streams.
• Few actual examples of this class of parallel computer have ever existed. One is the experimental Carnegie-Mellon C.mmp computer (1971).
• Some conceivable uses might be: Multiple frequency filters operating on a single signal stream. Multiple cryptography algorithms attempting to crack a single coded message.

Multiple Instruction, Multiple Data (MIMD):

• Currently, the most common type of parallel computer. Most modern computers fall into this category.
• Multiple Instruction: every processor may be executing a different instruction stream
• Multiple Data: every processor may be working with a different data stream
• Execution can be synchronous or asynchronous, deterministic or non-deterministic
• Examples: most current supercomputers, networked parallel computer clusters and "grids", multi-processor SMP computers, multi-core PCs.
• Note: many MIMD architectures also include SIMD execution sub-components