Thread Performance Considerations

• Multi-process

• Multi-threaded

• Event-driven

Thread Metrics

Before we consider which model is useful, it is important to understand that it is all about perspectives and business needs. For example, do we care about total execution time or average execution time per request or stage in pipeline?

We know for sure that threads are useful to most cases.

• Achieve parallelization

• Specialization to keep cache hot

• Efficiency with memory requirement

• Hide latency of I/O operations

However, we need to evaluate the usefulness with actual metrics.

• Execution time

• Number of requests per time

• Response time

• System throughput

• Hardware utilization rate

Whether threads are actually useful depends on metrics and workloads.

Concurreny Models

Multi-process

Multiple process with single threading is often easy to program but the catch is that it uses more memory and context switch between processes on a CPU has a high cost.

Multi-threaded

Multiple threads share the same memory space, it is easy to exchange information with one another. It is also cheaper to context switch a thread than to context switch a process. However, it is rather challenging to implement multithreading code.

Event-driven

Event driven approach allows many concurrent operation to be interleaved in CPU execution. The core is hopping around many places. It will have many program counters. Dispatching of an event leads to the execution of a piece of handler code.

Oftentimes when we consider whether we want a multi-threading model given that we only have one CPU, we need to ask ourselves the question. if thread.idle_time > 2 * thread.ctx_switch_time, then we should use multi-threading model. Otherwise it would be better to use event-driven model, given that we don't perform any blocking I/O operation on the thread.

The key to use event-driven model is that we must have async I/O operations.

Flash

Flash is a event-driven web server. It uses asymmetric helper processes to achieve async I/O. It has the following hiearchy.

• Handlers

• Event Dispatcher

• Helpers

The single thread will always execute handler code. Whenever it needs to perform an I/O, it delegates the work to helpers. Once helper completes its work, it will emit an event via the event dispatcher.

• Helpers are used for disk reads

• Pipes are used for communication with dispatcher

• Helpers reads file in memory via mmap

• Dispatcher checks if pages are in mmap to decide which handler to invoke

Apache

Apache has a core module which is primarily responsible for handling requests and writing responses. It has multiple sub modules for different functionality. You can have security module, dynamic content management and etc... Apache is a combination of multi-process and multi-threaded model. Each process is a boss/worker process with dynamic thread pool.

Experiment Methodology

• What systems are you comparing?

• What workloads will be used?

• How will you measure performance?

  1. Bandwidth

  2. Request per unit of time