Kami adalah komuniti perkongsian. Sila muat naik ** 1 ** dokumen sebelum muat turun:

ATAU FACE TO DOWNLOAD

Digital Signal Processing

PDF generated using the open source mwlib toolkit. See http://code.pediapress.com/ for more information. PDF generated at: Sat, 07 Jun 2014 14:28:26 UTC

Contents Articles Digital signal processing

1

Discrete signal

6

Sampling Sampling (signal processing)

8 8

Sample and hold

13

Digital-to-analog converter

15

Analog-to-digital converter

21

Window function

32

Quantization (signal processing)

53

Quantization error

63

ENOB

74

Sampling rate

75

Nyquist–Shannon sampling theorem

80

Nyquist frequency

91

Nyquist rate

93

Oversampling

95

Undersampling

97

Delta-sigma modulation

99

Jitter

112

Aliasing

117

Anti-aliasing filter

122

Flash ADC

124

Successive approximation ADC

126

Integrating ADC

129

Time-stretch analog-to-digital converter

137

Fourier Transforms, Discrete and Fast

142

Discrete Fourier transform

142

Fast Fourier transform

157

Cooley-Tukey FFT algorithm

165

Butterfly diagram

171

Codec

173

FFTW

175

Wavelets

177

Wavelet

177

Discrete wavelet transform

188

Fast wavelet transform

196

Haar wavelet

198

Filtering

205

Digital filter

205

Finite impulse response

211

Infinite impulse response

218

Nyquist ISI criterion

221

Pulse shaping

223

Raised-cosine filter

225

Root-raised-cosine filter

228

Adaptive filter

229

Kalman filter

234

Wiener filter

255

Receivers

260

References Article Sources and Contributors

261

Image Sources, Licenses and Contributors

265

Article Licenses License

268

Digital signal processing

Digital signal processing Digital signal processing (DSP) is the mathematical manipulation of an information signal to modify or improve it in some way. It is characterized by the representation of discrete time, discrete frequency, or other discrete domain signals by a sequence of numbers or symbols and the processing of these signals. The goal of DSP is usually to measure, filter and/or compress continuous real-world analog signals. The first step is usually to convert the signal from an analog to a digital form, by sampling and then digitizing it using an analog-to-digital converter (ADC), which turns the analog signal into a stream of numbers. However, often, the required output signal is another analog output signal, which requires a digital-to-analog converter (DAC). Even if this process is more complex than analog processing and has a discrete value range, the application of computational power to digital signal processing allows for many advantages over analog processing in many applications, such as error detection and correction in transmission as well as data compression. Digital signal processing and analog signal processing are subfields of signal processing. DSP applications include: audio and speech signal processing, sonar and radar signal processing, sensor array processing, spectral estimation, statistical signal processing, digital image processing, signal processing for communications, control of systems, biomedical signal processing, seismic data processing, etc. DSP algorithms have long been run on standard computers, as well as on specialized processors called digital signal processor and on purpose-built hardware such as application-specific integrated circuit (ASICs). Today there are additional technologies used for digital signal processing including more powerful general purpose microprocessors, field-programmable gate arrays (FPGAs), digital signal controllers (mostly for industrial apps such as motor control), and stream processors, among others. Digital signal processing can involve linear or nonlinear operations. Nonlinear signal processing is closely related to nonlinear system identification [1] and can be implemented in the time, frequency, and spatio-temporal domains.

Signal sampling Main article: Sampling (signal processing) With the increasing use of computers the usage of and need for digital signal processing has increased. To use an analog signal on a computer, it must be digitized with an analog-to-digital converter. Sampling is usually carried out in two stages, discretization and quantization. In the discretization stage, the space of signals is partitioned into equivalence classes and quantization is carried out by replacing the signal with representative signal of the corresponding equivalence class. In the quantization stage the representative signal values are approximated by values from a finite set. The Nyquist–Shannon sampling theorem states that a signal can be exactly reconstructed from its samples if the sampling frequency is greater than twice the highest frequency of the signal; but requires an infinite number of samples. In practice, the sampling frequency is often significantly more than twice that required by the signal's limited bandwidth. Some (continuous-time) periodic signals become non-periodic after sampling, and some non-periodic signals become periodic after sampling. In general, for a periodic signal with period T to be periodic (with period N) after sampling with sampling interval Ts, the following must be satisfied: where k is an integer.

1

Digital signal processing

DSP domains In DSP, engineers usually study digital signals in one of the following domains: time domain (one-dimensional signals), spatial domain (multidimensional signals), frequency domain, and wavelet domains. They choose the domain in which to process a signal by making an informed guess (or by trying different possibilities) as to which domain best represents the essential characteristics of the signal. A sequence of samples from a measuring device produces a time or spatial domain representation, whereas a discrete Fourier transform produces the frequency domain information, that is the frequency spectrum. Autocorrelation is defined as the cross-correlation of the signal with itself over varying intervals of time or space.

Time and space domains Main article: Time domain The most common processing approach in the time or space domain is enhancement of the input signal through a method called filtering. Digital filtering generally consists of some linear transformation of a number of surrounding samples around the current sample of the input or output signal. There are various ways to characterize filters; for example: • A "linear" filter is a linear transformation of input samples; other filters are "non-linear". Linear filters satisfy the superposition condition, i.e. if an input is a weighted linear combination of different signals, the output is an equally weighted linear combination of the corresponding output signals. • A "causal" filter uses only previous samples of the input or output signals; while a "non-causal" filter uses future input samples. A non-causal filter can usually be changed into a causal filter by adding a delay to it. • A "time-invariant" filter has constant properties over time; other filters such as adaptive filters change in time. • A "stable" filter produces an output that converges to a constant value with time, or remains bounded within a finite interval. An "unstable" filter can produce an output that grows without bounds, with bounded or even zero input. • A "finite impulse response" (FIR) filter uses only the input signals, while an "infinite impulse response" filter (IIR) uses both the input signal and previous samples of the output signal. FIR filters are always stable, while IIR filters may be unstable. A filter can be represented by a block diagram, which can then be used to derive a sample processing algorithm to implement the filter with hardware instructions. A filter may also be described as a difference equation, a collection of zeroes and poles or, if it is an FIR filter, an impulse response or step response. The output of a linear digital filter to any given input may be calculated by convolving the input signal with the impulse response.

Frequency domain Main article: Frequency domain Signals are converted from time or space domain to the frequency domain usually through the Fourier transform. The Fourier transform converts the signal information to a magnitude and phase component of each frequency. Often the Fourier transform is converted to the power spectrum, which is the magnitude of each frequency component squared. The most common purpose for analysis of signals in the frequency domain is analysis of signal properties. The engineer can study the spectrum to determine which frequencies are present in the input signal and which are missing. In addition to frequency information, phase information is often needed. This can be obtained from the Fourier transform. With some applications, how the phase varies with frequency can be a significant consideration.

2

Digital signal processing Filtering, particularly in non-realtime work can also be achieved by converting to the frequency domain, applying the filter and then converting back to the time domain. This is a fast, O(n log n) operation, and can give essentially any filter shape including excellent approximations to brickwall filters. There are some commonly used frequency domain transformations. For example, the cepstrum converts a signal to the frequency domain through Fourier transform, takes the logarithm, then applies another Fourier transform. This emphasizes the harmonic structure of the original spectrum. Frequency domain analysis is also called spectrum- or spectral analysis.

Z-plane analysis Main article: Z-transform Whereas analog filters are usually analyzed in terms of transfer functions in the s plane using Laplace transforms, digital filters are analyzed in the z plane in terms of Z-transforms. A digital filter may be described in the z plane by its characteristic collection of zeroes and poles. The z plane provides a means for mapping digital frequency (samples/second) to real and imaginary z components, where for continuous periodic signals and (

is the digital frequency). This is useful for providing a visualization of the frequency response of a

digital system or signal.

Wavelet Main article: Discrete wavelet transform In numerical analysis and functional analysis, a discrete wavelet transform (DWT) is any wavelet transform for which the wavelets are discretely sampled. As with other wavelet transforms, a key advantage it has over Fourier transforms is temporal resolution: it captures both frequency and location information (location in time).

Applications The main applications of DSP are audio signal processing, audio compression, digital image processing, video compression, speech processing, speech recognition, digital communications, RADAR, SONAR, Financial signal processing, seismology and biomedicine. Specific examples are speech compression An example of the 2D discrete wavelet transform that is used in JPEG2000. The and transmission in digital mobile phones, original image is high-pass filtered, yielding the three large images, each room correction of sound in hi-fi and sound describing local changes in brightness (details) in the original image. It is then low-pass filtered and downscaled, yielding an approximation image; this image is reinforcement applications, weather high-pass filtered to produce the three smaller detail images, and low-pass filtered forecasting, economic forecasting, seismic to produce the final approximation image in the upper-left. data processing, analysis and control of industrial processes, medical imaging such as CAT scans and MRI, MP3 compression, computer graphics, image manipulation, hi-fi loudspeaker crossovers and equalization, and audio effects for use with electric guitar amplifiers.

3

Digital signal processing

Implementation Depending on the requirements of the application, digital signal processing tasks can be implemented on general purpose computers (e.g. supercomputers, mainframe computers, or personal computers) or with embedded processors that may or may not include specialized microprocessors called digital signal processors. Often when the processing requirement is not real-time, processing is economically done with an existing general-purpose computer and the signal data (either input or output) exists in data files. This is essentially no different from any other data processing, except DSP mathematical techniques (such as the FFT) are used, and the sampled data is usually assumed to be uniformly sampled in time or space. For example: processing digital photographs with software such as Photoshop. However, when the application requirement is real-time, DSP is often implemented using specialized microprocessors such as the DSP56000, the TMS320, or the SHARC. These often process data using fixed-point arithmetic, though some more powerful versions use floating point arithmetic. For faster applications FPGAs might be used. Beginning in 2007, multicore implementations of DSPs have started to emerge from companies including Freescale and Stream Processors, Inc. For faster applications with vast usage, ASICs might be designed specifically. For slow applications, a traditional slower processor such as a microcontroller may be adequate. Also a growing number of DSP applications are now being implemented on Embedded Systems using powerful PCs with a Multi-core processor.

Techniques • • • • • • • • • •

Bilinear transform Discrete Fourier transform Discrete-time Fourier transform Filter design LTI system theory Minimum phase Transfer function Z-transform Goertzel algorithm s-plane

Related fields • • • • • • • • • • • •

Analog signal processing Automatic control Computer Engineering Computer Science Data compression Dataflow programming Electrical engineering Fourier Analysis Information theory Machine Learning Real-time computing Stream processing

• Telecommunication • Time series

4

Digital signal processing • Wavelet

References [1] Billings S.A. "Nonlinear System Identification: NARMAX Methods in the Time, Frequency, and Spatio-Temporal Domains". Wiley, 2013

Further reading • Alan V. Oppenheim, Ronald W. Schafer, John R. Buck : Discrete-Time Signal Processing, Prentice Hall, ISBN 0-13-754920-2 • Boaz Porat: A Course in Digital Signal Processing, Wiley, ISBN 0-471-14961-6 • Richard G. Lyons: Understanding Digital Signal Processing, Prentice Hall, ISBN 0-13-108989-7 • Jonathan Yaakov Stein, Digital Signal Processing, a Computer Science Perspective, Wiley, ISBN 0-471-29546-9 • Sen M. Kuo, Woon-Seng Gan: Digital Signal Processors: Architectures, Implementations, and Applications, Prentice Hall, ISBN 0-13-035214-4 • Bernard Mulgrew, Peter Grant, John Thompson: Digital Signal Processing - Concepts and Applications, Palgrave Macmillan, ISBN 0-333-96356-3 • Steven W. Smith (2002). Digital Signal Processing: A Practical Guide for Engineers and Scientists (http://www. dspguide.com). Newnes. ISBN 0-7506-7444-X. • Paul A. Lynn, Wolfgang Fuerst: Introductory Digital Signal Processing with Computer Applications, John Wiley & Sons, ISBN 0-471-97984-8 • James D. Broesch: Digital Signal Processing Demystified, Newnes, ISBN 1-878707-16-7 • John G. Proakis, Dimitris Manolakis: Digital Signal Processing: Principles, Algorithms and Applications, 4th ed, Pearson, April 2006, ISBN 978-0131873742 • Hari Krishna Garg: Digital Signal Processing Algorithms, CRC Press, ISBN 0-8493-7178-3 • P. Gaydecki: Foundations Of Digital Signal Processing: Theory, Algorithms And Hardware Design, Institution of Electrical Engineers, ISBN 0-85296-431-5 • Paul M. Embree, Damon Danieli: C++ Algorithms for Digital Signal Processing, Prentice Hall, ISBN 0-13-179144-3 • Vijay Madisetti, Douglas B. Williams: The Digital Signal Processing Handbook, CRC Press, ISBN 0-8493-8572-5 • Stergios Stergiopoulos: Advanced Signal Processing Handbook: Theory and Implementation for Radar, Sonar, and Medical Imaging Real-Time Systems, CRC Press, ISBN 0-8493-3691-0 • Joyce Van De Vegte: Fundamentals of Digital Signal Processing, Prentice Hall, ISBN 0-13-016077-6 • Ashfaq Khan: Digital Signal Processing Fundamentals, Charles River Media, ISBN 1-58450-281-9 • Jonathan M. Blackledge, Martin Turner: Digital Signal Processing: Mathematical and Computational Methods, Software Development and Applications, Horwood Publishing, ISBN 1-898563-48-9 • Doug Smith: Digital Signal Processing Technology: Essentials of the Communications Revolution, American Radio Relay League, ISBN 0-87259-819-5 • Charles A. Schuler: Digital Signal Processing: A Hands-On Approach, McGraw-Hill, ISBN 0-07-829744-3 • James H. McClellan, Ronald W. Schafer, Mark A. Yoder: Signal Processing First, Prentice Hall, ISBN 0-13-090999-8 • John G. Proakis: A Self-Study Guide for Digital Signal Processing, Prentice Hall, ISBN 0-13-143239-7 • N. Ahmed and K.R. Rao (1975). Orthogonal Transforms for Digital Signal Processing. Springer-Verlag (Berlin – Heidelberg – New York), ISBN 3-540-06556-3.

5

Discrete signal

6

Discrete signal A discrete signal or discrete-time signal is a time series consisting of a sequence of quantities. In other words, it is a time series that is a function over a domain of integers. Unlike a continuous-time signal, a discrete-time signal is not a function of a continuous argument; however, it may have been obtained by sampling from a continuous-time signal, and then each value in the sequence is called a sample. When a discrete-time signal obtained by sampling a sequence corresponding to uniformly spaced times, it has an associated sampling rate; the sampling rate is not apparent in the data sequence, and so needs to be associated as a characteristic unit of the system.

Discrete sampled signal

Acquisition Discrete signals may have several origins, but can usually be classified into one of two groups:[1] • By acquiring values of an analog signal at constant or variable rate. This process is called sampling.[2]

Digital signal

• By recording the number of events of a given kind over finite time periods. For example, this could be the number of people taking a certain elevator every day.

Digital signals A digital signal is a discrete-time signal for which not only the time but also the amplitude has been made discrete; in other words, its samples take on only values from a discrete set (a countable set that can be mapped one-to-one to a subset of integers). If that discrete set is finite, the discrete values can be represented with digital words of a finite width. Most commonly, these discrete values are represented as fixed-point words (either proportional to the waveform values or companded) or floating-point words.

Discrete cosine waveform with frequency of 50 Hz and a sampling rate of 1000 samples/sec, easily satisfying the sampling theorem for reconstruction of the original cosine function from samples.

The process of converting a continuous-valued discrete-time signal to a digital (discrete-valued discrete-time) signal is known as analog-to-digital conversion. It usually proceeds by replacing each original sample value by an approximation selected from a given discrete set (for example by truncating or rounding, but much more sophisticated methods exist), a process known as quantization. This process loses information, and so discrete-valued signals are only an

Discrete signal approximation of the converted continuous-valued discrete-time signal, itself only an approximation of the original continuous-valued continuous-time signal. Common practical digital signals are represented as 8-bit (256 levels), 16-bit (65,536 levels), 32-bit (4.3 billion levels), and so on, though any number of quantization levels is possible, not just powers of two.

References [1] "Digital Signal Processing" Prentice Hall - Pages 11-12 [2] "Digital Signal Processing: Instant access." Butterworth-Heinemann - Page 8

• Gershenfeld, Neil A. (1999). The Nature of mathematical Modeling. Cambridge University Press. ISBN 0-521-57095-6. • Wagner, Thomas Charles Gordon (1959). Analytical transients. Wiley.

7

8

Sampling Sampling (signal processing) In signal processing, sampling is the reduction of a continuous signal to a discrete signal. A common example is the conversion of a sound wave (a continuous signal) to a sequence of samples (a discrete-time signal). A sample refers to a value or set of values at a point in time and/or space. A sampler is a subsystem or operation that extracts samples from a continuous signal. A theoretical ideal sampler produces samples equivalent to the instantaneous value of the continuous signal at the desired points.

Signal sampling representation. The continuous signal is represented with a green colored line while the discrete samples are indicated by the blue vertical lines.

Theory See also: Nyquist–Shannon sampling theorem Sampling can be done for functions varying in space, time, or any other dimension, and similar results are obtained in two or more dimensions. For functions that vary with time, let s(t) be a continuous function (or "signal") to be sampled, and let sampling be performed by measuring the value of the continuous function every T seconds, which is called the sampling interval. Then the sampled function is given by the sequence: s(nT), for integer values of n. The sampling frequency or sampling rate, fs, is defined as the number of samples obtained in one second (samples per second), thus fs = 1/T. Reconstructing a continuous function from samples is done by interpolation algorithms. The Whittaker–Shannon interpolation formula is mathematically equivalent to an ideal lowpass filter whose input is a sequence of Dirac delta functions that are modulated (multiplied) by the sample values. When the time interval between adjacent samples is a constant (T), the sequence of delta functions is called a Dirac comb. Mathematically, the modulated Dirac comb is equivalent to the product of the comb function with s(t). That purely mathematical abstraction is sometimes referred to as impulse sampling. Most sampled signals are not simply stored and reconstructed. But the fidelity of a theoretical reconstruction is a customary measure of the effectiveness of sampling. That fidelity is reduced when s(t) contains frequency components higher than fs/2 Hz, which is known as the Nyquist frequency of the sampler. Therefore s(t) is usually the output of a lowpass filter, functionally known as an anti-aliasing filter. Without an anti-aliasing filter, frequencies higher than the Nyquist frequency will influence the samples in a way that is misinterpreted by the interpolation process.[1] For details, see Aliasing.

Sampling (signal processing)

Practical considerations In practice, the continuous signal is sampled using an analog-to-digital converter (ADC), a device with various physical limitations. This results in deviations from the theoretically perfect reconstruction, collectively referred to as distortion. Various types of distortion can occur, including: • Aliasing. Some amount of aliasing is inevitable because only theoretical, infinitely long, functions can have no frequency content above the Nyquist frequency. Aliasing can be made arbitrarily small by using a sufficiently large order of the anti-aliasing filter. • Aperture error results from the fact that the sample is obtained as a time average within a sampling region, rather than just being equal to the signal value at the sampling instant. In a capacitor-based sample and hold circuit, aperture error is introduced because the capacitor cannot instantly change voltage thus requiring the sample to have non-zero width. • Jitter or deviation from the precise sample timing intervals. • Noise, including thermal sensor noise, analog circuit noise, etc. • Slew rate limit error, caused by the inability of the ADC input value to change sufficiently rapidly. • Quantization as a consequence of the finite precision of words that represent the converted values. • Error due to other non-linear effects of the mapping of input voltage to converted output value (in addition to the effects of quantization). Although the use of oversampling can completely eliminate aperture error and aliasing by shifting them out of the pass band, this technique cannot be practically used above a few GHz, and may be prohibitively expensive at much lower frequencies. Furthermore, while oversampling can reduce quantization error and non-linearity, it cannot eliminate these entirely. Consequently, practical ADCs at audio frequencies typically do not exhibit aliasing, aperture error, and are not limited by quantization error. Instead, analog noise dominates. At RF and microwave frequencies where oversampling is impractical and filters are expensive, aperture error, quantization error and aliasing can be significant limitations. Jitter, noise, and quantization are often analyzed by modeling them as random errors added to the sample values. Integration and zero-order hold effects can be analyzed as a form of low-pass filtering. The non-linearities of either ADC or DAC are analyzed by replacing the ideal linear function mapping with a proposed nonlinear function.

Applications Audio sampling Digital audio uses pulse-code modulation and digital signals for sound reproduction. This includes analog-to-digital conversion (ADC), digital-to-analog conversion (DAC), storage, and transmission. In effect, the system commonly referred to as digital is in fact a discrete-time, discrete-level analog of a previous electrical analog. While modern systems can be quite subtle in their methods, the primary usefulness of a digital system is the ability to store, retrieve and transmit signals without any loss of quality. Sampling rate When it is necessary to capture audio covering the entire 20–20,000 Hz range of human hearing, such as when recording music or many types of acoustic events, audio waveforms are typically sampled at 44.1 kHz (CD), 48 kHz (professional audio), 88.2 kHz, or 96 kHz. The approximately double-rate requirement is a consequence of the Nyquist theorem. Sampling rates higher than about 50 kHz to 60 kHz cannot supply more usable information for human listeners. Early professional audio equipment manufacturers chose sampling rates in the region of 50 kHz for this reason.

9

Sampling (signal processing)

10

There has been an industry trend towards sampling rates well beyond the basic requirements: such as 96 kHz and even 192 kHz This is in contrast with laboratory experiments, which have failed to show that ultrasonic frequencies are audible to human observers; however in some cases ultrasonic sounds do interact with and modulate the audible part of the frequency spectrum (intermodulation distortion). It is noteworthy that intermodulation distortion is not present in the live audio and so it represents an artificial coloration to the live sound. One advantage of higher sampling rates is that they can relax the low-pass filter design requirements for ADCs and DACs, but with modern oversampling sigma-delta converters this advantage is less important. The Audio Engineering Society recommends 48 kHz sample rate for most applications but gives recognition to 44.1 kHz for Compact Disc and other consumer uses, 32 kHz for transmission-related application, and 96 kHz for higher bandwidth or relaxed anti-aliasing filtering. A more complete list of common audio sample rates is: Sampling rate

Use

8,000 Hz

[2] Telephone and encrypted walkie-talkie, wireless intercom and wireless microphone transmission; adequate for human speech but without sibilance; ess sounds like eff (/s/, /f/).

11,025 Hz

One quarter the sampling rate of audio CDs; used for lower-quality PCM, MPEG audio and for audio analysis of subwoofer bandpasses.Wikipedia:Citation needed

16,000 Hz

Wideband frequency extension over standard telephone narrowband 8,000 Hz. Used in most modern VoIP and VVoIP [3] communication products.

22,050 Hz

One half the sampling rate of audio CDs; used for lower-quality PCM and MPEG audio and for audio analysis of low frequency energy. Suitable for digitizing early 20th century audio formats such as 78s.

32,000 Hz

miniDV digital video camcorder, video tapes with extra channels of audio (e.g. DVCAM with 4 Channels of audio), DAT (LP mode), Germany's Digitales Satellitenradio, NICAM digital audio, used alongside analogue television sound in some countries. High-quality digital wireless microphones. Suitable for digitizing FM radio.Wikipedia:Citation needed

44,056 Hz

Used by digital audio locked to NTSC color video signals (245 lines by 3 samples by 59.94 fields per second = 29.97 frames per second).

44,100 Hz

Audio CD, also most commonly used with MPEG-1 audio (VCD, SVCD, MP3). Originally chosen by Sony because it could be recorded on modified video equipment running at either 25 frames per second (PAL) or 30 frame/s (using an NTSC monochrome video recorder) and cover the 20 kHz bandwidth thought necessary to match professional analog recording equipment of the time. A PCM adaptor would fit digital audio samples into the analog video channel of, for example, PAL video tapes using 588 lines by 3 samples by 25 frames per second.

47,250 Hz

world's first commercial PCM sound recorder by Nippon Columbia (Denon)

48,000 Hz

The standard audio sampling rate used by professional digital video equipment such as tape recorders, video servers, vision mixers and so on. This rate was chosen because it could deliver a 22 kHz frequency response and work with 29.97 frames per second NTSC video - as well as 25 frame/s, 30 frame/s and 24 frame/s systems. With 29.97 frame/s systems it is necessary to handle 1601.6 audio samples per frame delivering an integer number of audio samples only every fifth video frame. Also used for sound with consumer video formats like DV, digital TV, DVD, and films. The professional Serial Digital Interface (SDI) and High-definition Serial Digital Interface (HD-SDI) used to connect broadcast television equipment together uses this audio sampling frequency. Most professional audio gear uses 48 kHz sampling, including mixing consoles, and digital recording devices.

50,000 Hz

First commercial digital audio recorders from the late 70s from 3M and Soundstream.

50,400 Hz

Sampling rate used by the Mitsubishi X-80 digital audio recorder.

88,200 Hz

Sampling rate used by some professional recording equipment when the destination is CD (multiples of 44,100 Hz). Some pro audio gear uses (or is able to select) 88.2 kHz sampling, including mixers, EQs, compressors, reverb, crossovers and recording devices.

96,000 Hz

DVD-Audio, some LPCM DVD tracks, BD-ROM (Blu-ray Disc) audio tracks, HD DVD (High-Definition DVD) audio tracks. Some professional recording and production equipment is able to select 96 kHz sampling. This sampling frequency is twice the 48 kHz standard commonly used with audio on professional equipment.

176,400 Hz

Sampling rate used by HDCD recorders and other professional applications for CD production.

Sampling (signal processing)

192,000 Hz

DVD-Audio, some LPCM DVD tracks, BD-ROM (Blu-ray Disc) audio tracks, and HD DVD (High-Definition DVD) audio tracks, High-Definition audio recording devices and audio editing software. This sampling frequency is four times the 48 kHz standard commonly used with audio on professional video equipment.

352,800 Hz

Digital eXtreme Definition, used for recording and editing Super Audio CDs, as 1-bit DSD is not suited for editing. Eight times the frequency of 44.1 kHz.

2,822,400 Hz SACD, 1-bit delta-sigma modulation process known as Direct Stream Digital, co-developed by Sony and Philips. 5,644,800 Hz Double-Rate DSD, 1-bit Direct Stream Digital at 2x the rate of the SACD. Used in some professional DSD recorders.

Bit depth See also: Audio bit depth Audio is typically recorded at 8-, 16-, and 20-bit depth, which yield a theoretical maximum Signal-to-quantization-noise ratio (SQNR) for a pure sine wave of, approximately, 49.93 dB, 98.09 dB and 122.17 dB. CD quality audio uses 16-bit samples. Thermal noise limits the true number of bits that can be used in quantization. Few analog systems have signal to noise ratios (SNR) exceeding 120 dB. However, digital signal processing operations can have very high dynamic range, consequently it is common to perform mixing and mastering operations at 32-bit precision and then convert to 16 or 24 bit for distribution. Speech sampling Speech signals, i.e., signals intended to carry only human speech, can usually be sampled at a much lower rate. For most phonemes, almost all of the energy is contained in the 5Hz-4 kHz range, allowing a sampling rate of 8 kHz. This is the sampling rate used by nearly all telephony systems, which use the G.711 sampling and quantization specifications.

Video sampling Standard-definition television (SDTV) uses either 720 by 480 pixels (US NTSC 525-line) or 704 by 576 pixels (UK PAL 625-line) for the visible picture area. High-definition television (HDTV) uses 720p (progressive), 1080i (interlaced), and 1080p (progressive, also known as Full-HD). In digital video, the temporal sampling rate is defined the frame rate – or rather the field rate – rather than the notional pixel clock. The image sampling frequency is the repetition rate of the sensor integration period. Since the integration period may be significantly shorter than the time between repetitions, the sampling frequency can be different from the inverse of the sample time: • 50 Hz – PAL video • 60 / 1.001 Hz ~= 59.94 Hz – NTSC video Video digital-to-analog converters operate in the megahertz range (from ~3 MHz for low quality composite video scalers in early games consoles, to 250 MHz or more for the highest-resolution VGA output). When analog video is converted to digital video, a different sampling process occurs, this time at the pixel frequency, corresponding to a spatial sampling rate along scan lines. A common pixel sampling rate is: • 13.5 MHz – CCIR 601, D1 video Spatial sampling in the other direction is determined by the spacing of scan lines in the raster. The sampling rates and resolutions in both spatial directions can be measured in units of lines per picture height. Spatial aliasing of high-frequency luma or chroma video components shows up as a moiré pattern.

11

Sampling (signal processing)

12

3D sampling • X-ray computed tomography uses 3 dimensional space • Voxel

Undersampling Main article: Undersampling When a bandpass signal is sampled slower than its Nyquist rate, the samples are indistinguishable from samples of a low-frequency alias of the high-frequency signal. That is often done purposefully in such a way that the lowest-frequency alias satisfies the Nyquist criterion, because the bandpass signal is still uniquely represented and recoverable. Such undersampling is also known as bandpass sampling, harmonic sampling, IF sampling, and direct IF to digital conversion.

The top 2 graphs depict Fourier transforms of 2 different functions that produce the same results when sampled at a particular rate. The baseband function is sampled faster than its Nyquist rate, and the bandpass function is undersampled, effectively converting it to baseband. The lower graphs indicate how identical spectral results are created by the aliases of the sampling process.

Oversampling Main article: Oversampling Oversampling is used in most modern analog-to-digital converters to reduce the distortion introduced by practical digital-to-analog converters, such as a zero-order hold instead of idealizations like the Whittaker–Shannon interpolation formula.

Complex sampling Complex sampling (I/Q sampling) refers to the simultaneous sampling of two different, but related, waveforms, resulting in pairs of samples that are subsequently treated as complex numbers.[4] When one waveform is the Hilbert transform of the other waveform

the complex-valued function,

is called an analytic signal, whose Fourier transform is zero for all negative values of frequency. In that case, the Nyquist rate for a waveform with no frequencies ≥ B can be reduced to just B (complex samples/sec), instead of 2B (real samples/sec).[5] More apparently, the equivalent baseband waveform, also has a Nyquist rate of B, because all of its non-zero frequency content is shifted into the interval [-B/2, B/2). Although complex-valued samples can be obtained as described above, they are also created by manipulating samples of a real-valued waveform. For instance, the equivalent baseband waveform can be created without [6] explicitly computing by processing the product sequence through a digital lowpass filter whose cutoff frequency is B/2.[7] Computing only every other sample of the output sequence reduces the sample-rate commensurate with the reduced Nyquist rate. The result is half as many complex-valued samples as the original number of real samples. No information is lost, and the original s(t) waveform can be recovered, if necessary.

Sampling (signal processing)

13

Notes [1] C. E. Shannon, "Communication in the presence of noise", Proc. Institute of Radio Engineers, vol. 37, no.1, pp. 10–21, Jan. 1949. Reprint as classic paper in: Proc. IEEE, Vol. 86, No. 2, (Feb 1998) (http:/ / www. stanford. edu/ class/ ee104/ shannonpaper. pdf) [2] HME DX200 encrypted wireless intercom (http:/ / www. hme. com/ proDX200. cfm) [3] http:/ / www. voipsupply. com/ cisco-hd-voice [4] Sample-pairs are also sometimes viewed as points on a constellation diagram. [5] When the complex sample-rate is B, a frequency component at 0.6 B, for instance, will have an alias at −0.4 B, which is unambiguous because of the constraint that the pre-sampled signal was analytic. Also see Aliasing#Complex_sinusoids [6] When s(t) is sampled at the Nyquist frequency (1/T = 2B), the product sequence simplifies to UNIQ-math-0-fdcc463dc40e329d-QINU [7] The sequence of complex numbers is convolved with the impulse response of a filter with real-valued coefficients. That is equivalent to separately filtering the sequences of real parts and imaginary parts and reforming complex pairs at the outputs.

Citations Further reading • Matt Pharr and Greg Humphreys, Physically Based Rendering: From Theory to Implementation, Morgan Kaufmann, July 2004. ISBN 0-12-553180-X. The chapter on sampling ( available online (http://graphics. stanford.edu/~mmp/chapters/pbrt_chapter7.pdf)) is nicely written with diagrams, core theory and code sample.

External links • Journal devoted to Sampling Theory (http://www.stsip.org) • I/Q Data for Dummies (http://whiteboard.ping.se/SDR/IQ) A page trying to answer the question Why I/Q Data?

Sample and hold For Neil Young song, see Trans (album). For remix album by Simian Mobile Disco, see Sample and Hold. In electronics, a sample and hold (S/H, also "follow-and-hold"[1]) circuit is an analog device that samples (captures, grabs) the voltage of a continuously varying analog signal and holds (locks, freezes) its value at a constant level for a specified minimum period of time. Sample and hold circuits and related peak detectors are the elementary analog memory devices. They are typically used in analog-to-digital converters to eliminate variations in input signal that can corrupt the conversion process.[2] A typical sample and hold circuit stores electric charge in a capacitor and contains at least one fast FET switch and at least one operational amplifier. To sample the input signal the switch connects the capacitor to the output of a buffer amplifier. The buffer amplifier charges or discharges the capacitor so that the voltage across the capacitor is practically equal, or proportional

A simplified sample and hold circuit diagram. AI is an analog input, AO — an analog output, C — a control signal.

Sample times.

Sample and hold

to, input voltage. In hold mode the switch disconnects the capacitor from the buffer. The capacitor is invariably discharged by its own leakage currents and useful load currents, which makes the circuit inherently volatile, but the loss of voltage (voltage drop) within a specified hold time remains within an acceptable error margin. For practically all commercial liquid crystal active matrix displays based on TN, IPS or VA electro-optic LC cells (excluding bi-stable Sample and hold. phenomena), each pixel represents a small capacitor, which has to be periodically charged to a level corresponding to the greyscale value (contrast) desired for a picture element. In order to maintain the level during a scanning cycle (frame period), an additional electric capacitor is attached in parallel to each LC pixel to better hold the voltage. A thin-film FET switch is addressed to select a particular LC pixel and charge the picture information for it. In contrast to an S/H in general electronics, there is no output operational amplifier and no electrical signal AO. Instead, the charge on the hold capacitors controls the deformation of the LC molecules and thereby the optical effect as its output. The invention of this concept and its implementation in thin-film technology have been honored with the IEEE Jun-ichi Nishizawa Medal in 2011.[3] During a scanning cycle, the picture doesn’t follow the input signal. This does not allow the eye to refresh and can lead to blurring during motion sequences, also the transition is visible between frames because the backlight is constantly illuminated, adding to display motion blur.[4][5]

Purpose Sample and hold circuits are used in linear systems. In some kinds of analog-to-digital converters, the input is compared to a voltage generated internally from a digital-to-analog converter (DAC). The circuit tries a series of values and stops converting once the voltages are equal, within some defined error margin. If the input value was permitted to change during this comparison process, the resulting conversion would be inaccurate and possibly completely unrelated to the true input value. Such successive approximation converters will often incorporate internal sample and hold circuitry. In addition, sample and hold circuits are often used when multiple samples need to be measured at the same time. Each value is sampled and held, using a common sample clock.

Implementation To keep the input voltage as stable as possible, it is essential that the capacitor have very low leakage, and that it not be loaded to any significant degree which calls for a very high input impedance.

Notes [1] [2] [3] [4]

Horowitz and Hill, p. 220. Kefauver and Patschke, p. 37. Press release IEEE, Aug. 2011 (http:/ / www. ieee. org/ about/ news/ 2011/ honors_ceremony/ releases_nishizawa. html) Charles Poynton is an authority on artifacts related to HDTV, and discusses motion artifacts succinctly and specifically (http:/ / www. poynton. com/ PDFs/ Motion_portrayal. pdf) [5] Eye-tracking based motion blur on LCD (http:/ / ieeexplore. ieee. org/ xpls/ abs_all. jsp?arnumber=5583881& tag=1)

14

Sample and hold

References • Paul Horowitz, Winfield Hill (2001 ed.). The Art of Electronics (http://books.google.com/ books?id=bkOMDgwFA28C&pg=PA220&dq=sample+and+hold&cd=1#v=onepage&q=sample and hold& f=false). Cambridge University Press. ISBN 0-521-37095-7. • Alan P. Kefauver, David Patschke (2007). Fundamentals of digital audio (http://books.google.com/ books?id=UpzqCrj7QxYC&pg=PA60&dq=sample+and+hold&cd=7#v=onepage&q=sample and hold& f=false). A-R Editions, Inc. ISBN 0-89579-611-2. • Analog Devices 21 page Tutorial "Sample and Hold Amplifiers" http://www.analog.com/static/imported-files/ tutorials/MT-090.pdf • Ndjountche, Tertulien (2011). CMOS Analog Integrated Circuits: High-Speed and Power-Efficient Design (http:/ /www.crcpress.com/ecommerce_product/product_detail.jsf?isbn=0&catno=k12557). Boca Raton, FL, USA: CRC Press. p. 925. ISBN 978-1-4398-5491-4. • Applications of Monolithic Sample and hold Amplifiers-Intersil (http://www.intersil.com/data/an/an517.pdf)

Digital-to-analog converter For digital television converter boxes, see digital television adapter. In electronics, a digital-to-analog converter (DAC, D/A, D2A or D-to-A) is a function that converts digital data (usually binary) into an analog signal (current, voltage, or electric charge). An analog-to-digital converter (ADC) performs the reverse function. Unlike analog signals, digital data can be transmitted, manipulated, and stored without degradation, albeit with more complex equipment. But a DAC is needed to convert the digital signal to analog to drive an earphone or loudspeaker amplifier in order to produce sound (analog air pressure waves). DACs and their inverse, ADCs, are part of an enabling technology 8-channel digital-to-analog converter Cirrus Logic CS4382 as used in a soundcard. that has contributed greatly to the digital revolution. To illustrate, consider a typical long-distance telephone call. The caller's voice is converted into an analog electrical signal by a microphone, then the analog signal is converted to a digital stream by an ADC. The digital stream is then divided into packets where it may be mixed with other digital data, not necessarily audio. The digital packets are then sent to the destination, but each packet may take a completely different route and may not even arrive at the destination in the correct time order. The digital voice data is then extracted from the packets and assembled into a digital data stream. A DAC converts this into an analog electrical signal, which drives an audio amplifier, which in turn drives a loudspeaker, which finally produces sound. There are several DAC architectures; the suitability of a DAC for a particular application is determined by six main parameters: physical size, power consumption, resolution, speed, accuracy, cost. Due to the complexity and the need for precisely matched components, all but the most specialist DACs are implemented as integrated circuits (ICs). Digital-to-analog conversion can degrade a signal, so a DAC should be specified that that has insignificant errors in terms of the application. DACs are commonly used in music players to convert digital data streams into analog audio signals. They are also used in televisions and mobile phones to convert digital video data into analog video signals which connect to the screen drivers to display monochrome or color images. These two applications use DACs at opposite ends of the speed/resolution trade-off. The audio DAC is a low speed high resolution type while the video DAC is a high speed low to medium resolution type. Discrete DACs would typically be extremely high speed low resolution power

15

Digital-to-analog converter

16

hungry types, as used in military radar systems. Very high speed test equipment, especially sampling oscilloscopes, may also use discrete DACS.

Overview A DAC converts an abstract finite-precision number (usually a fixed-point binary number) into a physical quantity (e.g., a voltage or a pressure). In particular, DACs are often used to convert finite-precision time series data to a continually varying physical signal. A typical DAC converts the abstract numbers into a concrete sequence of impulses that are then processed by a reconstruction filter using some form of interpolation to fill in data between the impulses. Other DAC methods (e.g., methods based on delta-sigma modulation) produce a pulse-density modulated signal that can then be filtered in a similar way to produce a smoothly varying signal.

Ideally sampled signal.

As per the Nyquist–Shannon sampling theorem, a DAC can reconstruct the original signal from the sampled data provided that its bandwidth meets certain requirements (e.g., a baseband signal with bandwidth less than the Nyquist frequency). Digital sampling introduces quantization error that manifests as low-level noise added to the reconstructed signal.

Practical operation Instead of impulses, usually the sequence of numbers update the analog voltage at uniform sampling intervals, which are then often interpolated via a reconstruction filter to continuously varied levels. These numbers are written to the DAC, typically with a clock signal that causes each number to be latched in sequence, at which time the DAC output voltage changes rapidly from the previous value to the value represented by the currently latched number. The effect of this is that the output voltage is held in time at the current value until the next input number is latched, resulting in a piecewise constant or staircase-shaped output. This is equivalent to a zero-order hold operation and has an effect on the frequency response of the reconstructed signal.

Piecewise constant output of an idealized DAC lacking a reconstruction filter. In a practical DAC, a filter or the finite bandwidth of the device smooths out the step response into a continuous curve.

The fact that DACs output a sequence of piecewise constant values (known as zero-order hold in sample data textbooks) or rectangular pulses causes multiple harmonics above the Nyquist frequency. Usually, these are removed with a low pass filter acting as a reconstruction filter in applications that require it.

Digital-to-analog converter

17

Applications Audio Most modern audio signals are stored in digital form (for example MP3s and CDs) and in order to be heard through speakers they must be converted into an analog signal. DACs are therefore found in CD players, digital music players, and PC sound cards.

A simplified functional diagram of an 8-bit DAC

Specialist standalone DACs can also be found in high-end hi-fi systems. These normally take the digital output of a compatible CD player or dedicated transport (which is basically a CD player with no internal DAC) and convert the signal into an analog line-level output that can then be fed into an amplifier to drive speakers. Similar digital-to-analog converters can be found in digital speakers such as USB speakers, and in sound cards. In VoIP (Voice over IP) applications, the source must first be digitized for transmission, so it undergoes conversion via an analog-to-digital converter, and is then reconstructed into analog using a DAC on the receiving party's end.

Video Video sampling tends to work on a completely different scale altogether thanks to the highly nonlinear response both of cathode ray tubes (for which the vast majority of digital video foundation work was targeted) and the human eye, using a "gamma curve" to provide an appearance of evenly distributed brightness steps across the display's full dynamic range - hence the need to use RAMDACs in computer video applications with deep enough colour resolution to make engineering a hardcoded value into the DAC for each output level of Top-loading CD player and external each channel impractical (e.g. an Atari ST or Sega Genesis would digital-to-analog converter. require 24 such values; a 24-bit video card would need 768...). Given this inherent distortion, it is not unusual for a television or video projector to truthfully claim a linear contrast ratio (difference between darkest and brightest output levels) of 1000:1 or greater, equivalent to 10 bits of audio precision even though it may only accept signals with 8-bit precision and use an LCD panel that only represents 6 or 7 bits per channel. Video signals from a digital source, such as a computer, must be converted to analog form if they are to be displayed on an analog monitor. As of 2007, analog inputs were more commonly used than digital, but this changed as flat panel displays with DVI and/or HDMI connections became more widespread.Wikipedia:Citation needed A video DAC is, however, incorporated in any digital video player with analog outputs. The DAC is usually integrated with some memory (RAM), which contains conversion tables for gamma correction, contrast and brightness, to make a device called a RAMDAC. A device that is distantly related to the DAC is the digitally controlled potentiometer, used to control an analog signal digitally.

Digital-to-analog converter

Mechanical An unusual application of digital-to-analog conversion was the whiffletree electromechanical digital-to-analog converter linkage in the IBM Selectric typewriter.Wikipedia:Citation needed

DAC types The most common types of electronic DACs are: • The pulse-width modulator, the simplest DAC type. A stable current or voltage is switched into a low-pass analog filter with a duration determined by the digital input code. This technique is often used for electric motor speed control, but has many other applications as well. • Oversampling DACs or interpolating DACs such as the delta-sigma DAC, use a pulse density conversion technique. The oversampling technique allows for the use of a lower resolution DAC internally. A simple 1-bit DAC is often chosen because the oversampled result is inherently linear. The DAC is driven with a pulse-density modulated signal, created with the use of a low-pass filter, step nonlinearity (the actual 1-bit DAC), and negative feedback loop, in a technique called delta-sigma modulation. This results in an effective high-pass filter acting on the quantization (signal processing) noise, thus steering this noise out of the low frequencies of interest into the megahertz frequencies of little interest, which is called noise shaping. The quantization noise at these high frequencies is removed or greatly attenuated by use of an analog low-pass filter at the output (sometimes a simple RC low-pass circuit is sufficient). Most very high resolution DACs (greater than 16 bits) are of this type due to its high linearity and low cost. Higher oversampling rates can relax the specifications of the output low-pass filter and enable further suppression of quantization noise. Speeds of greater than 100 thousand samples per second (for example, 192 kHz) and resolutions of 24 bits are attainable with delta-sigma DACs. A short comparison with pulse-width modulation shows that a 1-bit DAC with a simple first-order integrator would have to run at 3 THz (which is physically unrealizable) to achieve 24 meaningful bits of resolution, requiring a higher-order low-pass filter in the noise-shaping loop. A single integrator is a low-pass filter with a frequency response inversely proportional to frequency and using one such integrator in the noise-shaping loop is a first order delta-sigma modulator. Multiple higher order topologies (such as MASH) are used to achieve higher degrees of noise-shaping with a stable topology. • The binary-weighted DAC, which contains individual electrical components for each bit of the DAC connected to a summing point. These precise voltages or currents sum to the correct output value. This is one of the fastest conversion methods but suffers from poor accuracy because of the high precision required for each individual voltage or current. Such high-precision components are expensive, so this type of converter is usually limited to 8-bit resolution or less.Wikipedia:Citation needed • Switched resistor DAC contains of a parallel resistor network. Individual resistors are enabled or bypassed in the network based on the digital input. • Switched current source DAC, from which different current sources are selected based on the digital input. • Switched capacitor DAC contains a parallel capacitor network. Individual capacitors are connected or disconnected with switches based on the input. • The R-2R ladder DAC which is a binary-weighted DAC that uses a repeating cascaded structure of resistor values R and 2R. This improves the precision due to the relative ease of producing equal valued-matched resistors (or current sources). However, wide converters perform slowly due to increasingly large RC-constants for each added R-2R link. • The Successive-Approximation or Cyclic DAC, which successively constructs the output during each cycle. Individual bits of the digital input are processed each cycle until the entire input is accounted for. • The thermometer-coded DAC, which contains an equal resistor or current-source segment for each possible value of DAC output. An 8-bit thermometer DAC would have 255 segments, and a 16-bit thermometer DAC would have 65,535 segments. This is perhaps the fastest and highest precision DAC architecture but at the expense of

18

Digital-to-analog converter high cost. Conversion speeds of >1 billion samples per second have been reached with this type of DAC. • Hybrid DACs, which use a combination of the above techniques in a single converter. Most DAC integrated circuits are of this type due to the difficulty of getting low cost, high speed and high precision in one device. • The segmented DAC, which combines the thermometer-coded principle for the most significant bits and the binary-weighted principle for the least significant bits. In this way, a compromise is obtained between precision (by the use of the thermometer-coded principle) and number of resistors or current sources (by the use of the binary-weighted principle). The full binary-weighted design means 0% segmentation, the full thermometer-coded design means 100% segmentation. • Most DACs, shown earlier in this list, rely on a constant reference voltage to create their output value. Alternatively, a multiplying DAC takes a variable input voltage for their conversion. This puts additional design constraints on the bandwidth of the conversion circuit.

DAC performance DACs are very important to system performance. The most important characteristics of these devices are: Resolution The number of possible output levels the DAC is designed to reproduce. This is usually stated as the number of bits it uses, which is the base two logarithm of the number of levels. For instance a 1 bit DAC is designed to reproduce 2 (21) levels while an 8 bit DAC is designed for 256 (28) levels. Resolution is related to the effective number of bits which is a measurement of the actual resolution attained by the DAC. Resolution determines color depth in video applications and audio bit depth in audio applications. Maximum sampling rate A measurement of the maximum speed at which the DACs circuitry can operate and still produce the correct output. As stated in the Nyquist–Shannon sampling theorem defines a relationship between the sampling frequency and bandwidth of the sampled signal. Monotonicity The ability of a DAC's analog output to move only in the direction that the digital input moves (i.e., if the input increases, the output doesn't dip before asserting the correct output.) This characteristic is very important for DACs used as a low frequency signal source or as a digitally programmable trim element. Total harmonic distortion and noise (THD+N) A measurement of the distortion and noise introduced to the signal by the DAC. It is expressed as a percentage of the total power of unwanted harmonic distortion and noise that accompany the desired signal. This is a very important DAC characteristic for dynamic and small signal DAC applications. Dynamic range A measurement of the difference between the largest and smallest signals the DAC can reproduce expressed in decibels. This is usually related to resolution and noise floor. Other measurements, such as phase distortion and jitter, can also be very important for some applications, some of which (e.g. wireless data transmission, composite video) may even rely on accurate production of phase-adjusted signals. Linear PCM audio sampling usually works on the basis of each bit of resolution being equivalent to 6 decibels of amplitude (a 2x increase in volume or precision). Non-linear PCM encodings (A-law / μ-law, ADPCM, NICAM) attempt to improve their effective dynamic ranges by a variety of methods - logarithmic step sizes between the output signal strengths represented by each data bit (trading greater quantisation distortion of loud signals for better performance of quiet signals)

19

Digital-to-analog converter

DAC figures of merit • Static performance: • Differential nonlinearity (DNL) shows how much two adjacent code analog values deviate from the ideal 1 LSB step.[1] • Integral nonlinearity (INL) shows how much the DAC transfer characteristic deviates from an ideal one. That is, the ideal characteristic is usually a straight line; INL shows how much the actual voltage at a given code value differs from that line, in LSBs (1 LSB steps). • Gain • Offset • Noise is ultimately limited by the thermal noise generated by passive components such as resistors. For audio applications and in room temperatures, such noise is usually a little less than 1 μV (microvolt) of white noise. This limits performance to less than 20~21 bits even in 24-bit DACs. • Frequency domain performance • Spurious-free dynamic range (SFDR) indicates in dB the ratio between the powers of the converted main signal and the greatest undesired spur. • Signal-to-noise and distortion ratio (SNDR) indicates in dB the ratio between the powers of the converted main signal and the sum of the noise and the generated harmonic spurs • i-th harmonic distortion (HDi) indicates the power of the i-th harmonic of the converted main signal • Total harmonic distortion (THD) is the sum of the powers of all HDi • If the maximum DNL error is less than 1 LSB, then the D/A converter is guaranteed to be monotonic. However, many monotonic converters may have a maximum DNL greater than 1 LSB. • Time domain performance: • Glitch impulse area (glitch energy) • Response uncertainty • Time nonlinearity (TNL)

References [1] ADC and DAC Glossary - Maxim (http:/ / www. maxim-ic. com/ appnotes. cfm/ appnote_number/ 641/ )

Further reading • Kester, Walt, The Data Conversion Handbook (http://www.analog.com/library/analogDialogue/archives/ 39-06/data_conversion_handbook.html), ISBN 0-7506-7841-0 • S. Norsworthy, Richard Schreier, Gabor C. Temes, Delta-Sigma Data Converters. ISBN 0-7803-1045-4. • Mingliang Liu, Demystifying Switched-Capacitor Circuits. ISBN 0-7506-7907-7. • Behzad Razavi, Principles of Data Conversion System Design. ISBN 0-7803-1093-4. • Phillip E. Allen, Douglas R. Holberg, CMOS Analog Circuit Design. ISBN 0-19-511644-5. • Robert F. Coughlin, Frederick F. Driscoll, Operational Amplifiers and Linear Integrated Circuits. ISBN 0-13-014991-8. • A Anand Kumar, Fundamentals of Digital Circuits. ISBN 81-203-1745-9, ISBN 978-81-203-1745-1.

20

Digital-to-analog converter

21

External links • ADC and DAC Glossary (http://www.maxim-ic.com/appnotes.cfm/an_pk/641/CMP/WP-36)

Analog-to-digital converter An analog-to-digital converter (abbreviated ADC, A/D or A to D) is a device that converts a continuous physical quantity (usually voltage) to a digital number that represents the quantity's amplitude. The conversion involves quantization of the input, so it necessarily introduces a small amount of error. Instead of doing a single conversion, an ADC often performs the conversions ("samples" the input) periodically. The result is a sequence of digital values that have been converted from a continuous-time and continuous-amplitude analog signal to a discrete-time and discrete-amplitude digital signal.

4-channel stereo multiplexed analog-to-digital converter WM8775SEDS made by Wolfson Microelectronics placed on an X-Fi Fatal1ty Pro sound card.

An ADC is defined by its bandwidth (the range of frequencies it can measure) and its signal to noise ratio (how accurately it can measure a signal relative to the noise it introduces). The actual bandwidth of an ADC is characterized primarily by its sampling rate, and to a lesser extent by how it handles errors such as aliasing. The dynamic range of an ADC is influenced by many factors, including the resolution (the number of output levels it can quantize a signal to), linearity and accuracy (how well the quantization levels match the true analog signal) and jitter (small timing errors that introduce additional noise). The dynamic range of an ADC is often summarized in terms of its effective number of bits (ENOB), the number of bits of each measure it returns that are on average not noise. An ideal ADC has an ENOB equal to its resolution. ADCs are chosen to match the bandwidth and required signal to noise ratio of the signal to be quantized. If an ADC operates at a sampling rate greater than twice the bandwidth of the signal, then perfect reconstruction is possible given an ideal ADC and neglecting quantization error. The presence of quantization error limits the dynamic range of even an ideal ADC, however, if the dynamic range of the ADC exceeds that of the input signal, its effects may be neglected resulting in an essentially perfect digital representation of the input signal. An ADC may also provide an isolated measurement such as an electronic device that converts an input analog voltage or current to a digital number proportional to the magnitude of the voltage or current. However, some non-electronic or only partially electronic devices, such as rotary encoders, can also be considered ADCs. The digital output may use different coding schemes. Typically the digital output will be a two's complement binary number that is proportional to the input, but there are other possibilities. An encoder, for example, might output a Gray code. The inverse operation is performed by a digital-to-analog converter (DAC).

Analog-to-digital converter

22

Concepts Resolution The resolution of the converter indicates the number of discrete values it can produce over the range of analog values. The resolution determines the magnitude of the quantization error and therefore determines the maximum possible average signal to noise ratio for an ideal ADC without the use of oversampling. The values are usually stored electronically in binary form, so the resolution is usually expressed in bits. In consequence, the number of discrete values available, or "levels", is assumed to be a power of two. For example, an ADC with a resolution of 8 bits can encode an analog input to one in 256 different levels, since 28 = 256. The values can represent the ranges from 0 to 255 (i.e. unsigned integer) or from −128 to 127 (i.e. signed integer), depending on the application.

Fig. 1. An 8-level ADC coding scheme.

Resolution can also be defined electrically, and expressed in volts. The minimum change in voltage required to guarantee a change in the output code level is called the least significant bit (LSB) voltage. The resolution Q of the ADC is equal to the LSB voltage. The voltage resolution of an ADC is equal to its overall voltage measurement range divided by the number of discrete values:

where M is the ADC's resolution in bits and EFSR is the full scale voltage range (also called 'span'). EFSR is given by where VRefHi and VRefLow are the upper and lower extremes, respectively, of the voltages that can be coded. Normally, the number of voltage intervals is given by

where M is the ADC's resolution in bits. That is, one voltage interval is assigned in between two consecutive code levels. Example: • • • •

Coding scheme as in figure 1 (assume input signal x(t) = Acos(t), A = 5V) Full scale measurement range = -5 to 5 volts ADC resolution is 8 bits: 28 = 256 quantization levels (codes) ADC voltage resolution, Q = (10 V − 0 V) / 256 = 10 V / 256 ≈ 0.039 V ≈ 39 mV.

In practice, the useful resolution of a converter is limited by the best signal-to-noise ratio (SNR) that can be achieved for a digitized signal. An ADC can resolve a signal to only a certain number of bits of resolution, called the effective number of bits (ENOB). One effective bit of resolution changes the signal-to-noise ratio of the digitized signal by 6 dB, if the resolution is limited by the ADC. If a preamplifier has been used prior to A/D conversion, the noise introduced by the amplifier can be an important contributing factor towards the overall SNR.

Analog-to-digital converter

23

Quantization error Main article: Quantization error Quantization error is the noise introduced by quantization in an ideal ADC. It is a rounding error between the analog input voltage to the ADC and the output digitized value. The noise is non-linear and signal-dependent. In an ideal analog-to-digital converter, where the quantization error is uniformly distributed between −1/2 LSB and +1/2 LSB, and the signal has a uniform distribution covering all quantization levels, the Signal-to-quantization-noise ratio (SQNR) can be calculated from

Comparison of quantizing a sinusoid to 64 levels (6 bits) and 256 levels (8 bits). The additive noise created by 6-bit quantization is 12 dB greater than the noise created by 8-bit quantization. When the spectral distribution is flat, as in this example, the 12 dB difference manifests as a measurable difference in the noise floors.

Where Q is the number of quantization bits. For example, a 16-bit ADC has a maximum signal-to-noise ratio of 6.02 × 16 = 96.3 dB, and therefore the quantization error is 96.3 dB below the maximum level. Quantization error is distributed from DC to the Nyquist frequency, consequently if part of the ADC's bandwidth is not used (as in oversampling), some of the quantization error will fall out of band, effectively improving the SQNR. In an oversampled system, noise shaping can be used to further increase SQNR by forcing more quantization error out of the band. Dither Main article: dither In ADCs, performance can usually be improved using dither. This is a very small amount of random noise (white noise), which is added to the input before conversion. Its effect is to cause the state of the LSB to randomly oscillate between 0 and 1 in the presence of very low levels of input, rather than sticking at a fixed value. Rather than the signal simply getting cut off altogether at this low level (which is only being quantized to a resolution of 1 bit), it extends the effective range of signals that the ADC can convert, at the expense of a slight increase in noise – effectively the quantization error is diffused across a series of noise values which is far less objectionable than a hard cutoff. The result is an accurate representation of the signal over time. A suitable filter at the output of the system can thus recover this small signal variation. An audio signal of very low level (with respect to the bit depth of the ADC) sampled without dither sounds extremely distorted and unpleasant. Without dither the low level may cause the least significant bit to "stick" at 0 or 1. With dithering, the true level of the audio may be calculated by averaging the actual quantized sample with a series of other samples [the dither] that are recorded over time. A virtually identical process, also called dither or dithering, is often used when quantizing photographic images to a fewer number of bits per pixel—the image becomes noisier but to the eye looks far more realistic than the quantized image, which otherwise becomes banded. This analogous process may help to visualize the effect of dither on an analogue audio signal that is converted to digital. Dithering is also used in integrating systems such as electricity meters. Since the values are added together, the dithering produces results that are more exact than the LSB of the analog-to-digital converter.

Analog-to-digital converter

24

Note that dither can only increase the resolution of a sampler, it cannot improve the linearity, and thus accuracy does not necessarily improve.

Accuracy An ADC has several sources of errors. Quantization error and (assuming the ADC is intended to be linear) non-linearity are intrinsic to any analog-to-digital conversion. These errors are measured in a unit called the least significant bit (LSB). In the above example of an eight-bit ADC, an error of one LSB is 1/256 of the full signal range, or about 0.4%. Non-linearity All ADCs suffer from non-linearity errors caused by their physical imperfections, causing their output to deviate from a linear function (or some other function, in the case of a deliberately non-linear ADC) of their input. These errors can sometimes be mitigated by calibration, or prevented by testing. Important parameters for linearity are integral non-linearity (INL) and differential non-linearity (DNL). These non-linearities reduce the dynamic range of the signals that can be digitized by the ADC, also reducing the effective resolution of the ADC.

Jitter When digitizing a sine wave

, the use of a non-ideal sampling clock will result in some

uncertainty in when samples are recorded. Provided that the actual sampling time uncertainty due to the clock jitter is

, the error caused by this phenomenon can be estimated as

. This will

result in additional recorded noise that will reduce the effective number of bits (ENOB) below that predicted by quantization error alone. The error is zero for DC, small at low frequencies, but significant when high frequencies have high amplitudes. This effect can be ignored if it is drowned out by the quantizing error. Jitter requirements can be calculated using the following formula:

, where q is the number of ADC bits. Output size (bits)

Signal Frequency 1 Hz

1 kHz 10 kHz 1 MHz 10 MHz 100 MHz 1 GHz

8 1,243 µs 1.24 µs

124 ns 1.24 ns

311 ns 31.1 ns

124 ps

12.4 ps 1.24 ps

10

311 µs

311 ps

31.1 ps

3.11 ps 0.31 ps

12

77.7 µs 77.7 ns 7.77 ns 77.7 ps

7.77 ps

0.78 ps 0.08 ps

14

19.4 µs 19.4 ns 1.94 ns 19.4 ps

1.94 ps

0.19 ps 0.02 ps

16

4.86 µs 4.86 ns

486 ps 4.86 ps

0.49 ps

0.05 ps

18

1.21 ns

121 ps 6.32 ps 1.21 ps

0.12 ps

–

–

20

304 ps 30.4 ps 1.58 ps 0.16 ps

–

–

–

–

Clock jitter is caused by phase noise.[1] The resolution of ADCs with a digitization bandwidth between 1 MHz and 1 GHz is limited by jitter. When sampling audio signals at 44.1 kHz, the anti-aliasing filter should have eliminated all frequencies above 22 kHz. The input frequency (in this case, < 22 kHz kHz), not the ADC clock frequency, is the determining factor with respect to jitter performance.[2]

Analog-to-digital converter

Sampling rate Main article: Sampling rate See also: Sampling (signal processing) The analog signal is continuous in time and it is necessary to convert this to a flow of digital values. It is therefore required to define the rate at which new digital values are sampled from the analog signal. The rate of new values is called the sampling rate or sampling frequency of the converter. A continuously varying bandlimited signal can be sampled (that is, the signal values at intervals of time T, the sampling time, are measured and stored) and then the original signal can be exactly reproduced from the discrete-time values by an interpolation formula. The accuracy is limited by quantization error. However, this faithful reproduction is only possible if the sampling rate is higher than twice the highest frequency of the signal. This is essentially what is embodied in the Shannon-Nyquist sampling theorem. Since a practical ADC cannot make an instantaneous conversion, the input value must necessarily be held constant during the time that the converter performs a conversion (called the conversion time). An input circuit called a sample and hold performs this task—in most cases by using a capacitor to store the analog voltage at the input, and using an electronic switch or gate to disconnect the capacitor from the input. Many ADC integrated circuits include the sample and hold subsystem internally. Aliasing Main article: Aliasing See also: Undersampling An ADC works by sampling the value of the input at discrete intervals in time. Provided that the input is sampled above the Nyquist rate, defined as twice the highest frequency of interest, then all frequencies in the signal can be reconstructed. If frequencies above half the Nyquist rate are sampled, they are incorrectly detected as lower frequencies, a process referred to as aliasing. Aliasing occurs because instantaneously sampling a function at two or fewer times per cycle results in missed cycles, and therefore the appearance of an incorrectly lower frequency. For example, a 2 kHz sine wave being sampled at 1.5 kHz would be reconstructed as a 500 Hz sine wave. To avoid aliasing, the input to an ADC must be low-pass filtered to remove frequencies above half the sampling rate. This filter is called an anti-aliasing filter, and is essential for a practical ADC system that is applied to analog signals with higher frequency content. In applications where protection against aliasing is essential, oversampling may be used to greatly reduce or even eliminate it. Although aliasing in most systems is unwanted, it should also be noted that it can be exploited to provide simultaneous down-mixing of a band-limited high frequency signal (see undersampling and frequency mixer). The alias is effectively the lower heterodyne of the signal frequency and sampling frequency. Oversampling Main article: Oversampling Signals are often sampled at the minimum rate required, for economy, with the result that the quantization noise introduced is white noise spread over the whole pass band of the converter. If a signal is sampled at a rate much higher than the Nyquist frequency and then digitally filtered to limit it to the signal bandwidth there are the following advantages: • digital filters can have better properties (sharper rolloff, phase) than analogue filters, so a sharper anti-aliasing filter can be realised and then the signal can be downsampled giving a better result • a 20-bit ADC can be made to act as a 24-bit ADC with 256× oversampling • the signal-to-noise ratio due to quantization noise will be higher than if the whole available band had been used. With this technique, it is possible to obtain an effective resolution larger than that provided by the converter alone

25

Analog-to-digital converter • The improvement in SNR is 3 dB (equivalent to 0.5 bits) per octave of oversampling which is not sufficient for many applications. Therefore, oversampling is usually coupled with noise shaping (see sigma-delta modulators). With noise shaping, the improvement is 6L+3 dB per octave where L is the order of loop filter used for noise shaping. e.g. – a 2nd order loop filter will provide an improvement of 15 dB/octave. Oversampling is typically used in audio frequency ADCs where the required sampling rate (typically 44.1 or 48 kHz) is very low compared to the clock speed of typical transistor circuits (>1 MHz). In this case, by using the extra bandwidth to distribute quantization error onto out of band frequencies, the accuracy of the ADC can be greatly increased at no cost. Furthermore, as any aliased signals are also typically out of band, aliasing can often be completely eliminated using very low cost filters.

Relative speed and precision The speed of an ADC varies by type. The Wilkinson ADC is limited by the clock rate which is processable by current digital circuits. Currently,Wikipedia:Manual of Style/Dates and numbers#Chronological items frequencies up to 300 MHz are possible.[3] For a successive-approximation ADC, the conversion time scales with the logarithm of the resolution, e.g. the number of bits. Thus for high resolution, it is possible that the successive-approximation ADC is faster than the Wilkinson. However, the time consuming steps in the Wilkinson are digital, while those in the successive-approximation are analog. Since analog is inherently slower than digital, as the resolution increases, the time required also increases. Thus there are competing processes at work. Flash ADCs are certainly the fastest type of the three. The conversion is basically performed in a single parallel step. For an 8-bit unit, conversion takes place in a few tens of nanoseconds. There is, as expected, somewhat of a tradeoff between speed and precision. Flash ADCs have drifts and uncertainties associated with the comparator levels. This results in poor linearity. For successive-approximation ADCs, poor linearity is also present, but less so than for flash ADCs. Here, non-linearity arises from accumulating errors from the subtraction processes. Wilkinson ADCs have the highest linearity of the three. These have the best differential non-linearity. The other types require channel smoothing to achieve the level of the Wilkinson.

The sliding scale principle The sliding scale or randomizing method can be employed to greatly improve the linearity of any type of ADC, but especially flash and successive approximation types. For any ADC the mapping from input voltage to digital output value is not exactly a floor or ceiling function as it should be. Under normal conditions, a pulse of a particular amplitude is always converted to a digital value. The problem lies in that the ranges of analog values for the digitized values are not all of the same width, and the differential linearity decreases proportionally with the divergence from the average width. The sliding scale principle uses an averaging effect to overcome this phenomenon. A random, but known analog voltage is added to the sampled input voltage. It is then converted to digital form, and the equivalent digital amount is subtracted, thus restoring it to its original value. The advantage is that the conversion has taken place at a random point. The statistical distribution of the final levels is decided by a weighted average over a region of the range of the ADC. This in turn desensitizes it to the width of any specific level.

ADC types These are the most common ways of implementing an electronic ADC: • A direct-conversion ADC or flash ADC has a bank of comparators sampling the input signal in parallel, each firing for their decoded voltage range. The comparator bank feeds a logic circuit that generates a code for each voltage range. Direct conversion is very fast, capable of gigahertz sampling rates, but usually has only 8 bits of resolution or fewer, since the number of comparators needed, 2N – 1, doubles with each additional bit, requiring a large, expensive circuit. ADCs of this type have a large die size, a high input capacitance, high power dissipation, and are prone to produce glitches at the output (by outputting an out-of-sequence code). Scaling to newer

26

Analog-to-digital converter submicrometre technologies does not help as the device mismatch is the dominant design limitation. They are often used for video, wideband communications or other fast signals in optical storage. • A successive-approximation ADC uses a comparator to successively narrow a range that contains the input voltage. At each successive step, the converter compares the input voltage to the output of an internal digital to analog converter which might represent the midpoint of a selected voltage range. At each step in this process, the approximation is stored in a successive approximation register (SAR). For example, consider an input voltage of 6.3 V and the initial range is 0 to 16 V. For the first step, the input 6.3 V is compared to 8 V (the midpoint of the 0–16 V range). The comparator reports that the input voltage is less than 8 V, so the SAR is updated to narrow the range to 0–8 V. For the second step, the input voltage is compared to 4 V (midpoint of 0–8). The comparator reports the input voltage is above 4 V, so the SAR is updated to reflect the input voltage is in the range 4–8 V. For the third step, the input voltage is compared with 6 V (halfway between 4 V and 8 V); the comparator reports the input voltage is greater than 6 volts, and search range becomes 6–8 V. The steps are continued until the desired resolution is reached. • A ramp-compare ADC produces a saw-tooth signal that ramps up or down then quickly returns to zero. When the ramp starts, a timer starts counting. When the ramp voltage matches the input, a comparator fires, and the timer's value is recorded. Timed ramp converters require the least number of transistors. The ramp time is sensitive to temperature because the circuit generating the ramp is often a simple oscillator. There are two solutions: use a clocked counter driving a DAC and then use the comparator to preserve the counter's value, or calibrate the timed ramp. A special advantage of the ramp-compare system is that comparing a second signal just requires another comparator, and another register to store the voltage value. A very simple (non-linear) ramp-converter can be implemented with a microcontroller and one resistor and capacitor.[4] Vice versa, a filled capacitor can be taken from an integrator, time-to-amplitude converter, phase detector, sample and hold circuit, or peak and hold circuit and discharged. This has the advantage that a slow comparator cannot be disturbed by fast input changes. • The Wilkinson ADC was designed by D. H. Wilkinson in 1950. The Wilkinson ADC is based on the comparison of an input voltage with that produced by a charging capacitor. The capacitor is allowed to charge until its voltage is equal to the amplitude of the input pulse (a comparator determines when this condition has been reached). Then, the capacitor is allowed to discharge linearly, which produces a ramp voltage. At the point when the capacitor begins to discharge, a gate pulse is initiated. The gate pulse remains on until the capacitor is completely discharged. Thus the duration of the gate pulse is directly proportional to the amplitude of the input pulse. This gate pulse operates a linear gate which receives pulses from a high-frequency oscillator clock. While the gate is open, a discrete number of clock pulses pass through the linear gate and are counted by the address register. The time the linear gate is open is proportional to the amplitude of the input pulse, thus the number of clock pulses recorded in the address register is proportional also. Alternatively, the charging of the capacitor could be monitored, rather than the discharge. • An integrating ADC (also dual-slope or multi-slope ADC) applies the unknown input voltage to the input of an integrator and allows the voltage to ramp for a fixed time period (the run-up period). Then a known reference voltage of opposite polarity is applied to the integrator and is allowed to ramp until the integrator output returns to zero (the run-down period). The input voltage is computed as a function of the reference voltage, the constant run-up time period, and the measured run-down time period. The run-down time measurement is usually made in units of the converter's clock, so longer integration times allow for higher resolutions. Likewise, the speed of the converter can be improved by sacrificing resolution. Converters of this type (or variations on the concept) are used in most digital voltmeters for their linearity and flexibility. • A delta-encoded ADC or counter-ramp has an up-down counter that feeds a digital to analog converter (DAC). The input signal and the DAC both go to a comparator. The comparator controls the counter. The circuit uses negative feedback from the comparator to adjust the counter until the DAC's output is close enough to the input signal. The number is read from the counter. Delta converters have very wide ranges and high resolution, but the

27

Analog-to-digital converter conversion time is dependent on the input signal level, though it will always have a guaranteed worst-case. Delta converters are often very good choices to read real-world signals. Most signals from physical systems do not change abruptly. Some converters combine the delta and successive approximation approaches; this works especially well when high frequencies are known to be small in magnitude. • A pipeline ADC (also called subranging quantizer) uses two or more steps of subranging. First, a coarse conversion is done. In a second step, the difference to the input signal is determined with a digital to analog converter (DAC). This difference is then converted finer, and the results are combined in a last step. This can be considered a refinement of the successive-approximation ADC wherein the feedback reference signal consists of the interim conversion of a whole range of bits (for example, four bits) rather than just the next-most-significant bit. By combining the merits of the successive approximation and flash ADCs this type is fast, has a high resolution, and only requires a small die size. • A sigma-delta ADC (also known as a delta-sigma ADC) oversamples the desired signal by a large factor and filters the desired signal band. Generally, a smaller number of bits than required are converted using a Flash ADC after the filter. The resulting signal, along with the error generated by the discrete levels of the Flash, is fed back and subtracted from the input to the filter. This negative feedback has the effect of noise shaping the error due to the Flash so that it does not appear in the desired signal frequencies. A digital filter (decimation filter) follows the ADC which reduces the sampling rate, filters off unwanted noise signal and increases the resolution of the output (sigma-delta modulation, also called delta-sigma modulation). • A time-interleaved ADC uses M parallel ADCs where each ADC samples data every M:th cycle of the effective sample clock. The result is that the sample rate is increased M times compared to what each individual ADC can manage. In practice, the individual differences between the M ADCs degrade the overall performance reducing the SFDR. However, technologies exist to correct for these time-interleaving mismatch errors. • An ADC with intermediate FM stage first uses a voltage-to-frequency converter to convert the desired signal into an oscillating signal with a frequency proportional to the voltage of the desired signal, and then uses a frequency counter to convert that frequency into a digital count proportional to the desired signal voltage. Longer integration times allow for higher resolutions. Likewise, the speed of the converter can be improved by sacrificing resolution. The two parts of the ADC may be widely separated, with the frequency signal passed through an opto-isolator or transmitted wirelessly. Some such ADCs use sine wave or square wave frequency modulation; others use pulse-frequency modulation. Such ADCs were once the most popular way to show a digital display of the status of a remote analog sensor.[5][6][7][8][9] There can be other ADCs that use a combination of electronics and other technologies: • A time-stretch analog-to-digital converter (TS-ADC) digitizes a very wide bandwidth analog signal, that cannot be digitized by a conventional electronic ADC, by time-stretching the signal prior to digitization. It commonly uses a photonic preprocessor frontend to time-stretch the signal, which effectively slows the signal down in time and compresses its bandwidth. As a result, an electronic backend ADC, that would have been too slow to capture the original signal, can now capture this slowed down signal. For continuous capture of the signal, the frontend also divides the signal into multiple segments in addition to time-stretching. Each segment is individually digitized by a separate electronic ADC. Finally, a digital signal processor rearranges the samples and removes any distortions added by the frontend to yield the binary data that is the digital representation of the original analog signal.

28

Analog-to-digital converter

Commercial analog-to-digital converters Commercial ADCs are usually implemented as integrated circuits. Most converters sample with 6 to 24 bits of resolution, and produce fewer than 1 megasample per second. Thermal noise generated by passive components such as resistors masks the measurement when higher resolution is desired. For audio applications and in room temperatures, such noise is usually a little less than 1 μV (microvolt) of white noise. If the MSB corresponds to a standard 2 V of output signal, this translates to a noise-limited performance that is less than 20~21 bits, and obviates the need for any dithering. As of February 2002, Mega- and giga-sample per second converters are available. Mega-sample converters are required in digital video cameras, video capture cards, and TV tuner cards to convert full-speed analog video to digital video files. Commercial converters usually have ±0.5 to ±1.5 LSB error in their output. In many cases, the most expensive part of an integrated circuit is the pins, because they make the package larger, and each pin has to be connected to the integrated circuit's silicon. To save pins, it is common for slow ADCs to send their data one bit at a time over a serial interface to the computer, with the next bit coming out when a clock signal changes state, say from 0 to 5 V. This saves quite a few pins on the ADC package, and in many cases, does not make the overall design any more complex (even microprocessors which use memory-mapped I/O only need a few bits of a port to implement a serial bus to an ADC). Commercial ADCs often have several inputs that feed the same converter, usually through an analog multiplexer. Different models of ADC may include sample and hold circuits, instrumentation amplifiers or differential inputs, where the quantity measured is the difference between two voltages.

Applications Music recording Analog-to-digital converters are integral to current music reproduction technology. People produce much music on computers using an analog recording and therefore need analog-to-digital converters to create the pulse-code modulation (PCM) data streams that go onto compact discs and digital music files. The current crop of analog-to-digital converters utilized in music can sample at rates up to 192 kilohertz. Considerable literature exists on these matters, but commercial considerations often play a significant role. MostWikipedia:Citation needed high-profile recording studios record in 24-bit/192-176.4 kHz pulse-code modulation (PCM) or in Direct Stream Digital (DSD) formats, and then downsample or decimate the signal for Red-Book CD production (44.1 kHz) or to 48 kHz for commonly used radio and television broadcast applications.

Digital signal processing People must use ADCs to process, store, or transport virtually any analog signal in digital form. TV tuner cards, for example, use fast video analog-to-digital converters. Slow on-chip 8, 10, 12, or 16 bit analog-to-digital converters are common in microcontrollers. Digital storage oscilloscopes need very fast analog-to-digital converters, also crucial for software defined radio and their new applications.

Scientific instruments Digital imaging systems commonly use analog-to-digital converters in digitizing pixels. Some radar systems commonly use analog-to-digital converters to convert signal strength to digital values for subsequent signal processing. Many other in situ and remote sensing systems commonly use analogous technology. The number of binary bits in the resulting digitized numeric values reflects the resolution, the number of unique discrete levels of quantization (signal processing). The correspondence between the analog signal and the digital

29

Analog-to-digital converter signal depends on the quantization error. The quantization process must occur at an adequate speed, a constraint that may limit the resolution of the digital signal. Many sensors produce an analog signal; temperature, pressure, pH, light intensity etc. All these signals can be amplified and fed to an ADC to produce a digital number proportional to the input signal.

Electrical Symbol

Testing Testing an Analog to Digital Converter requires an analog input source, hardware to send control signals and capture digital data output. Some ADCs also require an accurate source of reference signal. The key parameters to test a SAR ADC are following: 1. 2. 3. 4. 5. 6. 7. 8.

DC Offset Error DC Gain Error Signal to Noise Ratio (SNR) Total Harmonic Distortion (THD) Integral Non Linearity (INL) Differential Non Linearity (DNL) Spurious Free Dynamic Range Power Dissipation

Notes [1] Maxim App 800: "Design a Low-Jitter Clock for High-Speed Data Converters" (http:/ / www. maxim-ic. com/ appnotes. cfm/ an_pk/ 800/ ). maxim-ic.com (July 17, 2002). [2] Redmayne, Derek and Steer, Alison (8 December 2008) Understanding the effect of clock jitter on high-speed ADCs (http:/ / www. eetimes. com/ design/ automotive-design/ 4010074/ Understanding-the-effect-of-clock-jitter-on-high-speed-ADCs-Part-1-of-2-). eetimes.com [3] 310 Msps ADC by Linear Technology, http:/ / www. linear. com/ product/ LTC2158-14. [4] Atmel Application Note AVR400: Low Cost A/D Converter (http:/ / www. atmel. com/ dyn/ resources/ prod_documents/ doc0942. pdf). atmel.com [5] Analog Devices MT-028 Tutorial: "Voltage-to-Frequency Converters" (http:/ / www. analog. com/ static/ imported-files/ tutorials/ MT-028. pdf) by Walt Kester and James Bryant 2009, apparently adapted from Kester, Walter Allan (2005) Data conversion handbook (http:/ / books. google. com/ books?id=0aeBS6SgtR4C& pg=RA2-PA274), Newnes, p. 274, ISBN 0750678410. [6] Microchip AN795 "Voltage to Frequency / Frequency to Voltage Converter" (http:/ / ww1. microchip. com/ downloads/ en/ AppNotes/ 00795a. pdf) p. 4: "13-bit A/D converter" [7] Carr, Joseph J. (1996) Elements of electronic instrumentation and measurement (http:/ / books. google. com/ books?id=1yBTAAAAMAAJ), Prentice Hall, p. 402, ISBN 0133416860. [8] "Voltage-to-Frequency Analog-to-Digital Converters" (http:/ / www. globalspec. com/ reference/ 3127/ Voltage-to-Frequency-Analog-to-Digital-Converters). globalspec.com [9] Pease, Robert A. (1991) Troubleshooting Analog Circuits (http:/ / books. google. com/ books?id=3kY4-HYLqh0C& pg=PA130), Newnes, p. 130, ISBN 0750694998.

30

Analog-to-digital converter

References • Knoll, Glenn F. (1989). Radiation Detection and Measurement (2nd ed.). New York: John Wiley & Sons. ISBN 0471815047. • Nicholson, P. W. (1974). Nuclear Electronics. New York: John Wiley & Sons. pp. 315–316. ISBN 0471636975.

Further reading • Allen, Phillip E.; Holberg, Douglas R. CMOS Analog Circuit Design. ISBN 0-19-511644-5. • Fraden, Jacob (2010). Handbook of Modern Sensors: Physics, Designs, and Applications. Springer. ISBN 978-1441964656. • Kester, Walt, ed. (2005). The Data Conversion Handbook (http://www.analog.com/library/analogDialogue/ archives/39-06/data_conversion_handbook.html). Elsevier: Newnes. ISBN 0-7506-7841-0. • Johns, David; Martin, Ken. Analog Integrated Circuit Design. ISBN 0-471-14448-7. • Liu, Mingliang. Demystifying Switched-Capacitor Circuits. ISBN 0-7506-7907-7. • Norsworthy, Steven R.; Schreier, Richard; Temes, Gabor C. (1997). Delta-Sigma Data Converters. IEEE Press. ISBN 0-7803-1045-4. • Razavi, Behzad (1995). Principles of Data Conversion System Design. New York, NY: IEEE Press. ISBN 0-7803-1093-4. • Staller, Len (February 24, 2005). "Understanding analog to digital converter specifications" (http://www. embedded.com/design/configurable-systems/4025078/ Understanding-analog-to-digital-converter-specifications). Embedded Systems Design. • Walden, R. H. (1999). "Analog-to-digital converter survey and analysis". IEEE Journal on Selected Areas in Communications 17 (4): 539–550. doi: 10.1109/49.761034 (http://dx.doi.org/10.1109/49.761034).

External links • Counting Type ADC (http://ikalogic.com/tut_adc.php) A simple tutorial showing how to build your first ADC. • An Introduction to Delta Sigma Converters (http://www.beis.de/Elektronik/DeltaSigma/DeltaSigma.html) A very nice overview of Delta-Sigma converter theory. • Digital Dynamic Analysis of A/D Conversion Systems through Evaluation Software based on FFT/DFT Analysis (http://www.ieee.li/pdf/adc_evaluation_rf_expo_east_1987.pdf) RF Expo East, 1987 • Which ADC Architecture Is Right for Your Application? (http://www.analog.com/library/analogDialogue/ archives/39-06/architecture.html) article by Walt Kester • ADC and DAC Glossary (http://www.maxim-ic.com/appnotes.cfm/an_pk/641/CMP/ELK-11) Defines commonly used technical terms. • Introduction to ADC in AVR (http://www.robotplatform.com/knowledge/ADC/adc_tutorial.html) – Analog to digital conversion with Atmel microcontrollers • Signal processing and system aspects of time-interleaved ADCs. (http://userver.ftw.at/~vogel/TIADC.html) • Explanation of analog-digital converters with interactive principles of operations. (http://www.onmyphd.com/ ?p=analog.digital.converter)

31

Window function

32

Window function For the term used in SQL statements, see Window function (SQL) In signal processing, a window function (also known as an apodization function or tapering function) is a mathematical function that is zero-valued outside of some chosen interval. For instance, a function that is constant inside the interval and zero elsewhere is called a rectangular window, which describes the shape of its graphical representation. When another function or waveform/data-sequence is multiplied by a window function, the product is also zero-valued outside the interval: all that is left is the part where they overlap, the "view through the window". Applications of window functions include spectral analysis, filter design, and beamforming. In typical applications, the window functions used are non-negative smooth "bell-shaped" curves, though rectangle, triangle, and other functions can be used. A more general definition of window functions does not require them to be identically zero outside an interval, as long as the product of the window multiplied by its argument is square integrable, and, more specifically, that the function goes sufficiently rapidly toward zero.

Applications Applications of window functions include spectral analysis and the design of finite impulse response filters.

Spectral analysis The Fourier transform of the function cos ωt is zero, except at frequency ±ω. However, many other functions and waveforms do not have convenient closed form transforms. Alternatively, one might be interested in their spectral content only during a certain time period. In either case, the Fourier transform (or something similar) can be applied on one or more finite intervals of the waveform. In general, the transform is applied to the product of the waveform and a window function. Any window (including rectangular) affects the spectral estimate computed by this method. Windowing Windowing of a simple waveform like cos ωt causes its Fourier transform to develop non-zero values (commonly called spectral leakage) at frequencies other than ω. The leakage tends to be worst (highest) near ω and least at frequencies farthest from ω. If the waveform under analysis comprises two sinusoids of different frequencies, leakage can interfere with the ability to distinguish them spectrally. If their frequencies are dissimilar and one component is weaker, then leakage from the larger component can obscure the weaker

Figure 1: Zoomed view of spectral leakage

Window function one’s presence. But if the frequencies are similar, leakage can render them unresolvable even when the sinusoids are of equal strength. The rectangular window has excellent resolution characteristics for sinusoids of comparable strength, but it is a poor choice for sinusoids of disparate amplitudes. This characteristic is sometimes described as low-dynamic-range. At the other extreme of dynamic range are the windows with the poorest resolution. These high-dynamic-range low-resolution windows are also poorest in terms of sensitivity; this is, if the input waveform contains random noise close to the frequency of a sinusoid, the response to noise, compared to the sinusoid, will be higher than with a higher-resolution window. In other words, the ability to find weak sinusoids amidst the noise is diminished by a high-dynamic-range window. High-dynamic-range windows are probably most often justified in wideband applications, where the spectrum being analyzed is expected to contain many different components of various amplitudes. In between the extremes are moderate windows, such as Hamming and Hann. They are commonly used in narrowband applications, such as the spectrum of a telephone channel. In summary, spectral analysis involves a tradeoff between resolving comparable strength components with similar frequencies and resolving disparate strength components with dissimilar frequencies. That tradeoff occurs when the window function is chosen. Discrete-time signals When the input waveform is time-sampled, instead of continuous, the analysis is usually done by applying a window function and then a discrete Fourier transform (DFT). But the DFT provides only a coarse sampling of the actual DTFT spectrum. Figure 1 shows a portion of the DTFT for a rectangularly windowed sinusoid. The actual frequency of the sinusoid is indicated as "0" on the horizontal axis. Everything else is leakage, exaggerated by the use of a logarithmic presentation. The unit of frequency is "DFT bins"; that is, the integer values on the frequency axis correspond to the frequencies sampled by the DFT. So the figure depicts a case where the actual frequency of the sinusoid happens to coincide with a DFT sample,[1] and the maximum value of the spectrum is accurately measured by that sample. When it misses the maximum value by some amount (up to 1/2 bin), the measurement error is referred to as scalloping loss (inspired by the shape of the peak). But the most interesting thing about this case is that all the other samples coincide with nulls in the true spectrum. (The nulls are actually zero-crossings, which cannot be shown on a logarithmic scale such as this.) So in this case, the DFT creates the illusion of no leakage. Despite the unlikely conditions of this example, it is a common misconception that visible leakage is some sort of artifact of the DFT. But since any window function causes leakage, its apparent absence (in this contrived example) is actually the DFT artifact.

33

Window function

Noise bandwidth The concepts of resolution and dynamic range tend to be somewhat subjective, depending on what the user is actually trying to do. But they also tend to be highly correlated with the total leakage, which is quantifiable. It is usually expressed as an equivalent bandwidth, B. It can be thought of as redistributing the DTFT into a rectangular shape with height equal to the spectral maximum and width B.[2] The more leakage, the greater the bandwidth. It is sometimes called noise equivalent bandwidth or equivalent noise bandwidth, because it is proportional to the average power that will be registered by each DFT bin when the input signal contains a random noise component (or is just random noise). A graph of the power This figure compares the processing losses of three window functions for sinusoidal spectrum, averaged over time, inputs, with both minimum and maximum scalloping loss. typically reveals a flat noise floor, caused by this effect. The height of the noise floor is proportional to B. So two different window functions can produce different noise floors.

Processing gain and losses In signal processing, operations are chosen to improve some aspect of quality of a signal by exploiting the differences between the signal and the corrupting influences. When the signal is a sinusoid corrupted by additive random noise, spectral analysis distributes the signal and noise components differently, often making it easier to detect the signal's presence or measure certain characteristics, such as amplitude and frequency. Effectively, the signal to noise ratio (SNR) is improved by distributing the noise uniformly, while concentrating most of the sinusoid's energy around one frequency. Processing gain is a term often used to describe an SNR improvement. The processing gain of spectral analysis depends on the window function, both its noise bandwidth (B) and its potential scalloping loss. These effects partially offset, because windows with the least scalloping naturally have the most leakage. The figure at right depicts the effects of three different window functions on the same data set, comprising two equal strength sinusoids in additive noise. The frequencies of the sinusoids are chosen such that one encounters no scalloping and the other encounters maximum scalloping. Both sinusoids suffer less SNR loss under the Hann window than under the Blackman–Harris window. In general (as mentioned earlier), this is a deterrent to using high-dynamic-range windows in low-dynamic-range applications.

34

Window function

Filter design Main article: Filter design Windows are sometimes used in the design of digital filters, in particular to convert an "ideal" impulse response of infinite duration, such as a sinc function, to a finite impulse response (FIR) filter design. That is called the window method.[3][4]

Symmetry and asymmetry Window functions generated for digital Sampled window functions are generated differently for filter design and spectral analysis filter design are symmetrical applications. And the asymmetrical ones often used in spectral analysis are also generated in a couple of different ways. Using the triangular function, for example, 3 different sequences, usually an odd length with outcomes for an 8-point window sequence are illustrated. a single maximum at the center. Windows for DFT/FFT usage, such as in spectral analysis, are often created by deleting the right-most coefficient of an odd-length, symmetrical window. Such truncated sequences are known as periodic.[5] The deleted coefficient is effectively restored (by a virtual copy of the symmetrical left-most coefficient) when the truncated sequence is periodically extended (which is the time-domain equivalent of sampling the DTFT). A different way of saying the same thing is that the DFT "samples" the DTFT of the Three different ways to create an 8-point Hann window sequence. window at the exact points that are not affected by spectral leakage from the discontinuity. The advantage of this trick is that a 512 length window (for example) enjoys the slightly better performance metrics of a 513 length design. Such a window is generated by the Matlab function hann(512,'periodic'), for instance. To generate it with the formula in this article (below), the window length (N) is 513, and the 513th coefficient of the generated sequence is discarded. Another type of asymmetric window, called DFT-even, is limited to even length sequences. The generated sequence is offset (cyclically) from its zero-phase [6] counterpart by exactly half the sequence length. In the frequency domain, that corresponds to a multiplication by the trivial sequence (-1)k, which can have implementation advantages for windows defined by their frequency domain form. Compared to a symmetrical window, the DFT-even sequence has an offset of ½ sample. As illustrated in the figure at right, that means the asymmetry is limited to just one missing coefficient. Therefore, as in the periodic case, it is effectively restored (by a virtual copy of the symmetrical left-most coefficient) when the truncated sequence is periodically extended.

35

Window function

36

Applications for which windows should not be used In some applications, it is preferable not to use a window function. For example: • In impact modal testing, when analyzing transient signals such as an excitation signal from hammer blow (see Impulse excitation technique), where most of the energy is located at the beginning of the recording. Using a non-rectangular window would attenuate most of the energy and spread the frequency response unnecessarily.[7] • A generalization of above, when measuring a self-windowing signal, such as an impulse, a shock response, a sine burst, a chirp burst, noise burst. Such signals are used in modal analysis. Applying a window function in this case would just deteriorate the signal-to-noise ratio. • When measuring a pseudo-random noise (PRN) excitation signal with period T, and using the same recording period T. A PRN signal is periodic and therefore all spectral components of the signal will coincide with FFT bin centers with no leakage.[8] • When measuring a repetitive signal locked-in to the sampling frequency, for example measuring the vibration spectrum analysis during Shaft alignment, fault diagnosis of bearings, engines, gearboxes etc. Since the signal is repetitive, all spectral energy is confined to multiples of the base repetition frequency. • In an OFDM receiver, the input signal is directly multiplied by FFT without a window function. The frequency sub-carriers (aka symbols) are designed to align exactly to the FFT frequency bins. A cyclic prefix is usually added to the transmitted signal, allowing frequency-selective fading due to multipath to be modeled as circular convolution, thus avoiding intersymbol interference, which in OFDM is equivalent to spectral leakage.

A list of window functions Terminology: • N represents the width, in samples, of a discrete-time, symmetrical window function When N is an odd number, the non-flat windows have a singular maximum point. When N is even, they have a double maximum. • It is sometimes useful to express

as the lagged version of a sequence of samples of a zero-phase [6]

function:

• For instance, for even values of N we can describe the related DFT-even window as discussed in the previous section. The DFT of such a sequence, in terms of the DFT of the

as sequence, is

• Each figure label includes the corresponding noise equivalent bandwidth metric (B), in units of DFT bins.

B-spline windows B-spline windows can be obtained as k-fold convolutions of the rectangular window. They include the rectangular window itself (k = 1), the triangular window (k = 2) and the Parzen window (k = 4). Alternative definitions sample the appropriate normalized B-spline basis functions instead of convolving discrete-time windows. A kth order B-spline basis function is a piece-wise polynomial function of degree k−1 that is obtained by k-fold self-convolution of the rectangular function.

Window function

37

Rectangular window The rectangular window (sometimes known as the boxcar or Dirichlet window) is the simplest window, equivalent to replacing all but N values of a data sequence by zeros, making it appear as though the waveform suddenly turns on and off:

Other windows are designed to moderate these sudden changes Rectangular window; B = 1.0000. because discontinuities have undesirable effects on the discrete-time Fourier transform (DTFT) and/or the algorithms that produce samples of the DTFT.[9][10] The rectangular window is the 1st order B-spline window as well as the 0th power cosine window. Triangular window Triangular windows are given by:

where L can be N,[11] N+1, or N-1.[12] The last one is also known as Bartlett window. All three definitions converge at large N. The triangular window is the 2nd order Triangular window or equivalently the Bartlett window; B = 1.3333. B-spline window and can be seen as the convolution of two half-sized rectangular windows, giving it twice the width of the regular windows. Parzen window Not to be confused with Kernel density estimation. The Parzen window, also known as the de la Vallée Poussin window, is the 4th order B-spline window.

Other polynomial windows

Parzen window; B = 1.92.

Window function

38

Welch window The Welch window consists of a single parabolic section: . The defining quadratic polynomial reaches a value of zero at the samples just outside the span of the window.

Generalized Hamming windows

Welch window; B = 1.20.

Generalized Hamming windows are of the form: . They have only three non-zero DFT coefficients and share the benefits of a sparse frequency domain representation with higher-order generalized cosine windows. Hann (Hanning) window Main article: Hann function The Hann window named after Julius von Hann and also known as the Hanning (for being similar in name and form to the Hamming window), von Hann and the raised cosine window is defined by:[13][14]

• zero-phase version: Hann window; B = 1.5000.

The ends of the cosine just touch zero, so the side-lobes roll off at about 18 dB per octave.[15]

Window function

39

Hamming window The window with these particular coefficients was proposed by Richard W. Hamming. The window is optimized to minimize the maximum (nearest) side lobe, giving it a height of about one-fifth that of the Hann window.[16]

with Hamming window, α = 0.53836 and β = 0.46164; B = 1.37. The original Hamming window would have α = 0.54 and β = 0.46; B = 1.3628.

instead of both constants being equal to 1/2 in the Hann window. The constants are approximations of values α = 25/46 and β = 21/46, which cancel the first sidelobe of the Hann window by placing a zero at frequency 5π/(N − 1). Approximation of the constants to two decimal places substantially lowers the level of sidelobes, to a nearly equiripple condition. In the equiripple sense, the optimal values for the coefficients are α = 0.53836 and β = 0.46164. • zero-phase version:

Higher-order generalized cosine windows Windows of the form:

have only 2K + 1 non-zero DFT coefficients, which makes them good choices for applications that require windowing by convolution in the frequency-domain. In those applications, the DFT of the unwindowed data vector is needed for a different purpose than spectral analysis. (see Overlap-save method). Generalized cosine windows with just two terms (K = 1) belong in the subfamily generalized Hamming windows.

Window function

40

Blackman windows Blackman windows are defined as:

By common convention, the unqualified term Blackman window refers to α = 0.16, as this most closely approximates the "exact [17] Blackman", with a0 = 7938/18608 ≈ 0.42659, a1 = 9240/18608 ≈ 0.49656, and a2 = 1430/18608 ≈ 0.076849.[18] These exact values place zeros at the third and fourth sidelobes.

Blackman window; α = 0.16; B = 1.73.

Nuttall window, continuous first derivative Considering n as a real number, the function and its first derivative are continuous everywhere.

Nuttall window, continuous first derivative; B = 2.0212.

Blackman–Nuttall window

Blackman–Nuttall window; B = 1.9761.

Window function

41

Blackman–Harris window A generalization of the Hamming family, produced by adding more shifted sinc functions, meant to minimize side-lobe levels[19][20]

Blackman–Harris window; B = 2.0044.

Flat top window A flat top window is a partially negative-valued window that has a flat top in the frequency domain. Such windows have been made available in spectrum analyzers for the measurement of amplitudes of sinusoidal frequency components. They have a low amplitude measurement error suitable for this purpose, achieved by the spreading of the energy of a sine wave over multiple SRS flat top window; B = 3.7702. bins in the spectrum. This ensures that the unattenuated amplitude of the sinusoid can be found on at least one of the neighboring bins. The drawback of the broad bandwidth is poor frequency resolution. To compensate, a longer window length may be chosen. Flat top windows can be designed using low-pass filter design methods, or they may be of the usual sum-of-cosine-terms variety. An example of the latter is the flat top window available in the Stanford Research Systems (SRS) SR785 spectrum analyzer:

Rife–Vincent window Rife and Vincent define three classes of windows constructed as sums of cosines; the classes are generalizations of the Hanning window. Their order-P windows are of the form (normalized to have unity average as opposed to unity max as the windows above are): . For order 1, this formula can match the Hanning window for a1 = −1; this is the Rife–Vincent class-I window, defined by minimizing the high-order sidelobe amplitude. The class-I order-2 Rife–Vincent window has a1 = −4/3 and a2 = 1/3. Coefficients for orders up to 4 are tabulated. For orders greater than 1, the Rife–Vincent window

Window function

42

coefficients can be optimized for class II, meaning minimized main-lobe width for a given maximum side-lobe, or for class III, a compromise for which order 2 resembles Blackmann's window. Given the wide variety of Rife–Vincent windows, plots are not given here.

Power-of-cosine windows Window functions in the power-of-cosine family are of the form:

The rectangular window (α = 0), the cosine window (α = 1), and the Hann window (α = 2) are members of this family. Cosine window

The cosine window is also known as the sine window. Cosine window describes the shape of A cosine window convolved by itself is known as the Bohman window.

Adjustable windows Cosine window; B = 1.23.

Gaussian window The Fourier transform of a Gaussian is also a Gaussian (it is an eigenfunction of the Fourier Transform). Since the Gaussian function extends to infinity, it must either be truncated at the ends of the window, or itself windowed with another zero-ended window.[21] Since the log of a Gaussian produces a parabola, this can be used for exact quadratic interpolation in frequency estimation.[22][23][24]

Gaussian window, σ = 0.4; B = 1.45.

The standard deviation of the Gaussian function is σ(N−1)/2 sampling periods.

Window function

43

Confined Gaussian window The confined Gaussian window yields the smallest possible root mean square frequency width σω for a given temporal width σt. These windows optimize the RMS time-frequency bandwidth products. They are computed as the minimum eigenvectors of a parameter-dependent matrix. The confined Gaussian window family contains the cosine window and the Gaussian window in the limiting cases of large and small σt, respectively.

Confined Gaussian window, σt = 0.1N; B = 1.9982.

Approximate confined Gaussian window A confined Gaussian window of temporal width σt is well approximated by:

with the Gaussian:

Approximate confined Gaussian window, σt = 0.1N; B = 1.9979.

The temporal width of the approximate window is asymptotically equal to σt for σt 1, then the conditions result in what is sometimes referred to as undersampling, bandpass sampling, or using a sampling rate less than the Nyquist rate (2fH). For the case of a given sampling frequency, simpler formulae for the constraints on the signal's spectral band are given below. Example: Consider FM radio to illustrate the idea of undersampling. In the US, FM radio operates on the frequency band from fL = 88 MHz to fH = 108 MHz. The bandwidth is given by

The sampling conditions are satisfied for

Therefore, n can be 1, 2, 3, 4, or 5. The value n = 5 gives the lowest sampling frequencies interval and this is a

Spectrum of the FM radio band (88–108 MHz) and its baseband alias under 44 MHz (n = 5) sampling. An anti-alias filter quite tight to the FM radio band is required, and there's not room for stations at nearby expansion channels such as 87.9 without aliasing.

scenario of undersampling. In this case, the signal spectrum fits between 2 and 2.5 times the sampling rate (higher than 86.4–88 MHz but lower than 108–110 MHz). A lower value of n will also lead to a useful sampling rate. For example, using n = 4, the FM band spectrum fits easily between 1.5 and 2.0 times the sampling rate, for a sampling rate near 56 MHz (multiples of the Nyquist frequency being 28, 56, 84, 112, etc.). See the illustrations at the right.

Spectrum of the FM radio band (88–108 MHz) and its baseband alias under 56 MHz (n = 4) sampling, showing plenty of room for bandpass anti-aliasing filter transition bands. The baseband image is frequency-reversed in this case (even n).

When undersampling a real-world signal, the sampling circuit must be fast enough to capture the highest signal frequency of interest. Theoretically, each sample should be taken during an infinitesimally short interval, but this is not practically feasible. Instead, the sampling of the signal should be made in a short enough interval that it can represent the instantaneous value of the signal with the highest frequency. This means that in the FM radio example above, the sampling circuit must be able to capture a signal with a frequency of 108 MHz, not 43.2 MHz. Thus, the sampling frequency may be only a little bit greater than 43.2 MHz, but the input bandwidth of the system must be at least 108 MHz. Similarly, the accuracy of the sampling timing, or aperture uncertainty of the sampler, frequently the analog-to-digital converter, must be appropriate for the frequencies being sampled 108MHz, not the lower sample rate. If the sampling theorem is interpreted as requiring twice the highest frequency, then the required sampling rate would be assumed to be greater than the Nyquist rate 216 MHz. While this does satisfy the last condition on the sampling rate, it is grossly oversampled. Note that if a band is sampled with n > 1, then a band-pass filter is required for the anti-aliasing filter, instead of a lowpass filter.

Undersampling

99

As we have seen, the normal baseband condition for reversible sampling is that X(f) = 0 outside the interval: and the reconstructive interpolation function, or lowpass filter impulse response, is To accommodate undersampling, the bandpass condition is that X(f) = 0 outside the union of open positive and negative frequency bands for some positive integer

.

which includes the normal baseband condition as case n = 1 (except that where the intervals come together at 0 frequency, they can be closed). The corresponding interpolation function is the bandpass filter given by this difference of lowpass impulse responses: . On the other hand, reconstruction is not usually the goal with sampled IF or RF signals. Rather, the sample sequence can be treated as ordinary samples of the signal frequency-shifted to near baseband, and digital demodulation can proceed on that basis, recognizing the spectrum mirroring when n is even. Further generalizations of undersampling for the case of signals with multiple bands are possible, and signals over multidimensional domains (space or space-time) and have been worked out in detail by Igor Kluvánek.

References

Delta-sigma modulation "Sigma delta" redirects here. For the sorority, see Sigma Delta. Delta-sigma (ΔΣ; or sigma-delta, ΣΔ) modulation is a digital signal processing, or DSP method for encoding analog signals into digital signals as found in an ADC. It is also used to transfer higher-resolution digital signals into lower-resolution digital signals as part of the process to convert digital signals into analog. In a conventional ADC, an analog signal is integrated, or sampled, with a sampling frequency and subsequently quantized in a multi-level quantizer into a digital signal. This process introduces quantization error noise. The first step in a delta-sigma modulation is delta modulation. In delta modulation the change in the signal (its delta) is encoded, rather than the absolute value. The result is a stream of pulses, as opposed to a stream of numbers as is the case with PCM. In delta-sigma modulation, the accuracy of the modulation is improved by passing the digital output through a 1-bit DAC and adding (sigma) the resulting analog signal to the input signal, thereby reducing the error introduced by the delta-modulation. This technique has found increasing use in modern electronic components such as converters, frequency synthesizers, switched-mode power supplies and motor controllers, primarily because of its cost efficiency and reduced circuit complexity.[1] Both analog-to-digital converters (ADCs) and digital-to-analog converters (DACs) can employ delta-sigma modulation. A delta-sigma ADC first encodes an analog signal using high-frequency delta-sigma modulation, and then applies a digital filter to form a higher-resolution but lower sample-frequency digital output. On the other hand, a delta-sigma DAC encodes a high-resolution digital input signal into a lower-resolution but higher sample-frequency signal that is mapped to voltages, and then smoothed with an analog filter. In both cases, the temporary use of a lower-resolution signal simplifies circuit design and improves efficiency.

Delta-sigma modulation

100

The coarsely-quantized output of a delta-sigma modulator is occasionally used directly in signal processing or as a representation for signal storage. For example, the Super Audio CD (SACD) stores the output of a delta-sigma modulator directly on a disk.

Motivation Why convert an analog signal into a stream of pulses? In brief, because it is very easy to regenerate pulses at the receiver into the ideal form transmitted. The only part of the transmitted waveform required at the receiver is the time at which the pulse occurred. Given the timing information the transmitted waveform can be reconstructed electronically with great precision. In contrast, without conversion to a pulse stream but simply transmitting the analog signal directly, all noise in the system is added to the analog signal, reducing its quality. Each pulse is made up of a step up followed after a short interval by a step down. It is possible, even in the presence of electronic noise, to recover the timing of these steps and from that regenerate the transmitted pulse stream almost noiselessly. Then the accuracy of the transmission process reduces to the accuracy with which the transmitted pulse stream represents the input waveform.

Why delta-sigma modulation? Delta-sigma modulation converts the analog voltage into a pulse frequency and is alternatively known as Pulse Density modulation or Pulse Frequency modulation. In general, frequency may vary smoothly in infinitesimal steps, as may voltage, and both may serve as an analog of an infinitesimally varying physical variable such as acoustic pressure, light intensity, etc. The substitution of frequency for voltage is thus entirely natural and carries in its train the transmission advantages of a pulse stream. The different names for the modulation method are the result of pulse frequency modulation by different electronic implementations, which all produce similar transmitted waveforms.

Why the delta-sigma analog to digital conversion? The ADC converts the mean of an analog voltage into the mean of an analog pulse frequency and counts the pulses in a known interval so that the pulse count divided by the interval gives an accurate digital representation of the mean analog voltage during the interval. This interval can be chosen to give any desired resolution or accuracy. The method is cheaply produced by modern methods; and it is widely used.

Analog to digital conversion Description The ADC generates a pulse stream in which the frequency of pulses in the stream is proportional to the analog voltage input, , so that the frequency, where k is a constant for the particular implementation. A counter sums the number of pulses that occur in a predetermined period,

so that the sum,

, is

. is chosen so that a digital display of the count, Because

, is a display of

with a predetermined scaling factor.

may take any designed value it may be made large enough to give any desired resolution or accuracy.

Each pulse of the pulse stream has a known, constant amplitude

and duration

, and thus has a known integral

but variable separating interval. In a formal analysis an impulse such as integral

is treated as the Dirac δ (delta) function and is specified by the

step produced on integration. Here we indicate that step as

.

Delta-sigma modulation

101

The interval between pulses, p, is determined by a feedback loop arranged so that

.

The action of the feedback loop is to monitor the integral of v and when that integral has incremented by is indicated by the integral waveform crossing a threshold, then subtracting combined waveform sawtooths between the threshold and ( threshold -

, which

from the integral of v so that the

). At each step a pulse is added to the

pulse stream. Between impulses the slope of the integral is proportional to

. Whence

. It is the pulse stream which is transmitted for delta-sigma modulation but the pulses are counted to form sigma in the case of analogue to digital conversion.

Analysis Shown below the block diagram illustrated in Fig. 1 are waveforms at points designated by numbers 1 to 5 for an input of 0.2 volts on the left and 0.4 volts on the right. In most practical applications the summing interval is large compared with the impulse duration and for signals which are a significant fraction of full scale the variable separating interval is also small compared with the summing interval. The Nyquist–Shannon sampling theorem requires two samples to render a varying input signal. The samples appropriate to this criterion are two successive Σ counts taken in two successive summing intervals. The summing interval, which must accommodate a large count in order to achieve adequate precision, is inevitably long so that the converter can only render relatively low frequencies. Hence it is convenient and fair to represent the input voltage (1) as constant over a few impulses. Consider first the waveforms on the left. 1 is the input and for this short interval is constant at 0.2 V. The stream of

Fig. 1: Block diagram and waveforms for a sigma delta ADC.

Delta-sigma modulation

delta impulses is shown at 2 and the difference between 1 and 2 is shown at 3. This difference is integrated to produce the waveform 4. The threshold detector generates a pulse 5 which starts as the waveform 4 crosses the threshold and is sustained until the waveform 4 falls below the threshold. Within the loop 5 triggers the impulse generator and external to the loop increments the counter. The summing interval is a prefixed time and at its expiry the count is strobed into the buffer and the counter reset. It is necessary that the ratio between Fig. 1a: Effect of clocking impulses the impulse interval and the summing interval is equal to the maximum (full scale) count. It is then possible for the impulse duration and the summing interval to be defined by the same clock with a suitable arrangement of logic and counters. This has the advantage that neither interval has to be defined with absolute precision as only the ratio is important. Then to achieve overall accuracy it is only necessary that the amplitude of the impulse be accurately defined. On the right the input is now 0.4 V and the sum during the impulse is −0.6 V as opposed to −0.8 V on the left. Thus the negative slope during the impulse is lower on the right than on the left. Also the sum is 0.4 V on the right during the interval as opposed to 0.2 V on the left. Thus the positive slope outside the impulse is higher on the right than on the left. The resultant effect is that the integral (4) crosses the threshold more quickly on the right than on the left. A full analysis would show that in fact the interval between threshold crossings on the right is half that on the left. Thus the frequency of impulses is doubled. Hence the count increments at twice the speed on the right to that on the left which is consistent with the input voltage being doubled. Construction of the waveforms illustrated at (4) is aided by concepts associated with the Dirac delta function in that all impulses of the same strength produce the same step when integrated, by definition. Then (4) is constructed using an intermediate step (6) in which each integrated impulse is represented by a step of the assigned strength which decays to zero at the rate determined by the input voltage. The effect of the finite duration of the impulse is constructed in (4) by drawing a line from the base of the impulse step at zero volts to intersect the decay line from (6) at the full duration of the impulse. As stated, Fig. 1 is a simplified block diagram of the delta-sigma ADC in which the various functional elements have been separated out for individual treatment and which tries to be independent of any particular implementation. Many particular implementations seek to define the impulse duration and the summing interval from the same clock as discussed above but in such a way that the start of the impulse is delayed until the next occurrence of the appropriate clock pulse boundary. The effect of this delay is illustrated in Fig. 1a for a sequence of impulses which occur at a nominal 2.5 clock intervals, firstly for impulses generated immediately the threshold is crossed as previously discussed and secondly for impulses delayed by the clock. The effect of the delay is firstly that the ramp continues until the onset of the impulse, secondly that the impulse produces a fixed amplitude step so that the integral retains the excess it acquired during the impulse delay and so the ramp restarts from a higher point and is now on the same locus as the unclocked integral. The effect is that, for this example, the undelayed impulses will occur at clock points 0, 2.5, 5, 7.5, 10, etc. and the clocked impulses will occur at 0, 3, 5, 8, 10, etc. The maximum error that can occur due to clocking is marginally less than one count. Although the Sigma-Delta converter is

102

Delta-sigma modulation

103

generally implemented using a common clock to define the impulse duration and the summing interval it is not absolutely necessary and an implementation in which the durations are independently defined avoids one source of noise, the noise generated by waiting for the next common clock boundary. Where noise is a primary consideration that overrides the need for absolute amplitude accuracy; e.g., in bandwidth limited signal transmission, separately defined intervals may be implemented.

Practical Implementation A circuit diagram for a practical implementation is illustrated, Fig 1b and the associated waveforms Fig. 1c. This circuit diagram is mainly for illustration purposes, details of particular manufacturers implementations will usually be available from the particular manufacturer. A scrap view of an alternative front end is shown in Fig. 1b which has the advantage that the voltage at the switch terminals are relatively constant and close to 0.0 V. Also the current generated through R by −Vref is constant at −Vref/R so that Fig. 1b: circuit diagram much less noise is radiated to adjacent parts of the circuit. Then this would be the preferred front end in practice but, in order to show the impulse as a voltage pulse so as to be consistent with previous discussion, the front end given here, which is an electrical equivalent, is used. From the top of Fig 1c the waveforms, labelled as they are on the circuit diagram, are:The clock. (a) Vin. This is shown as varying from 0.4 V initially to 1.0 V and then to zero volts to show the effect on the feedback loop. (b) The impulse waveform. It will be discovered how this acquires its form as we traverse the feedback loop. (c)

The

current

into

the

capacitor,

Ic,

is

the

linear

sum

of

the

impulse

voltage

Delta-sigma modulation

104

upon R and Vin upon R. To show this sum as a voltage the product R × Ic is plotted. The input impedance of the amplifier is regarded as so high that the current drawn by the input is neglected. (d) The negated integral of Ic. This negation is standard for the op. amp. implementation of an integrator and comes about because the current into the capacitor at the amplifier input is the current out of the capacitor at the amplifier output and the voltage is the integral of the current divided by the capacitance of C.

Fig. 1c: ADC waveforms

(e) The comparator output. The comparator is a very high gain amplifier with its plus input terminal connected for reference to 0.0 V. Whenever the negative input terminal is taken negative with respect the positive terminal of the amplifier the output saturates positive and conversely negative saturation for positive input. Thus the output saturates positive whenever the integral (d) goes below the 0 V reference level and remains there until (d) goes positive with respect to the reference level. (f) The impulse timer is a D type positive edge triggered flip flop. Input information applied at D is transferred to Q on the occurrence of the positive edge of the clock pulse. thus when the comparator output (e) is positive Q goes positive or remains positive at the next positive clock edge. Similarly, when (e) is negative Q goes negative at the next positive clock edge. Q controls the electronic switch to generate the current impulse into the integrator. Examination of the waveform (e) during the initial period illustrated, when Vin is 0.4 V, shows (e) crossing the threshold well before the trigger edge (positive edge of the clock pulse) so that there is an appreciable delay before the impulse starts. After the start of the impulse there is further delay while (e) climbs back past the threshold. During this time the comparator output remains high but goes low before the next trigger edge. At that next trigger edge the impulse timer goes low to follow the comparator. Thus the clock determines the duration of the impulse. For the next impulse the threshold is crossed immediately before the trigger edge and so the comparator is only briefly positive. Vin (a) goes to full scale, +Vref, shortly before the end of the next impulse. For the remainder of that impulse the capacitor current (c) goes to zero and hence the integrator slope briefly goes to zero. Following this impulse the full scale positive current is flowing (c) and the integrator sinks at its maximum rate and so crosses the threshold well before the next trigger edge. At that edge the impulse starts and the Vin current is now matched by the reference current so that the net capacitor current (c) is zero. Then the integration now has zero slope and remains at the negative value it had at the start of the impulse. This has the effect that the impulse current remains switched on because Q is stuck positive because the comparator is stuck positive at every trigger edge. This is consistent with

Delta-sigma modulation contiguous, butting impulses which is required at full scale input. Eventually Vin (a) goes to zero which means that the current sum (c) goes fully negative and the integral ramps up. It shortly thereafter crosses the threshold and this in turn is followed by Q, thus switching the impulse current off. The capacitor current (c) is now zero and so the integral slope is zero, remaining constant at the value it had acquired at the end of the impulse. (g) The countstream is generated by gating the negated clock with Q to produce this waveform. Thereafter the summing interval, sigma count and buffered count are produced using appropriate counters and registers. The Vin waveform is approximated by passing the countstream (g) into a low pass filter, however it suffers from the defect discussed in the context of Fig. 1a. One possibility for reducing this error is to halve the feedback pulse length to half a clock period and double its amplitude by halving the impulse defining resistor thus producing an impulse of the same strength but one which never butts onto its adjacent impulses. Then there will be a threshold crossing for every impulse. In this arrangement a monostable flip flop triggered by the comparator at the threshold crossing will closely follow the threshold crossings and thus eliminate one source of error, both in the ADC and the sigma delta modulator.

Remarks In this section we have mainly dealt with the analogue to digital converter as a stand alone function which achieves astonishing accuracy with what is now a very simple and cheap architecture. Initially the Delta-Sigma configuration was devised by INOSE et al. to solve problems in the accurate transmission of analog signals. In that application it was the pulse stream that was transmitted and the original analog signal recovered with a low pass filter after the received pulses had been reformed. This low pass filter performed the summation function associated with Σ. The highly mathematical treatment of transmission errors was introduced by them and is appropriate when applied to the pulse stream but these errors are lost in the accumulation process associated with Σ to be replaced with the errors associated with the mean of means when discussing the ADC. For those uncomfortable with this assertion consider this. It is well known that by Fourier analysis techniques the incoming waveform can be represented over the summing interval by the sum of a constant plus a fundamental and harmonics each of which has an exact integer number of cycles over the sampling period. It is also well known that the integral of a sine wave or cosine wave over one or more full cycles is zero. Then the integral of the incoming waveform over the summing interval reduces to the integral of the constant and when that integral is divided by the summing interval it becomes the mean over that interval. The interval between pulses is proportional to the inverse of the mean of the input voltage during that interval and thus over that interval, ts, is a sample of the mean of the input voltage proportional to V/ts. Thus the average of the input voltage over the summing period is VΣ/N and is the mean of means and so subject to little variance. Unfortunately the analysis for the transmitted pulse stream has, in many cases, been carried over, uncritically, to the ADC. It was indicated in section 2.2 Analysis that the effect of constraining a pulse to only occur on clock boundaries is to introduce noise, that generated by waiting for the next clock boundary. This will have its most deleterious effect on the high frequency components of a complex signal. Whilst the case has been made for clocking in the ADC environment, where it removes one source of error, namely the ratio between the impulse duration and the summing interval, it is deeply unclear what useful purpose clocking serves in a single channel transmission environment since it is a source of both noise and complexity but it is conceivable that it would be useful in a TDM (time division multiplex) environment. A very accurate transmission system with constant sampling rate may be formed using the full arrangement shown here by transmitting the samples from the buffer protected with redundancy error correction. In this case there will be a trade off between bandwidth and N, the size of the buffer. The signal recovery system will require redundancy

105

Delta-sigma modulation error checking, digital to analog conversion,and sample and hold circuitry. A possible further enhancement is to include some form of slope regeneration.This amounts to PCM (pulse code modulation) with digitization performed by a sigma-delta ADC. The above description shows why the impulse is called delta. The integral of an impulse is a step. A one bit DAC may be expected to produce a step and so must be a conflation of an impulse and an integration. The analysis which treats the impulse as the output of a 1-bit DAC hides the structure behind the name (sigma delta) and cause confusion and difficulty interpreting the name as an indication of function. This analysis is very widespread but is deprecated. A modern alternative method for generating voltage to frequency conversion is discussed in synchronous voltage to frequency converter (SVFC) which may be followed by a counter to produce a digital representation in a similar manner to that described above.[2]

Digital to analog conversion Discussion Delta-sigma modulators are often used in digital to analog converters (DACs). In general, a DAC converts a digital number representing some analog value into that analog value. For example, the analog voltage level into a speaker may be represented as a 20 bit digital number, and the DAC converts that number into the desired voltage. To actually drive a load (like a speaker) a DAC is usually connected to or integrated with an electronic amplifier. This can be done using a delta-sigma modulator in a Class D Amplifier. In this case, a multi-bit digital number is input to the delta-sigma modulator, which converts it into a faster sequence of 0's and 1's. These 0's and 1's are then converted into analog voltages. The conversion, usually with MOSFET drivers, is very efficient in terms of power because the drivers are usually either fully on or fully off, and in these states have low power loss. The resulting two-level signal is now like the desired signal, but with higher frequency components to change the signal so that it only has two levels. These added frequency components arise from the quantization error of the delta-sigma modulator, but can be filtered away by a simple low-pass filter. The result is a reproduction of the original, desired analog signal from the digital values. The circuit itself is relatively inexpensive. The digital circuit is small, and the MOSFETs used for the power amplification are simple. This is in contrast to a multi-bit DAC which can have very stringent design conditions to precisely represent digital values with a large number of bits. The use of a delta-sigma modulator in the digital to analog conversion has enabled a cost-effective, low power, and high performance solution.

106

Delta-sigma modulation

107

Relationship to Δ-modulation ΔΣ modulation (SDM) is inspired by Δ modulation (DM), as shown in Fig. 2. If quantization were homogeneous (e.g., if it were linear), the following would be a sufficient derivation of the equivalence of DM and SDM: 1. Start with a block diagram of a Δ-modulator/demodulator. 2. The linearity property of integration ( ) makes it possible to move the integrator, which reconstructs the analog signal in the demodulator section, in front of the Δ-modulator. 3. Again, the linearity property of the integration allows the two integrators to be combined and a ΔΣ-modulator/demodulator block diagram is obtained.

Fig. 2: Derivation of ΔΣ- from Δ-modulation

However, the quantizer is not homogeneous, and so this explanation is flawed. It's true that ΔΣ is inspired by Δ-modulation, but the two are distinct in operation. From the first block diagram in Fig. 2, the integrator in the feedback path can be removed if the feedback is taken directly from the input of the low-pass filter. Hence, for delta modulation of input signal , the low-pass filter sees the signal

However, sigma-delta modulation of the same input signal places at the low-pass filter

In other words, SDM and DM swap the position of the integrator and quantizer. The net effect is a simpler implementation that has the added benefit of shaping the quantization noise away from signals of interest (i.e., signals of interest are low-pass filtered while quantization noise is high-pass filtered). This effect becomes more dramatic with increased oversampling, which allows for quantization noise to be somewhat programmable. On the other hand, Δ-modulation shapes both noise and signal equally. Additionally, the quantizer (e.g., comparator) used in DM has a small output representing a small step up and down the quantized approximation of the input while the quantizer used in SDM must take values outside of the range of the input signal, as shown in Fig. 3.

Fig. 3: An example of SDM of 100 samples of one period a sine wave. 1-bit samples (e.g., comparator output) overlaid with sine wave where logic high (e.g., ) represented by blue and logic low (e.g., ) represented by white.

In general, ΔΣ has some advantages versus Δ modulation:

Delta-sigma modulation

108

• The whole structure is simpler: • Only one integrator is needed • The demodulator can be a simple linear filter (e.g., RC or LC filter) to reconstruct the signal • The quantizer (e.g., comparator) can have full-scale outputs • The quantized value is the integral of the difference signal, which makes it less sensitive to the rate of change of the signal.

Principle The principle of the ΔΣ architecture is explained at length in section 2. Initially, when a sequence starts, the circuit will have an arbitrary state which is dependant on the integral of all previous history. In mathematical terms this corresponds to the arbitrary integration constant of the indefinite integral. This follows from the fact that at the heart of the method there is an integrator which can have any arbitrary state dependant on previous input, see Fig. 1c (d). From the occurrence of the first pulse onward the frequency of the pulse stream is proportional to the input voltage to be transformed. A demonstration applet is available online to simulate the whole architecture.[3]

Variations There are many kinds of ADC that use this delta-sigma structure. The above analysis focuses on the simplest 1st-order, 2-level, uniform-decimation sigma-delta ADC. Many ADCs use a second-order 5-level sinc3 sigma-delta structure.

2nd order and higher order modulator The number of integrators, and consequently, the numbers of feedback loops, indicates the order of a ΔΣ-modulator; a 2nd order ΔΣ modulator is shown in Fig. 4. First order modulators are unconditionally stable, but stability analysis must be performed for higher order modulators.

Fig. 4: Block diagram of a 2nd order ΔΣ modulator

3-level and higher quantizer The modulator can also be classified by the number of bits it has in output, which strictly depends on the output of the quantizer. The quantizer can be realized with a N-level comparator, thus the modulator has log2N-bit output. A simple comparator has 2 levels and so is 1 bit quantizer; a 3-level quantizer is called a "1.5" bit quantizer; a 4-level quantizer is a 2 bit quantizer; a 5-level quantizer is called a "2.5 bit" quantizer.[4]

Decimation structures The conceptually simplest decimation structure is a counter that is reset to zero at the beginning of each integration period, then read out at the end of the integration period. The multi-stage noise shaping (MASH) structure has a noise shaping property, and is commonly used in digital audio and fractional-N frequency synthesizers. It comprises two or more cascaded overflowing accumulators, each of which is equivalent to a first-order sigma delta modulator. The carry outputs are combined through summations and delays to produce a binary output, the width of which depends on the number of stages (order) of the MASH. Besides its noise shaping function, it has two more attractive properties:

Delta-sigma modulation

109

• simple to implement in hardware; only common digital blocks such as accumulators, adders, and D flip-flops are required • unconditionally stable (there are no feedback loops outside the accumulators) A very popular decimation structure is the sinc filter. For 2nd order modulators, the sinc3 filter is close to optimum.[5][6]

Quantization theory formulas Main article: Quantization (signal processing) When a signal is quantized, the resulting signal approximately has the second-order statistics of a signal with independent additive white noise. Assuming that the signal value is in the range of one step of the quantized value with an equal distribution, the root mean square value of this quantization noise is

In reality, the quantization noise is of course not independent of the signal; this dependence is the source of idle tones and pattern noise in Sigma-Delta converters. Over sampling ratio (OSR), where

is the sampling frequency and

is Nyquist rate

The RMS noise voltage within the band of interest can be expressed in terms of OSR

Oversampling Main article: Oversampling Let's consider a signal at frequency and a sampling frequency of much higher than Nyquist rate (see fig. 5). ΔΣ modulation is based on the technique of oversampling to reduce the noise in the band of interest (green), which also avoids the use of high-precision analog circuits for the anti-aliasing filter. The quantization noise is the same both in a Nyquist converter (in yellow) and in an oversampling converter (in blue), but it is distributed over a larger spectrum. In ΔΣ-converters, noise is further reduced at low frequencies, which is the band Fig. 5: Noise shaping curves and noise spectrum in ΔΣ modulator where the signal of interest is, and it is increased at the higher frequencies, where it can be filtered. This technique is known as noise shaping. For a first order delta sigma modulator, the noise is shaped by a filter with transfer function Assuming that the sampling frequency approximated as:

.

, the quantization noise in the desired signal bandwidth can be

Delta-sigma modulation

110

. Similarly for a second order delta sigma modulator, the noise is shaped by a filter with transfer function . The in-band quantization noise can be approximated as: . In general, for a

-order ΔΣ-modulator, the variance of the in-band quantization noise: .

When the sampling frequency is doubled, the signal to quantization noise is improved by

for a

-order ΔΣ-modulator. The higher the oversampling ratio, the higher the signal-to-noise ratio and the higher the resolution in bits. Another key aspect given by oversampling is the speed/resolution tradeoff. In fact, the decimation filter put after the modulator not only filters the whole sampled signal in the band of interest (cutting the noise at higher frequencies), but also reduces the frequency of the signal increasing its resolution. This is obtained by a sort of averaging of the higher data rate bitstream.

Example of decimation Let's have, for instance, an 8:1 decimation filter and a 1-bit bitstream; if we have an input stream like 10010110, counting the number of ones, we get 4. Then the decimation result is 4/8 = 0.5. We can then represent it with a 3-bits number 100 (binary), which means half of the largest possible number. In other words, • the sample frequency is reduced by a factor of eight • the serial (1-bit) input bus becomes a parallel (3-bits) output bus.

Naming The technique was first presented in the early 1960s by professor Haruhiko Yasuda while he was a student at Waseda University, Tokyo, Japan.Wikipedia:Citation needed The name Delta-Sigma comes directly from the presence of a Delta modulator and an integrator, as firstly introduced by Inose et al. in their patent application.[7] That is, the name comes from integrating or "summing" differences, which are operations usually associated with Greek letters Sigma and Delta respectively. Both names Sigma-Delta and Delta-Sigma are frequently used.

References [1] http:/ / www. numerix-dsp. com/ appsnotes/ APR8-sigma-delta. pdf [2] Voltage-to-Frequency Converters (http:/ / www. analog. com/ static/ imported-files/ tutorials/ MT-028. pdf) by Walt Kester and James Bryant 2009. Analog Devices. [3] Analog Devices : Virtual Design Center : Interactive Design Tools : Sigma-Delta ADC Tutorial (http:/ / designtools. analog. com/ dt/ sdtutorial/ sdtutorial. html) [4] Sigma-delta class-D amplifier and control method for a sigma-delta class-D amplifier (http:/ / www. faqs. org/ patents/ app/ 20090072897) by Jwin-Yen Guo and Teng-Hung Chang [5] A Novel Architecture for DAQ in Multi-channel, Large Volume, Long Drift Liquid Argon TPC (http:/ / www. slac. stanford. edu/ econf/ C0604032/ papers/ 0232. PDF) by S. Centro, G. Meng, F. Pietropaola, S. Ventura 2006 [6] A Low Power Sinc3 Filter for ΣΔ Modulators (http:/ / ieeexplore. ieee. org/ xpl/ freeabs_all. jsp?arnumber=4253561) by A. Lombardi, E. Bonizzoni, P. Malcovati, F. Maloberti 2007 [7] H. Inose, Y. Yasuda, J. Murakami, "A Telemetering System by Code Manipulation -- ΔΣ Modulation", IRE Trans on Space Electronics and Telemetry, Sep. 1962, pp. 204-209.

• Walt Kester (October 2008). "ADC Architectures III: Sigma-Delta ADC Basics" (http://www.analog.com/ static/imported-files/tutorials/MT-022.pdf) (PDF). Analog Devices. Retrieved 2010-11-02.

Delta-sigma modulation • R. Jacob Baker (2009). CMOS Mixed-Signal Circuit Design (http://CMOSedu.com/) (2nd ed.). Wiley-IEEE. ISBN 978-0-470-29026-2. • R. Schreier, G. Temes (2005). Understanding Delta-Sigma Data Converters. ISBN 0-471-46585-2. • S. Norsworthy, R. Schreier, G. Temes (1997). Delta-Sigma Data Converters. ISBN 0-7803-1045-4. • J. Candy, G. Temes (1992). Oversampling Delta-sigma Data Converters. ISBN 0-87942-285-8.

External links • 1-bit A/D and D/A Converters (http://www.cs.tut.fi/sgn/arg/rosti/1-bit/) • Sigma-delta techniques extend DAC resolution (http://www.embedded.com/design/configurable-systems/ 4006431/Sigma-delta-techniques-extend-DAC-resolution) article by Tim Wescott 2004-06-23 • Tutorial on Designing Delta-Sigma Modulators: Part I (http://www.commsdesign.com/design_corner/ showArticle.jhtml?articleID=18402743) and Part II (http://www.commsdesign.com/design_corner/ showArticle.jhtml?articleID=18402763) by Mingliang (Michael) Liu • Gabor Temes' Publications (http://eecs.oregonstate.edu/research/members/temes/pubs.html) • Simple Sigma Delta Modulator example (http://electronjunkie.wordpress.com/tag/sigma-delta-modulation/) Contains Block-diagrams, code, and simple explanations • Example Simulink model & scripts for continuous-time sigma-delta ADC (http://www.circuitdesign.info/blog/ 2008/09/example-simulink-model-scripts/) Contains example matlab code and Simulink model • Bruce Wooley's Delta-Sigma Converter Projects (http://www-cis.stanford.edu/icl/wooley-grp/projects.html) • An Introduction to Delta Sigma Converters (http://www.beis.de/Elektronik/DeltaSigma/DeltaSigma.html) (which covers both ADC's and DAC's sigma-delta) • Demystifying Sigma-Delta ADCs (http://www.maxim-ic.com/appnotes.cfm/an_pk/1870/CMP/WP-10). This in-depth article covers the theory behind a Delta-Sigma analog-to-digital converter. • Motorola digital signal processors: Principles of sigma-delta modulation for analog-to-digital converters (http:// digitalsignallabs.com/SigmaDelta.pdf) • One-Bit Delta Sigma D/A Conversion Part I: Theory (http://www.digitalsignallabs.com/presentation.pdf) article by Randy Yates presented at the 2004 comp.dsp conference • MASH (Multi-stAge noise SHaping) structure (http://www.aholme.co.uk/Frac2/Mash.htm) with both theory and a block-level implementation of a MASH • Continuous time sigma-delta ADC noise shaping filter circuit architectures (http://www.circuitdesign.info/ blog/2008/11/continuous-time-sigma-delta-adc-noise-shaping-filter-circuit-architectures-2/) discusses architectural trade-offs for continuous-time sigma-delta noise-shaping filters • Some intuitive motivation for why a Delta Sigma modulator works (http://www.cardinalpeak.com/blog/ ?p=392/)

111

Jitter

112

Jitter For other meanings of this word, see Jitter (disambiguation). Jitter is the undesired deviation from true periodicity of an assumed periodic signal in electronics and telecommunications, often in relation to a reference clock source. Jitter may be observed in characteristics such as the frequency of successive pulses, the signal amplitude, or phase of periodic signals. Jitter is a significant, and usually undesired, factor in the design of almost all communications links (e.g., USB, PCI-e, SATA, OC-48). In clock recovery applications it is called timing jitter.[1] Jitter can be quantified in the same terms as all time-varying signals, e.g., RMS, or peak-to-peak displacement. Also like other time-varying signals, jitter can be expressed in terms of spectral density (frequency content). Jitter period is the interval between two times of maximum effect (or minimum effect) of a signal characteristic that varies regularly with time. Jitter frequency, the more commonly quoted figure, is its inverse. ITU-T G.810 classifies jitter frequencies below 10 Hz as wander and frequencies at or above 10 Hz as jitter. Jitter may be caused by electromagnetic interference (EMI) and crosstalk with carriers of other signals. Jitter can cause a display monitor to flicker, affect the performance of processors in personal computers, introduce clicks or other undesired effects in audio signals, and loss of transmitted data between network devices. The amount of tolerable jitter depends on the affected application.

Sampling jitter In analog to digital and digital to analog conversion of signals, the sampling is normally assumed to be periodic with a fixed period—the time between every two samples is the same. If there is jitter present on the clock signal to the analog-to-digital converter or a digital-to-analog converter, the time between samples varies and instantaneous signal error arises. The error is proportional to the slew rate of the desired signal and the absolute value of the clock error. Various effects such as noise (random jitter), or spectral components (periodic jitter)Wikipedia:Citing sources can come about depending on the pattern of the jitter in relation to the signal. In some conditions, less than a nanosecond of jitter can reduce the effective bit resolution of a converter with a Nyquist frequency of 22 kHz to 14 bits.Wikipedia:Citation needed This is a consideration in high-frequency signal conversion, or where the clock signal is especially prone to interference.

Packet jitter in computer networks Main article: Packet delay variation In the context of computer networks, jitter is the variation in latency as measured in the variability over time of the packet latency across a network. A network with constant latency has no variation (or jitter). Packet jitter is expressed as an average of the deviation from the network mean latency. However, for this use, the term is imprecise. The standards-based term is "packet delay variation" (PDV).[2] PDV is an important quality of service factor in assessment of network performance.

Compact disc seek jitter In the context of digital audio extraction from Compact Discs, seek jitter causes extracted audio samples to be doubled-up or skipped entirely if the Compact Disc drive re-seeks. The problem occurs because the Red Book does not require block-accurate addressing during seeking. As a result, the extraction process may restart a few samples early or late, resulting in doubled or omitted samples. These glitches often sound like tiny repeating clicks during playback. A successful approach to correction in software involves performing overlapping reads and fitting the data

Jitter

113 to find overlaps at the edges. Most extraction programs perform seek jitter correction. CD manufacturers avoid seek jitter by extracting the entire disc in one continuous read operation, using special CD drive models at slower speeds so the drive does not re-seek. A jitter meter is a testing instrument for measuring clock jitter values, and is used in manufacturing DVD and CD-ROM discs. Due to additional sector level addressing added in the Yellow Book, CD-ROM data discs are not subject to seek jitter.

Jitter metrics For clock jitter, there are three commonly used metrics: absolute jitter, period jitter, and cycle to cycle jitter. Absolute jitter is the absolute difference in the position of a clock's edge from where it would ideally be. Period jitter (aka cycle jitter) is the difference between any one clock period and the ideal/average clock period. Accordingly, it can be thought of as the discrete-time derivative of absolute jitter. Period jitter tends to be important in synchronous circuitry like digital state machines where the error-free operation of the circuitry is limited by the shortest possible clock period, and the performance of the circuitry is limited by the average clock period. Hence, synchronous circuitry benefits from minimizing period jitter, so that the shortest clock period approaches the average clock period. Cycle-to-cycle jitter is the difference in length/duration of any two adjacent clock periods. Accordingly, it can be thought of as the discrete-time derivative of period jitter. It can be important for some types of clock generation circuitry used in microprocessors and RAM interfaces. Since they have different generation mechanisms, different circuit effects, and different measurement methodology, it is useful to quantify them separately. In telecommunications, the unit used for the above types of jitter is usually the Unit Interval (abbreviated UI) which quantifies the jitter in terms of a fraction of the ideal period of a bit. This unit is useful because it scales with clock frequency and thus allows relatively slow interconnects such as T1 to be compared to higher-speed internet backbone links such as OC-192. Absolute units such as picoseconds are more common in microprocessor applications. Units of degrees and radians are also used. If jitter has a Gaussian distribution, it is usually quantified using the standard deviation of this distribution (aka. RMS). Often, jitter distribution is significantly non-Gaussian. This can occur if the jitter is caused by external sources such as power supply noise. In these cases, peak-to-peak measurements are more useful. Many efforts have been made to meaningfully quantify distributions In the normal distribution one standard deviation from the mean (dark blue) accounts for that are neither Gaussian nor have about 68% of the set, while two standard deviations from the mean (medium and dark blue) account for about 95% and three standard deviations (light, medium, and dark blue) meaningful peaks (which is the case in account for about 99.7%. all real jitter). All have shortcomings but most tend to be good enough for the purposes of engineering work. Note that typically, the reference point for jitter is defined such that the mean jitter is 0.

Jitter

114 In networking, in particular IP networks such as the Internet, jitter can refer to the variation (statistical dispersion) in the delay of the packets.

Types Random jitter Random Jitter, also called Gaussian jitter, is unpredictable electronic timing noise. Random jitter typically follows a Gaussian distribution or Normal distribution. It is believed to follow this pattern because most noise or jitter in an electrical circuit is caused by thermal noise, which has a Gaussian distribution. Another reason for random jitter to have a distribution like this is due to the central limit theorem. The central limit theorem states that composite effect of many uncorrelated noise sources, regardless of the distributions, approaches a Gaussian distribution. One of the main differences between random and deterministic jitter is that deterministic jitter is bounded and random jitter is unbounded.

Deterministic jitter Deterministic jitter is a type of clock timing jitter or data signal jitter that is predictable and reproducible. The peak-to-peak value of this jitter is bounded, and the bounds can easily be observed and predicted. Deterministic jitter can either be correlated to the data stream (data-dependent jitter) or uncorrelated to the data stream (bounded uncorrelated jitter). Examples of data-dependent jitter are duty-cycle dependent jitter (also known as duty-cycle distortion) and intersymbol interference. Deterministic jitter (or DJ) has a known non-Gaussian probability distribution. n

BER

6.4 10−10 6.7 10−11 7

10−12

7.3 10−13 7.6 10−14

Total jitter Total jitter (T) is the combination of random jitter (R) and deterministic jitter (D): T = Dpeak-to-peak + 2× n×Rrms, in which the value of n is based on the bit error rate (BER) required of the link. A common bit error rate used in communication standards such as Ethernet is 10−12.

Testing Testing for jitter and its measurement is of growing importance to electronics engineers because of increased clock frequencies in digital electronic circuitry to achieve higher device performance. Higher clock frequencies have commensurately smaller eye openings, and thus impose tighter tolerances on jitter. For example, modern computer motherboards have serial bus architectures with eye openings of 160 picoseconds or less. This is extremely small compared to parallel bus architectures with equivalent performance, which may have eye openings on the order of 1000 picoseconds.

Jitter

115 Testing of device performance for jitter tolerance often involves the injection of jitter into electronic components with specialized test equipment. Jitter is measured and evaluated in various ways depending on the type of circuitry under test. For example, jitter in serial bus architectures is measured by means of eye diagrams, according to industry accepted standards. A less direct approach—in which analog waveforms are digitized and the resulting data stream analyzed—is employed when measuring pixel jitter in frame grabbers. In all cases, the goal of jitter measurement is to verify that the jitter will not disrupt normal operation of the circuitry. There are standards for jitter measurement in serial bus architectures. The standards cover jitter tolerance, jitter transfer function and jitter generation, with the required values for these attributes varying among different applications. Where applicable, compliant systems are required to conform to these standards.

Mitigation Anti-jitter circuits Anti-jitter circuits (AJCs) are a class of electronic circuits designed to reduce the level of jitter in a regular pulse signal. AJCs operate by re-timing the output pulses so they align more closely to an idealised pulse signal. They are widely used in clock and data recovery circuits in digital communications, as well as for data sampling systems such as the analog-to-digital converter and digital-to-analog converter. Examples of anti-jitter circuits include phase-locked loop and delay-locked loop. Inside digital to analog converters jitter causes unwanted high-frequency distortions. In this case it can be suppressed with high fidelity clock signal usage.

Jitter buffers Jitter buffers or de-jitter buffers are used to counter jitter introduced by queuing in packet switched networks so that a continuous playout of audio (or video) transmitted over the network can be ensured. The maximum jitter that can be countered by a de-jitter buffer is equal to the buffering delay introduced before starting the play-out of the mediastream. In the context of packet-switched networks, the term packet delay variation is often preferred over jitter. Some systems use sophisticated delay-optimal de-jitter buffers that are capable of adapting the buffering delay to changing network jitter characteristics. These are known as adaptive de-jitter buffers and the adaptation logic is based on the jitter estimates computed from the arrival characteristics of the media packets. Adaptive de-jittering involves introducing discontinuities in the media play-out, which may appear offensive to the listener or viewer. Adaptive de-jittering is usually carried out for audio play-outs that feature a VAD/DTX encoded audio, that allows the lengths of the silence periods to be adjusted, thus minimizing the perceptual impact of the adaptation.

Dejitterizer A dejitterizer is a device that reduces jitter in a digital signal. A dejitterizer usually consists of an elastic buffer in which the signal is temporarily stored and then retransmitted at a rate based on the average rate of the incoming signal. A dejitterizer is usually ineffective in dealing with low-frequency jitter, such as waiting-time jitter.

Filtering A filter can be designed to minimize the effect of sampling jitter. For more information, see the paper by S. Ahmed and T. Chen entitled, "Minimizing the effects of sampling jitters in wireless sensors networks".

Jitter

116

Video and image jitter Video or image jitter occurs when the horizontal lines of video image frames are randomly displaced due to the corruption of synchronization signals or electromagnetic interference during video transmission. Model based dejittering study has been carried out under the framework of digital image/video restoration.

Notes [1] Wolaver, 1991, p.211 [2] RFC 3393, IP Packet Delay Variation Metric for IP Performance Metrics (IPPM), IETF (2002)

References •

This article incorporates public domain material from the General Services Administration document "Federal Standard 1037C" (http://www.its.bldrdoc.gov/fs-1037/fs-1037c.htm) (in support of MIL-STD-188).

• Trischitta, Patrick R. and Varma, Eve L. (1989). Jitter in Digital Transmission Systems. Artech. ISBN 0-89006-248-X. • Wolaver, Dan H. (1991). Phase-Locked Loop Circuit Design. Prentice Hall. ISBN 0-13-662743-9. pages 211–237.

Further reading • Levin, Igor. Terms and concepts involved with digital clocking related to Jitter issues in professional quality digital audio (http://www.antelopeaudio.com/blog/word-clock-sync-jitter/) • Li, Mike P. Jitter and Signal Integrity Verification for Synchronous and Asynchronous I/Os at Multiple to 10 GHz/Gbps (http://www.altera.com/literature/cp/cp-01049-jitter-si-verification.pdf). Presented at International Test Conference 2008. • Li, Mike P. A New Jitter Classification Method Based on Statistical, Physical, and Spectroscopic Mechanisms (http://www.altera.com/literature/cp/cp-01052-jitter-classification.pdf). Presented at DesignCon 2009. • Liu, Hui, Hong Shi, Xiaohong Jiang, and Zhe Li. Pre-Driver PDN SSN, OPD, Data Encoding, and Their Impact on SSJ (http://www.altera.com/literature/cp/cp-01055-impact-ssj.pdf). Presented at Electronics Components and Technology Conference 2009. • Miki, Ohtani, and Kowalski Jitter Requirements (https://mentor.ieee.org/802.11/dcn/04/ 11-04-1458-00-000n-jitter-requirements.ppt) (Causes, solutions and recommended values for digital audio) • Zamek, Iliya. SOC-System Jitter Resonance and Its Impact on Common Approach to the PDN Impedance (http:// www.altera.com/literature/cp/cp-01048-jitter-resonance.pdf). Presented at International Test Conference 2008.

External links • Jitter in VoIP - Causes, solutions and recommended values (http://www.en.voipforo.com/QoS/QoS_Jitter. php) • Jitter Buffer (http://searchenterprisevoice.techtarget.com/sDefinition/0,,sid66_gci906844,00.html) • Definition of Jitter in a QoS Testing Methodology (ftp://ftp.iol.unh.edu/pub/mplsServices/other/ QoS_Testing_Methodology.pdf) • An Introduction to Jitter in Communications Systems (http://www.maxim-ic.com/appnotes.cfm/an_pk/1916/ CMP/WP-34) • Jitter Specifications Made Easy (http://www.maxim-ic.com/appnotes.cfm/an_pk/377/CMP/WP-35) A Heuristic Discussion of Fibre Channel and Gigabit Ethernet Methods

Jitter

117 • Jitter in Packet Voice Networks (http://www.cisco.com/en/US/tech/tk652/tk698/ technologies_tech_note09186a00800945df.shtml) • Phabrix SxE - Hand-held Tool for eye and jitter measurement and analysis (http://www.phabrix.com)

Aliasing This article is about aliasing in signal processing, including computer graphics. For aliasing in computer programming, see aliasing (computing). In signal processing and related disciplines, aliasing is an effect that causes different signals to become indistinguishable (or aliases of one another) when sampled. It also refers to the distortion or artifact that results when the signal reconstructed from samples is different from the original continuous signal. Aliasing can occur in signals sampled in time, for instance digital audio, and is referred to as temporal aliasing. Aliasing can also occur in spatially sampled signals, for instance digital images. Aliasing in spatially sampled signals is called spatial aliasing.

Properly sampled image of brick wall.

Spatial aliasing in the form of a Moiré pattern.

Aliasing

Description When a digital image is viewed, a reconstruction is performed by a display or printer device, and by the eyes and the brain. If the image data is not properly processed during sampling or reconstruction, the reconstructed image will differ from the original image, and an alias is seen. An example of spatial aliasing is the Moiré pattern one can Aliasing example of the A letter in Times New Roman. observe in a poorly pixelized image of a brick wall. Techniques Left: aliased image, right: anti-aliased image. that avoid such poor pixelizations are called spatial anti-aliasing. Aliasing can be caused either by the sampling stage or the reconstruction stage; these may be distinguished by calling sampling aliasing prealiasing and reconstruction aliasing postaliasing. Temporal aliasing is a major concern in the sampling of video and audio signals. Music, for instance, may contain high-frequency components that are inaudible to humans. If a piece of music is sampled at 32000 samples per second (Hz), any frequency components above 16000 Hz (the Nyquist frequency) will cause aliasing when the music is reproduced by a digital to analog converter (DAC). To prevent this an anti-aliasing filter is used to remove components above the Nyquist frequency prior to sampling. In video or cinematography, temporal aliasing results from the limited frame rate, and causes the wagon-wheel effect, whereby a spoked wheel appears to rotate too slowly or even backwards. Aliasing has changed its apparent frequency of rotation. A reversal of direction can be described as a negative frequency. Temporal aliasing frequencies in video and cinematography are determined by the frame rate of the camera, but the relative intensity of the aliased frequencies is determined by the shutter timing (exposure time) or the use of a temporal aliasing reduction filter during filming.[1] Like the video camera, most sampling schemes are periodic; that is, they have a characteristic sampling frequency in time or in space. Digital cameras provide a certain number of samples (pixels) per degree or per radian, or samples per mm in the focal plane of the camera. Audio signals are sampled (digitized) with an analog-to-digital converter, which produces a constant number of samples per second. Some of the most dramatic and subtle examples of aliasing occur when the signal being sampled also has periodic content.

Bandlimited functions Main article: Nyquist–Shannon sampling theorem Actual signals have finite duration and their frequency content, as defined by the Fourier transform, has no upper bound. Some amount of aliasing always occurs when such functions are sampled. Functions whose frequency content is bounded (bandlimited) have infinite duration. If sampled at a high enough rate, determined by the bandwidth, the original function can in theory be perfectly reconstructed from the infinite set of samples.

Bandpass signals Main article: Undersampling Sometimes aliasing is used intentionally on signals with no low-frequency content, called bandpass signals. Undersampling, which creates low-frequency aliases, can produce the same result, with less effort, as frequency-shifting the signal to lower frequencies before sampling at the lower rate. Some digital channelizers exploit aliasing in this way for computational efficiency. See Sampling (signal processing), Nyquist rate (relative to sampling), and Filter bank.

118

Aliasing

119

Sampling sinusoidal functions Sinusoids are an important type of periodic function, because realistic signals are often modeled as the summation of many sinusoids of different frequencies and different amplitudes (with a Fourier series or transform). Understanding what aliasing does to the individual sinusoids is useful in understanding what happens to their sum. Here, a plot depicts a set of samples whose sample-interval is 1, and two (of many) different sinusoids that could have produced the samples. The sample-rate in this case is . For instance, if the interval is 1 second, the rate is 1 sample per second. Nine cycles of the red sinusoid and 1 cycle of the blue sinusoid span an interval of 10. The respective sinusoid frequencies are and

Two different sinusoids that fit the same set of samples.

. In general, when a sinusoid of frequency

is sampled with frequency

resulting samples are indistinguishable from those of another sinusoid of frequency The values corresponding to N ≠ 0 are called images or aliases of frequency

the

for any integer N.

In our example, the N = ±1 aliases

of are and A negative frequency is equivalent to its absolute value, because sin(−wt + θ) = sin(wt − θ + π), and cos(−wt + θ) = cos(wt − θ). Therefore we can express all the image frequencies as for any integer N (with being the actual signal frequency). Then the N = 1 alias of

is

(and vice versa).

Aliasing matters when one attempts to reconstruct the original waveform from its samples. The most common reconstruction technique produces the smallest of the frequencies. So it is usually important that be the unique minimum. A necessary and sufficient condition for that is commonly called the Nyquist frequency of a system that samples at rate is satisfied if the original signal is the blue sinusoid ( ). But if will produce the blue sinusoid instead of the red one.

where

is

In our example, the Nyquist condition the usual reconstruction method

Folding In the example above, and symmetrical around the frequency

are

And in general, as increases from 0 to decreases from to Similarly, as to

increases from

continues decreasing from

to 0. The black dots are aliases of each other. The solid red line is an example of A graph of amplitude vs frequency for a adjusting amplitude vs frequency. The dashed red lines are the corresponding paths single sinusoid at frequency and some of the aliases. of its aliases at and would look like the 4 black dots in the adjacent figure. The red lines depict the paths (loci) of the 4 dots if we were to adjust the frequency and amplitude of the sinusoid along the solid red segment (between and ). No matter what function we choose to change the

amplitude vs frequency, the graph will exhibit symmetry between 0 and This symmetry is commonly referred to as folding, and another name for (the Nyquist frequency) is folding frequency. Folding is most often observed in practice when viewing the frequency spectrum of real-valued samples using a discrete Fourier transform.

Aliasing

120

Complex sinusoids Complex sinusoids are waveforms whose samples are complex numbers, and the concept of negative frequency is necessary to distinguish them. In that case, the frequencies of the aliases are given by just: Therefore, as increases from goes from

to

up to 0. Consequently,

complex sinusoids do not exhibit folding. Complex samples of real-valued sinusoids have zero-valued imaginary parts and do exhibit folding.

Two complex sinusoids, colored gold and cyan, that fit the same sets of real and imaginary sample points when sampled at the rate (fs) indicated by the grid lines. The case shown here is:

Sample frequency When the condition

is met for the

highest frequency component of the original signal, then it is met for all the frequency components, a condition known as the Nyquist criterion. That is typically approximated by filtering the original signal to attenuate high frequency components before it is sampled. They still generate low-frequency aliases, but at very low amplitude levels, so as not to cause a problem. A filter chosen in anticipation of a certain sample frequency is called an anti-aliasing filter. The filtered signal can subsequently be reconstructed without significant additional distortion, for example by the Whittaker–Shannon interpolation formula.

Illustration of 4 waveforms reconstructed from samples taken at six different rates. Two of the waveforms are sufficiently sampled to avoid aliasing at all six rates. The other two illustrate increasing distortion (aliasing) at the lower rates.

The Nyquist criterion presumes that the frequency content of the signal being sampled has an upper bound. Implicit in that assumption is that the signal's duration has no upper bound. Similarly, the Whittaker–Shannon interpolation formula represents an interpolation filter with an unrealizable frequency response. These assumptions make up a mathematical model that is an idealized approximation, at best, to any realistic situation. The conclusion, that perfect reconstruction is possible, is mathematically correct for the model, but only an approximation for actual samples of an actual signal.

Aliasing

121

Historical usage Historically the term aliasing evolved from radio engineering because of the action of superheterodyne receivers. When the receiver shifts multiple signals down to lower frequencies, from RF to IF by heterodyning, an unwanted signal, from an RF frequency equally far from the local oscillator (LO) frequency as the desired signal, but on the wrong side of the LO, can end up at the same IF frequency as the wanted one. If it is strong enough it can interfere with reception of the desired signal. This unwanted signal is known as an image or alias of the desired signal.

Angular aliasing Aliasing occurs whenever the use of discrete elements to capture or produce a continuous signal causes frequency ambiguity. Spatial aliasing, particular of angular frequency, can occur when reproducing a light field[2] or sound field with discrete elements, as in 3D displays or wave field synthesis of sound. This aliasing is visible in images such as posters with lenticular printing: if they have low angular resolution, then as one moves past them, say from left-to-right, the 2D image does not initially change (so it appears to move left), then as one moves to the next angular image, the image suddenly changes (so it jumps right) – and the frequency and amplitude of this side-to-side movement corresponds to the angular resolution of the image (and, for frequency, the speed of the viewer's lateral movement), which is the angular aliasing of the 4D light field. The lack of parallax on viewer movement in 2D images and in 3-D film produced by stereoscopic glasses (in 3D films the effect is called "yawing", as the image appears to rotate on its axis) can similarly be seen as loss of angular resolution, all angular frequencies being aliased to 0 (constant).

More examples Online audio example The qualitative effects of aliasing can be heard in the following audio demonstration. Six sawtooth waves are played in succession, with the first two sawtooths having a fundamental frequency of 440 Hz (A4), the second two having fundamental frequency of 880 Hz (A5), and the final two at 1760 Hz (A6). The sawtooths alternate between bandlimited (non-aliased) sawtooths and aliased sawtooths and the sampling rate is 22.05 kHz. The bandlimited sawtooths are synthesized from the sawtooth waveform's Fourier series such that no harmonics above the Nyquist frequency are present. The aliasing distortion in the lower frequencies is increasingly obvious with higher fundamental frequencies, and while the bandlimited sawtooth is still clear at 1760 Hz, the aliased sawtooth is degraded and harsh with a buzzing audible at frequencies lower than the fundamental. Sawtooth aliasing demo 440 Hz bandlimited, 440 Hz aliased, 880 Hz bandlimited, 880 Hz aliased, 1760 Hz bandlimited, 1760 Hz aliased

Problems playing this file? See media help.

Aliasing

122

Direction finding A form of spatial aliasing can also occur in antenna arrays or microphone arrays used to estimate the direction of arrival of a wave signal, as in geophysical exploration by seismic waves. Waves must be sampled at more than two points per wavelength, or the wave arrival direction becomes ambiguous.[3]

Further reading "Sampling and reconstruction," Chapter 7 [4] in.

References [1] [2] [3] [4]

Tessive, LLC (2010). "Time Filter Technical Explanation" (http:/ / tessive. com/ time-filter-technical-explanation) The (New) Stanford Light Field Archive (http:/ / lightfield. stanford. edu/ lfs. html) Flanagan J.L., ‘Beamwidth and useable bandwidth of delay- steered microphone arrays’, AT&T Tech. J., 1985, 64, pp. 983–995 http:/ / graphics. stanford. edu/ ~mmp/ chapters/ pbrt_chapter7. pdf

External links • Aliasing by a sampling oscilloscope (http://www.youtube.com/watch?v=g3svU5VJ8Gk&feature=plcp) by Tektronix Application Engineer • Anti-Aliasing Filter Primer (http://lavidaleica.com/content/anti-aliasing-filter-primer)Wikipedia:Link rot by La Vida Leica discusses its purpose and effect on the image recorded. • Frequency Aliasing Demonstration (http://burtonmackenzie.com/2006/07/i-cant-drive-55.html) by Burton MacKenZie using stop frame animation and a clock. • Interactive examples demonstrating the aliasing effect (http://www.onmyphd.com/?p=aliasing)

Anti-aliasing filter An anti-aliasing filter is a filter used before a signal sampler, to restrict the bandwidth of a signal to approximately satisfy the sampling theorem. Since the theorem states that unambiguous interpretation of the signal from its samples is possible when the power of frequencies above the Nyquist frequency is zero, a real anti-aliasing filter can generally not completely satisfy the theorem. A realizable anti-aliasing filter will typically permit some aliasing to occur; the amount of aliasing that does occur depends on a design trade-off between reduction of aliasing and maintaining signal up to the Nyquist frequency and the frequency content of the input signal.

Optical applications In the case of optical image sampling, as by image sensors in digital cameras, the anti-aliasing filter is also known as an optical lowpass filter or blur filter or AA filter. The mathematics of sampling in two spatial dimensions is similar to the mathematics of time-domain sampling, but the filter implementation technologies are different. The typical implementation in digital cameras is two layers of birefringent material such as lithium niobate, which spreads each optical point into a cluster of four points. The choice of spot separation for such a filter involves a tradeoff among sharpness, aliasing, and fill factor (the ratio of the active refracting area of a microlens array to the total contiguous area occupied by the array). In a monochrome or three-CCD or Foveon X3 camera, the microlens array alone, if near 100% effective, can provide a significant anti-aliasing effect, while in color filter array (CFA, e.g. Bayer filter) cameras, an additional filter is generally needed to reduce aliasing to an acceptable level.

Anti-aliasing filter

Sensor based anti-aliasing filter simulation The Pentax K-3 from Ricoh introduced a unique digital sensor based anti-aliasing filter. The filter works by micro vibrating the sensor element. A toggle can on-off anti-aliasing filter as the world's first camera which has its capability.

Audio applications Anti-aliasing filters are commonly used at the input of digital signal processing systems, for example in sound digitization systems; similar filters are used as reconstruction filters at the output of such systems, for example in music players. In the latter case, the filter functions to prevent aliasing in the conversion of samples back to a continuous signal, where again perfect stop-band rejection would be required to guarantee zero aliasing.

Oversampling A technique known as oversampling is commonly used in audio conversion, especially audio output. The idea is to use a higher intermediate digital sample rate, so that a nearly-ideal digital filter can sharply cut off aliasing near the original low Nyquist frequency, while a much simpler analog filter can stop frequencies above the new higher Nyquist frequency. The purpose of oversampling is to relax the requirements on the anti-aliasing filter, or to further reduce the aliasing. Since the initial anti-aliasing filter is analog, oversampling allows for the filter to be cheaper because the requirements are not as stringent, and also allows the anti-aliasing filter to have a smoother frequency response, and thus a less complex phase response. On input, an initial analog anti-aliasing filter is relaxed, the signal is sampled at a high rate, and then downsampled using a nearly ideal digital anti-aliasing filter.

Bandpass signals See also: Undersampling Often, an anti-aliasing filter is a low-pass filter; however, this is not a requirement. Generalizations of the Nyquist–Shannon sampling theorem allow sampling of other band-limited passband signals instead of baseband signals. For signals that are bandwidth limited, but not centered at zero, a band-pass filter can be used as an anti-aliasing filter. For example, this could be done with a single-sideband modulated or frequency modulated signal. If one desired to sample an FM radio broadcast centered at 87.9 MHz and bandlimited to a 200 kHz band, then an appropriate anti-alias filter would be centered on 87.9 MHz with 200 kHz bandwidth (or pass-band of 87.8 MHz to 88.0 MHz), and the sampling rate would be no less than 400 kHz, but should also satisfy other constraints to prevent aliasing.

Signal overload See also: Clipping (audio) It is very important to avoid input signal overload when using an anti-aliasing filter. If the signal is strong enough, it can cause clipping at the analog-to-digital converter, even after filtering. When distortion due to clipping occurs after the anti-aliasing filter, it can create components outside the passband of the anti-aliasing filter; these components can then alias, causing the reproduction of other non-harmonically-related frequencies.

123

Anti-aliasing filter

References

Flash ADC A Flash ADC (also known as a Direct conversion ADC) is a type of analog-to-digital converter that uses a linear voltage ladder with a comparator at each "rung" of the ladder to compare the input voltage to successive reference voltages. Often these reference ladders are constructed of many resistors; however modern implementations show that capacitive voltage division is also possible. The output of these comparators is generally fed into a digital encoder which converts the inputs into a binary value (the collected outputs from the comparators can be thought of as a unary value).

Benefits and drawbacks Flash converters are extremely fast compared to many other types of ADCs which usually narrow in on the "correct" answer over a series of stages. Compared to these, a Flash converter is also quite simple and, apart from the analog comparators, only requires logic for the final conversion to binary. For best accuracy often a track-and-hold circuit is inserted in front of the ADC input. This is needed for many ADC types (like successive approximation ADC), but for Flash ADCs there is no real need for this, because the comparators are the sampling devices. A Flash converter requires a huge number of comparators compared to other ADCs, especially as the precision increases. A Flash converter requires comparators for an n-bit conversion. The size, power consumption and cost of all those comparators makes Flash converters generally impractical for precisions much greater than 8 bits (255 comparators). In place of these comparators, most other ADCs substitute more complex logic and/or analog circuitry which can be scaled more easily for increased precision.

124

Flash ADC

Implementation Flash ADCs have been implemented in many technologies, varying from silicon based bipolar (BJT) and complementary metal oxide FETs (CMOS) technologies to rarely used III-V technologies. Often this type of ADC is used as a first medium sized analog circuit verification. The earliest implementations consisted of a reference ladder of well matched resistors connected to a reference voltage. Each tap at the resistor ladder is used for one comparator, possibly preceded by an amplification stage, and thus generates a logical '0' or '1' depending if the measured voltage is above or below the reference voltage of the resistor tap. The reason to add an amplifier is twofold: it amplifies the A 2-bit Flash ADC Example Implementation with Bubble Error Correction and Digital voltage difference and therefore Encoding suppresses the comparator offset, and the kick-back noise of the comparator towards the reference ladder is also strongly suppressed. Typically designs from 4-bit up to 6-bit, and sometimes 7-bit are produced. Designs with power-saving capacitive reference ladders have been demonstrated. In addition to clocking the comparator(s), these systems also sample the reference value on the input stage. As the sampling is done at a very high rate, the leakage of the capacitors is negligible. Recently, offset calibration has been introduced into flash ADC designs. Instead of high precision analog circuits (which increase component size to suppress variation) comparators with relatively large offset errors are measured and adjusted. A test signal is applied and the offset of each comparator is calibrated to below the LSB size of the ADC. Another improvement to many flash ADCs is the inclusion of digital error correction. When the ADC is used in harsh environments or constructed from very small integrated circuit processes, there is a heightened risk a single comparator will randomly change state resulting in a wrong code. Bubble error correction is a digital correction mechanism that will prevent a comparator that has, for example, tripped high from reporting logic high if it is surrounded by comparators that are reporting logic low.

Folding ADC The number of comparators can be reduced somewhat by adding a folding circuit in front, making a so called folding ADC. Instead of using the comparators in a Flash ADC only once, during a ramp input signal, the folding ADC re-uses the comparators multiple times. If a m-times folding circuit is used in an n-bit ADC, the actual number of comparator can be reduced from to (there is always one needed to detect the range crossover). Typical folding circuits are, e.g., the Gilbert multiplier, or analog wired-or circuits.

125

Flash ADC

Application The very high sample rate of this type of ADC enable gigahertz applications like radar detection, wide band radio receivers and optical communication links. More often the flash ADC is embedded in a large IC containing many digital decoding functions. Also a small flash ADC circuit may be present inside a delta-sigma modulation loop. Flash ADCs are also used in NAND Flash Memory, where up to 3 bits are stored per cell as 8 level voltages on floating gates.

References • Analog to Digital Conversion [1] • Understanding Flash ADCs [2] • "Integrated Analog-to-Digital and Digital-to-Analog Converters ", R. van de Plassche, ADCs, Kluwer Academic Publishers, 1994. • "A Precise Four-Quadrant Multiplier with Subnanosecond Response", Barrie Gilbert, IEEE Journal of Solid-State Circuits, Vol. 3, No. 4 (1968), pp. 365-373

References [1] http:/ / hyperphysics. phy-astr. gsu. edu/ hbase/ electronic/ adc. html#c4 [2] http:/ / www. maxim-ic. com/ appnotes. cfm/ appnote_number/ 810/ CMP/ WP-17

Successive approximation ADC "Successive Approximation" redirects here. For behaviorist B.F. Skinner's method of guiding learned behavior, see Shaping (psychology). A successive approximation ADC is a type of analog-to-digital converter that converts a continuous analog waveform into a discrete digital representation via a binary search through all possible quantization levels before finally converging upon a digital output for each conversion.

126

Successive approximation ADC

127

Block diagram Key • • • • •

DAC = Digital-to-Analog converter EOC = end of conversion SAR = successive approximation register S/H = sample and hold circuit Vin = input voltage

• Vref = reference voltage

Algorithm The successive approximation Analog to digital converter circuit typically consists of four chief subcircuits: 1. A sample and hold circuit to acquire the input voltage (Vin).

Successive Approximation ADC Block Diagram

2. An analog voltage comparator that compares Vin to the output of the internal DAC and outputs the result of the comparison to the successive approximation register (SAR). 3. A successive approximation register subcircuit designed to supply an approximate digital code of Vin to the internal DAC. 4. An internal reference DAC that, for comparison with V, supplies the comparator with an analog voltage equal to the digital code output of the SARin. The successive approximation register is initialized so that the most significant bit (MSB) is equal to a digital 1. This code is fed into the DAC, which then supplies the analog equivalent of this digital code (Vref/2) into the comparator circuit for comparison with the sampled input voltage. If this analog voltage exceeds Vin the comparator causes the SAR to reset this bit; otherwise, the bit is left a 1. Then the next bit is set to 1 and the same test is done, continuing this binary search until every bit in the SAR has been tested. The resulting code is the digital approximation of the sampled input voltage and is finally output by the SAR at the end of the conversion (EOC). Mathematically, let Vin = xVref, so x in [-1, 1] is the normalized input voltage. The objective is to approximately digitize x to an accuracy of 1/2n. The algorithm proceeds as follows: 1. Initial approximation x0 = 0. 2. ith approximation xi = xi-1 - s(xi-1 - x)/2i. where, s(x) is the signum-function(sgn(x)) (+1 for x ≥ 0, -1 for x < 0). It follows using mathematical induction that |xn - x| ≤ 1/2n. As shown in the above algorithm, a SAR ADC requires: 1. 2. 3. 4. 5.

An input voltage source Vin. A reference voltage source Vref to normalize the input. A DAC to convert the ith approximation xi to a voltage. A Comparator to perform the function s(xi - x) by comparing the DAC's voltage with the input voltage. A Register to store the output of the comparator and apply xi-1 - s(xi-1 - x)/2i.

Successive approximation ADC

128

Charge-redistribution successive approximation ADC One of the most common implementations of the successive approximation ADC, the charge-redistribution successive approximation ADC, uses a charge scaling DAC. The charge scaling DAC simply consists of an array of individually switched binary-weighted capacitors. The amount of charge upon each capacitor in the array is used to perform the aforementioned binary search in conjunction with a comparator internal to the DAC and the successive approximation register.

Charge Scaling DAC

1. First, the capacitor array is completely discharged to the offset voltage of the comparator, VOS. This step provides automatic offset cancellation(i.e. The offset voltage represents nothing but dead charge which can't be juggled by the capacitors). 2. Next, all of the capacitors within the array are switched to the input signal, vIN. The capacitors now have a charge equal to their respective capacitance times the input voltage minus the offset voltage upon each of them. 3. In the third step, the capacitors are then switched so that this charge is applied across the comparator's input, creating a comparator input voltage equal to -vIN. 4. Finally, the actual conversion process proceeds. First, the MSB capacitor is switched to VREF, which corresponds to the full-scale range of the ADC. Due to the binary-weighting of the array the MSB capacitor forms a 1:1 charge divider with the rest of the array. Thus, the input voltage to the comparator is now -vIN plus VREF/2. Subsequently, if vIN is greater than VREF/2 then the comparator outputs a digital 1 as the MSB, otherwise it outputs a digital 0 as the MSB. Each capacitor is tested in the same manner until the comparator input voltage converges to the offset voltage, or at least as close as possible given the resolution of the DAC.

Use with non-ideal analog circuits When implemented as an analog circuit - where the value of each successive bit is not perfectly 2^N (e.g. 1.1, 2.12, 4.05, 8.01, etc.) - a successive approximation approach might not output the ideal value because the binary search algorithm incorrectly removes what it believes to be half of the values the unknown input cannot be. Depending on the difference between actual and ideal performance, the maximum error can easily exceed several LSBs, especially as the error 3 bits simulation of a capacitive ADC between the actual and ideal 2^N becomes large for one or more bits. Since we don't know the actual unknown input, it is therefore very important that accuracy of the analog circuit used to implement a SAR ADC be very close to the ideal 2^N values; otherwise, we cannot guarantee a best match search. RECENT IMPROVEMENTS 1. New SAR ADC include now calibration to improve their accuracy from less than 10bits to up to 18bits 2. Another new technique use non-binary weighted DAC and/or redundancy to solve the problem of non-ideal analog circuits and improve speed ADVANTAGES

Successive approximation ADC 1. The conversion time is equal to the "n" clock cycle period for an n-bit ADC. Thus conversion time is very short. For example for a 10-bit ADC with a clock frequency of 1 MHz, the conversion time will be only 10*10^-6 i.e. 10 microseconds. 2. Conversion time is constant and independent of the amplitude of analog signal V to the base A

References • R. J. Baker, CMOS Circuit Design, Layout, and Simulation, Third Edition, Wiley-IEEE, 2010. ISBN 978-0-470-88132-3

External links • Understanding SAR ADCs [1]

References [1] http:/ / www. maxim-ic. com/ appnotes. cfm/ appnote_number/ 1080/ CMP/ WP-50

Integrating ADC An integrating ADC is a type of analog-to-digital converter that converts an unknown input voltage into a digital representation through the use of an integrator. In its most basic implementation, the unknown input voltage is applied to the input of the integrator and allowed to ramp for a fixed time period (the run-up period). Then a known reference voltage of opposite polarity is applied to the integrator and is allowed to ramp until the integrator output returns to zero (the run-down period). The input voltage is computed as a function of the reference voltage, the constant run-up time period, and the measured run-down time period. The run-down time measurement is usually made in units of the converter's clock, so longer integration times allow for higher resolutions. Likewise, the speed of the converter can be improved by sacrificing resolution. Converters of this type can achieve high resolution, but often do so at the expense of speed. For this reason, these converters are not found in audio or signal processing applications. Their use is typically limited to digital voltmeters and other instruments requiring highly accurate measurements.

Basic design The basic integrating ADC circuit consists of an integrator, a switch to select between the voltage to be measured and the reference voltage, a timer that determines how long to integrate the unknown and measures how long the reference integration took, a comparator to detect zero crossing, and a controller. Depending on the implementation, a switch may also be present in parallel with the integrator capacitor to allow Basic integrator of a Dual-slope Integrating ADC. the integrator to be reset (by discharging the integrator capacitor). The The comparator, the timer, and the controller are switches will be controlled electrically by means of the converter's not shown. controller (a microprocessor or dedicated control logic). Inputs to the controller include a clock (used to measure time) and the output of a comparator used to detect when the integrator's output reaches zero. The conversion takes place in two phases: the run-up phase, where the input to the integrator is the voltage to be measured, and the run-down phase, where the input to the integrator is a known reference voltage. During the run-up phase, the switch selects the measured voltage as the input to the integrator. The integrator is allowed to ramp for a

129

Integrating ADC

130

fixed period of time to allow a charge to build on the integrator capacitor. During the run-down phase, the switch selects the reference voltage as the input to the integrator. The time that it takes for the integrator's output to return to zero is measured during this phase. In order for the reference voltage to ramp the integrator voltage down, the reference voltage needs to have a polarity opposite to that of the input voltage. In most cases, for positive input voltages, this means that the reference voltage will be negative. To handle both positive and negative input voltages, a positive and negative reference voltage is required. The selection of which reference to use during the run-down phase would be based on the polarity of the integrator output at the end of the run-up phase. That is, if the integrator's output were negative at the end of the run-up phase, a negative reference voltage would be required. If the integrator's output were positive, a positive reference voltage would be required. The basic equation for the output of the integrator (assuming a constant input) is:

Assuming that the initial integrator voltage at the start of each conversion is zero and that the integrator voltage at the end of the run down period will be zero, we have the following two equations that cover the integrator's output during the two phases of the conversion:

The two equations can be combined and solved for

Integrator output voltage in a basic dual-slope integrating ADC

, the unknown input voltage:

From the equation, one of the benefits of the dual-slope integrating ADC becomes apparent: the measurement is independent of the values of the circuit elements (R and C). This does not mean, however, that the values of R and C are unimportant in the design of a dual-slope integrating ADC (as will be explained below). Note that in the graph to the right, the voltage is shown as going up during the run-up phase and down during the run-down phase. In reality, because the integrator uses the op-amp in a negative feedback configuration, applying a positive will cause the output of the integrator to go down. The up and down more accurately refer to the process of adding charge to the integrator capacitor during the run-up phase and removing charge during the run-down phase. The resolution of the dual-slope integrating ADC is determined primarily by the length of the run-down period and by the time measurement resolution (i.e., the frequency of the controller's clock). The required resolution (in number of bits) dictates the minimum length of the run-down period for a full-scale input ( ):

During the measurement of a full-scale input, the slope of the integrator's output will be the same during the run-up and run-down phases. This also implies that the time of the run-up period and run-down period will be equal ( ) and that the total measurement time will be . Therefore, the total measurement time for a full-scale input will be based on the desired resolution and the frequency of the controller's clock:

Integrating ADC If a resolution of 16 bits is required with a controller clock of 10 MHz, the measurement time will be 13.1 milliseconds (or a sampling rate of just 76 samples per second). However, the sampling time can be improved by sacrificing resolution. If the resolution requirement is reduced to 10 bits, the measurement time is also reduced to only 0.2 milliseconds (almost 4900 samples per second).

Limitations There are limits to the maximum resolution of the dual-slope integrating ADC. It is not possible to increase the resolution of the basic dual-slope ADC to arbitrarily high values by using longer measurement times or faster clocks. Resolution is limited by: • The range of the integrating amplifier. The voltage rails on an op-amp limit the output voltage of the integrator. An input left connected to the integrator for too long will eventually cause the op amp to limit its output to some maximum value, making any calculation based on the run-down time meaningless. The integrator's resistor and capacitor are therefore chosen carefully based on the voltage rails of the op-amp, the reference voltage and expected full-scale input, and the longest run-up time needed to achieve the desired resolution. • The accuracy of the comparator used as the null detector. Wideband circuit noise limits the ability of the comparator to identify exactly when the output of the integrator has reached zero. Goerke suggests a typical limit is a comparator resolution of 1 millivolt.[1] • The quality of the integrator's capacitor. Although the integrating capacitor need not be perfectly linear, it does need to be time-invariant. Dielectric absorption causes errors.[2]

Enhancements The basic design of the dual-slope integrating ADC has a limitations in both conversion speed and resolution. A number of modifications to the basic design have been made to overcome both of these to some degree.

Run-up improvements Enhanced dual-slope The run-up phase of the basic dual-slope design integrates the input voltage for a fixed period of time. That is, it allows an unknown amount of charge to build up on the integrator's capacitor. The run-down phase is then used to measure this unknown charge to determine the unknown voltage. For a full-scale input, half of the measurement time is spent in the run-up phase. For smaller inputs, an even larger percentage of the total measurement time is spent in the run-up phase. Reducing the amount of time spent in the run-up phase can significantly reduce the total measurement time. Enhanced run-up dual-slope integrating ADC A simple way to reduce the run-up time is to increase the rate that charge accumulates on the integrator capacitor by reducing the size of the resistor used on the input, a method referred to as enhanced dual-slope. This still allows the same total amount of charge accumulation, but it does so over a smaller period of time. Using the same algorithm for the run-down phase results in the following equation for the calculation of the unknown input voltage ( ):

Note that this equation, unlike the equation for the basic dual-slope converter, has a dependence on the values of the integrator resistors. Or, more importantly, it has a dependence on the ratio of the two resistance values. This modification does nothing to improve the resolution of the converter (since it doesn't address either of the resolution

131

Integrating ADC

132

limitations noted above). Multi-slope run-up One method to improve the resolution of the converter is to artificially increase the range of the integrating amplifier during the run-up phase. As mentioned above, the purpose of the run-up phase is to add an unknown amount of charge to the integrator to be later measured during the run-down phase. Having the ability to add larger quantities of charge allows for more higher-resolution measurements. For Circuit diagram for a multi-slope run-up converter example, assume that we are capable of measuring the charge on the integrator during the run-down phase to a granularity of 1 coulomb. If our integrator amplifier limits us to being able to add only up to 16 coulombs of charge to the integrator during the run-up phase, our total measurement will be limited to 4 bits (16 possible values). If we can increase the range of the integrator to allow us to add up to 32 coulombs, our measurement resolution is increased to 5 bits. One method to increase the integrator capacity is by periodically adding or subtracting known quantities of charge during the run-up phase in order to keep the integrator's output within the range of the integrator amplifier. Then, the total amount of artificially-accumulated charge is the charge introduced by the unknown input voltage plus the sum of the known charges that were added or subtracted. The circuit diagram shown to the right is an example of how multi-slope run-up could be implemented. The concept is that the unknown input voltage, , is always applied to the integrator. Positive and negative reference voltages controlled by the two independent switches add and subtract charge as needed to keep the output of the integrator within its limits. The reference resistors,

and

are necessarily smaller than

to ensure that the references

can overcome the charge introduced by the input. A comparator is connected to the output to compare the integrator's voltage with a threshold voltage. The output of the comparator is used by the converter's controller to decide which reference voltage should be applied. This can be a relatively simple algorithm: if the integrator's output above the threshold, enable the positive reference (to cause the output to go down); if the integrator's output is below the threshold, enable the negative reference (to cause the output to go up). The controller keeps track of how often each switch is turned on in order to estimate how much additional charge was placed onto (or removed from) the integrator capacitor as a result of the reference voltages. To the right is a graph of sample output from the integrator during a multi-slope run-up. Each dashed vertical line represents a decision point by the controller where it samples the polarity of the output and chooses to apply either the positive or negative reference voltage to the input. Ideally, the output voltage of the integrator at the end of the run-up period can be represented by the following equation: Output from multi-slope run-up

where

is the sampling period,

is the number of periods in which the positive reference is switched in,

the number of periods in which the negative reference is switched in, and

is

is the total number of periods in the

run-up phase. The resolution obtained during the run-up period can be determined by making the assumption that the integrator output at the end of the run-up phase is zero. This allows us to relate the unknown input, , to just the references and the

values:

Integrating ADC

133

The resolution can be expressed in terms of the difference between single steps of the converter's output. In this case, if we solve the above equation for using and (the sum of and

must always equal

), the difference will equal the smallest resolvable quantity. This results in an

equation for the resolution of the multi-slope run-up phase (in bits) of:

Using typical values of the reference resistors

and

of 10k ohms and an input resistor of 50k ohms, we can

achieve a 16 bit resolution during the run-up phase with 655360 periods (65.5 milliseconds with a 10 MHz clock). While it is possible to continue the multi-slope run-up indefinitely, it is not possible to increase the resolution of the converter to arbitrarily high levels just by using a longer run-up time. Error is introduced into the multi-slope run-up through the action of the switches controlling the references, cross-coupling between the switches, unintended switch charge injection, mismatches in the references, and timing errors.[3] Some of this error can be reduced by careful operation of the switches.[4] In particular, during the run-up period, each switch should be activated a constant number of times. The algorithm explained above does not do this and just toggles switches as needed to keep the integrator output within the limits. Activating each switch a constant number of times makes the error related to switching approximately constant. Any output offset that is a result of the switching error can be measured and then subtracted from the result.

Run-down improvements Multi-slope run-down The simple, single-slope run-down is slow. Typically, the run down time is measured in clock ticks, so to get four digit resolution, the rundown time may take as long as 10,000 clock cycles. A multi-slope run-down can speed the measurement up without sacrificing accuracy. By using 4 slope rates that are each a power of ten more gradual than the previous, four digit resolution can be achieved in roughly 40 or fewer clock ticks—a huge speed improvement.[1]

Multi-slope run-down integrating ADC

The circuit shown to the right is an example of a multi-slope run-down circuit with four run-down slopes with each being ten times more gradual than the previous. The switches control which slope is selected. The switch containing selects the steepest slope (i.e., will cause the integrator output to move toward zero the fastest). At the start of the run-down interval, the unknown input is removed from the circuit by opening the switch connected to and closing the switch. Once the integrator's output reaches zero (and the run-down time measured), the final slope of

switch is opened and the next slope is selected by closing the

switch. This repeats until the

has reached zero. The combination of the run-down times for each of the slopes determines the

value of the unknown input. In essence, each slope adds one digit of resolution to the result. In the example circuit, the slope resistors differ by a factor of 10. This value, known as the base (

), can be any

value. As explained below, the choice of the base affects the speed of the converter and determines the number of slopes needed to achieve the desired resolution.

Integrating ADC

134

The basis of this design is the assumption that there will always be overshoot when trying to find the zero crossing at the end of a run-down interval. This will necessarily be true given any hysteresis in the output of the comparator measuring the zero crossing and due to the periodic sampling of the comparator based on the converter's clock. If we assume that the converter switches from one slope to the next in a single clock cycle (which may or may not be possible), the maximum amount of overshoot for a given slope would be the largest integrator output change in one clock period:

To overcome this overshoot, the next slope would require no more than

Output of the multi-slope run-down integrating ADC

clock cycles, which helps to place a

bound on the total time of the run-down. The time for the first-run down (using the steepest slope) is dependent on the unknown input (i.e., the amount of charge placed on the integrator capacitor during the run-up phase). At most, this will be:

where

is the maximum number of clock periods for the first slope,

is the maximum integrator voltage

at the start of the run-down phase, and is the resistor used for the first slope. The remainder of the slopes have a limited duration based on the selected base, so the remaining time of the conversion (in converter clock periods) is:

where

is the number of slopes.

Converting the measured time intervals during the multi-slope run-down into a measured voltage is similar to the charge-balancing method used in the multi-slope run-up enhancement. Each slope adds or subtracts known amounts of charge to/from the integrator capacitor. The run-up will have added some unknown amount of charge to the integrator. Then, during the run-down, the first slope subtracts a large amount of charge, the second slope adds a smaller amount of charge, etc. with each subsequent slope moving a smaller amount in the opposite direction of the previous slope with the goal of reaching closer and closer to zero. Each slope adds or subtracts a quantity of charge proportional to the slope's resistor and the duration of the slope:

is necessarily an integer and will be less than or equal to circuit above as an example, the second slope,

for the second and subsequent slopes. Using the

, can contribute the following charge,

, to the

integrator: in steps of That is,

possible values with the largest equal to the first slope's smallest step, or one (base 10) digit of resolution

per slope. Generalizing this, we can represent the number of slopes, resolution,

, in terms of the base and the required

:

Substituting this back into the equation representing the run-down time required for the second and subsequent slopes gives us this:

Integrating ADC

Which, when evaluated, shows that the minimum run-down time can be achieved using a base of e. This base may be difficult to use both in terms of complexity in the calculation of the result and of finding an appropriate resistor network, so a base of 2 or 4 would be more common. Residue ADC When using run-up enhancements like the multi-slope run-up, where a portion of the converter's resolution is resolved during the run-up phase, it is possible to eliminate the run-down phase altogether by using a second type of analog-to-digital converter.[5] At the end of the run-up phase of a multi-slope run-up conversion, there will still be an unknown amount of charge remaining on the integrator's capacitor. Instead of using a traditional run-down phase to determine this unknown charge, the unknown voltage can be converted directly by a second converter and combined with the result from the run-up phase to determine the unknown input voltage. Assuming that multi-slope run-up as described above is being used, the unknown input voltage can be related to the multi-slope run-up counters, and , and the measured integrator output voltage, using the following equation (derived from the multi-slope run-up output equation):

This equation represents the theoretical calculation of the input voltage assuming ideal components. Since the equation depends on nearly all of the circuit's parameters, any variances in reference currents, the integrator capacitor, or other values will introduce errors in the result. A calibration factor is typically included in the term to account for measured errors (or, as described in the referenced patent, to convert the residue ADC's output into the units of the run-up counters). Instead of being used to eliminate the run-down phase completely, the residue ADC can also be used to make the run-down phase more accurate than would otherwise be possible.[6] With a traditional run-down phase, the run-down time measurement period ends with the integrator output crossing through zero volts. There is a certain amount of error involved in detecting the zero crossing using a comparator (one of the short-comings of the basic dual-slope design as explained above). By using the residue ADC to rapidly sample the integrator output (synchronized with the converter controller's clock, for example), a voltage reading can be taken both immediately before and immediately after the zero crossing (as measured with a comparator). As the slope of the integrator voltage is constant during the run-down phase, the two voltage measurements can be used as inputs to an interpolation function that more accurately determines the time of the zero-crossing (i.e., with a much higher resolution than the controller's clock alone would allow).

Other improvements Continuously-integrating converter By combining some of these enhancements to the basic dual-slope design (namely multi-slope run-up and the residue ADC), it is possible to construct an integrating analog-to-digital converter that is capable of operating continuously without the need for a run-down interval.[7] Conceptually, the multi-slope run-up algorithm is allowed to operate continuously. To start a conversion, two things happen simultaneously: the residue ADC is used to measure the approximate charge currently on the integrator capacitor and the counters monitoring the multi-slope run-up are reset. At the end of a conversion period, another residue ADC reading is taken and the values of the multi-slope run-up counters are noted. The unknown input is calculated using a similar equation as used for the residue ADC, except that two output voltages are included ( representing the measured integrator voltage at the start of the conversion, and

135

Integrating ADC representing the measured integrator voltage at the end of the conversion.

Such a continuously-integrating converter is very similar to a delta-sigma analog-to-digital converter.

Calibration In most variants of the dual-slope integrating converter, the converter's performance is dependent on one or more of the circuit parameters. In the case of the basic design, the output of the converter is in terms of the reference voltage. In more advanced designs, there are also dependencies on one or more resistors used in the circuit or on the integrator capacitor being used. In all cases, even using expensive precision components there may be other effects that are not accounted for in the general dual-slope equations (dielectric effect on the capacitor or frequency or temperature dependencies on any of the components). Any of these variations result in error in the output of the converter. In the best case, this is simply gain and/or offset error. In the worst case, nonlinearity or nonmonotonicity could result. Some calibration can be performed internal to the converter (i.e., not requiring any special external input). This type of calibration would be performed every time the converter is turned on, periodically while the converter is running, or only when a special calibration mode is entered. Another type of calibration requires external inputs of known quantities (e.g., voltage standards or precision resistance references) and would typically be performed infrequently (every year for equipment used in normal conditions, more often when being used in metrology applications). Of these types of error, offset error is the simplest to correct (assuming that there is a constant offset over the entire range of the converter). This is often done internal to the converter itself by periodically taking measurements of the ground potential. Ideally, measuring the ground should always result in a zero output. Any non-zero output indicates the offset error in the converter. That is, if the measurement of ground resulted in an output of 0.001 volts, one can assume that all measurements will be offset by the same amount and can subtract 0.001 from all subsequent results. Gain error can similarly be measured and corrected internally (again assuming that there is a constant gain error over the entire output range). The voltage reference (or some voltage derived directly from the reference) can be used as the input to the converter. If the assumption is made that the voltage reference is accurate (to within the tolerances of the converter) or that the voltage reference has been externally calibrated against a voltage standard, any error in the measurement would be a gain error in the converter. If, for example, the measurement of a converter's 5 volt reference resulted in an output of 5.3 volts (after accounting for any offset error), a gain multiplier of 0.94 (5 / 5.3) can be applied to any subsequent measurement results.

Footnotes [1] Goeke, HP Journal, page 9 [2] Hewlett-Packard Catalog, 1981, page 49, stating, "For small inputs, noise becomes a problem and for large inputs, the dielectric absorption of the capacitor becomes a problem." [3] Eng 1994 [4] Eng 1994, Goeke 1989 [5] Riedel 1992 [6] Regier 2001 [7] Goeke 1992

136

Integrating ADC

References • US 5321403 (http://worldwide.espacenet.com/textdoc?DB=EPODOC&IDX=US5321403), Eng, Jr., Benjamin & Don Matson, "Multiple Slope Analog-to-Digital Converter", issued 14 June 1994 • Goeke, Wayne (April 1989), "8.5-Digit Integrating Analog-to-Digital Converter with 16-Bit, 100,000-Sample-per-Second Performance" (http://www.hpl.hp.com/hpjournal/pdfs/IssuePDFs/1989-04. pdf), HP Journal 40 (2): 8–15 • US 5117227 (http://worldwide.espacenet.com/textdoc?DB=EPODOC&IDX=US5117227), Goeke, Wayne, "Continuously-integrating high-resolution analog-to-digital converter", issued 26 May 1992 • Kester, Walt, The Data Conversion Handbook (http://www.analog.com/library/analogDialogue/archives/ 39-06/data_conversion_handbook.html), ISBN 0-7506-7841-0 • US 6243034 (http://worldwide.espacenet.com/textdoc?DB=EPODOC&IDX=US6243034), Regier, Christopher, "Integrating analog to digital converter with improved resolution", issued 5 June 2001 • US 5101206 (http://worldwide.espacenet.com/textdoc?DB=EPODOC&IDX=US5101206), Riedel, Ronald, "Integrating analog to digital converter", issued 31 March 1992

Time-stretch analog-to-digital converter The time-stretch analog-to-digital converter (TS-ADC),[1][2][3] also known as the Time Stretch Enhanced Recorder (TiSER), is an analog-to-digital converter (ADC) system that has the capability of digitizing very high bandwidth signals that cannot be captured by conventional electronic ADCs. Alternatively, it is also known as the photonic time stretch (PTS) digitizer,[4] since it uses an optical frontend. It relies on the process of time-stretch, which effectively slows down the analog signal in time (or compresses its bandwidth) before it can be digitized by a slow electronic ADC.

Background There is a huge demand for very high speed analog-to-digital converters (ADCs), as they are needed for test and measurement equipment in laboratories and in high speed data communications systems. Most of the ADCs are based purely on electronic circuits, which have limited speeds and add a lot of impairments, limiting the bandwidth of the signals that can be digitized and the achievable signal-to-noise ratio. In the TS-ADC, this limitation is overcome by time-stretching the analog signal, which effectively slows down the signal in time prior to digitization. By doing so, the bandwidth (and carrier frequency) of the signal is compressed. Electronic ADCs that would have been too slow to digitize the original signal, can now be used to capture this slowed down signal.

137

Time-stretch analog-to-digital converter

138

Operation principle The basic operating principle of the TS-ADC is shown in Fig. 1. The time-stretch processor, which is generally an optical frontend, stretches the signal in time. It also divides the signal into multiple segments using a filter, for example a wavelength division multiplexing (WDM) filter, to ensure that the stretched replica of the original analog signal segments do not overlap each other in time after stretching. The time-stretched and slowed down signal segments are then converted into digital samples by slow electronic ADCs. Finally, these samples are collected by a digital signal processor (DSP) and rearranged in a manner such that output data is the digital representation of the original analog signal. Any distortion added to the signal by the time-stretch preprocessor is also removed by the DSP.

Fig. 1 A time-stretch analog-to-digital converter (with a stretch factor of 4) is shown. The original analog signal is time-stretched and segmented with the help of a time-stretch preprocessor (generally on optical frontend). Slowed down segments are captured by conventional electronic ADCs. The digitized samples are rearranged to obtain the digital representation of the original signal.

An optical front-end is commonly used to accomplish this process of time-stretch, as shown in Fig. 2. An ultrashort optical pulse (typically 100 to 200 femtoseconds long), also called Fig. 2 Optical frontend for a time-stretch analog-to-digital converter is shown. The a supercontinuum pulse, which has a original analog signal is modulated over a chirped optical pulse (obtained by dispersing an ultra-short supercontinuum pulse). Second dispersive medium stretches the optical broad optical bandwidth, is pulse further. At the photodetector (PD) output, stretched replica of original signal is time-stretched by dispersing it in a obtained. highly dispersive medium (such as a dispersion compensating fiber). This process results in (an almost) linear time-to-wavelength mapping in the stretched pulse, because different wavelengths travel at different speeds in the dispersive medium. The obtained pulse is called a chirped pulse as its frequency is changing with time, and it is typically a few nanoseconds long. The analog signal is modulated onto this chirped pulse using an electro-optic intensity modulator. Subsequently, the modulated pulse is stretched further in the second dispersive medium which has much higher dispersion value. Finally, this obtained optical pulse is converted to electrical domain by a photodetector, giving the stretched replica of the original analog signal. For continuous operation, a train of supercontinuum pulses is used. The chirped pulses arriving at the electro-optic modulator should be wide enough (in time) such that the trailing edge of one pulse overlaps the leading edge of the next pulse. For segmentation, optical filters separate the signal into multiple wavelength channels at the output of the second dispersive medium. For each channel, a separate photodetector and backend electronic ADC is used. Finally the output of these ADCs are passed on to the DSP which generates the desired digital output.

Time-stretch analog-to-digital converter

Impulse response of the photonic time-stretch (PTS) system The PTS processor is based on specialized analog optical (or microwave photonic) fiber links such as those used in cable TV distribution. While the dispersion of fiber is a nuisance in conventional analog optical links, time-stretch technique exploits it to slow down the electrical waveform in the optical domain. In the cable TV link, the light source is a continuous-wave (CW) laser. In PTS, the source is a chirped pulse laser. In a conventional analog optical link, dispersion causes the upper and lower modulation sidebands, foptical ± felectrical, to slip in relative phase. At certain frequencies, their beats with the optical carrier interfere destructively, creating nulls in the frequency response of the system. For practical systems the first null is at tens of GHz, which is sufficient for handling most electrical signals of interest. Although it may seem that the dispersion penalty places a fundamental limit on the impulse response (or the bandwidth) of Fig. 4 Capture of a 95-GHz RF tone using the photonic time-stretch digitizer. The signal the time-stretch system, it can be is captured at an effective sample rate of 10-Terasamples-per-second. eliminated. The dispersion penalty vanishes with single-sideband modulation. Alternatively, one can use the modulator’s secondary (inverse) output port to eliminate the dispersion penalty, in much the same way as two antennas can eliminate spatial nulls in wireless communication (hence the two antennas on top of a WiFi access point). This configuration is termed phase-diversity. For illustration, two calculated complementary transfer functions from a typical phase-diverse time-stretch configuration are plotted in Fig. 4.[5] Combining the complementary outputs using a maximal ratio combining (MRC) algorithm results in a transfer function with a flat response in the frequency domain. Thus, the impulse response (bandwidth) of a time-stretch system is limited only by the bandwidth of the electro-optic modulator, which is about 120 GHz—a value that is adequate for capturing most electrical waveforms of interest. Extremely large stretch factors can be obtained using long lengths of fiber, but at the cost of larger loss—a problem that has been overcome by employing Raman amplification within the dispersive fiber itself, leading to the world’s fastest real-time digitizer,[6] as shown in Fig. 3. Also, using PTS, capture of very high frequency signals with a world record resolution in 10-GHz bandwidth range has been achieved.[7]

Comparison with time lens imaging Another technique, temporal imaging using a time lens, can also be used to slow down (mostly optical) signals in time. The time-lens concept relies on the mathematical equivalence between spatial diffraction and temporal dispersion, the so-called space-time duality.[8] A lens held at fixed distance from an object produces a magnified visible image. The lens imparts a quadratic phase shift to the spatial frequency components of the optical waves; in conjunction with the free space propagation (object to lens, lens to eye), this generates a magnified image. Owing to the mathematical equivalence between paraxial diffraction and temporal dispersion, an optical waveform can be temporally imaged by a three-step process of dispersing it in time, subjecting it to a phase shift that is quadratic in time (the time lens itself), and dispersing it again. Theoretically, a focused aberration-free image is obtained under a specific condition when the two dispersive elements and the phase shift satisfy the temporal equivalent of the classic

139

Time-stretch analog-to-digital converter lens equation. Alternatively, the time lens can be used without the second dispersive element to transfer the waveform’s temporal profile to the spectral domain, analogous to the property that an ordinary lens produces the spatial Fourier transform of an object at its focal points.[9] In contrast to the time-lens approach, PTS is not based on the space-time duality – there is no lens equation that needs to be satisfied to obtain an error-free slowed-down version of the input waveform. Time-stretch technique also offers continuous-time acquisition performance, a feature needed for mainstream applications of oscilloscopes. Another important difference between the two techniques is that the time lens requires the input signal to be subjected to high amount of dispersion before further processing. For electrical waveforms, the electronic devices that have the required characteristics: (1) high dispersion to loss ratio, (2) uniform dispersion, and (3) broad bandwidths, do not exist. This renders time lens not suitable for slowing down wideband electrical waveforms. In contrast, PTS does not have such a requirement. It was developed specifically for slowing down electrical waveforms and enable high speed digitizers.

Application to imaging and spectroscopy In addition to wideband A/D conversion, photonic time-stretch (PTS) is also an enabling technology for high-throughput real-time instrumentation such as imaging [10] and spectroscopy.[11][12] The world's fastest optical imaging method called serial time-encoded amplified microscopy (STEAM) makes use of the PTS technology to acquire image using a single-pixel photodetector and commercial ADC. Wavelength-time spectroscopy, which also relies on photonic time-stretch technique, permits real-time single-shot measurements of rapidly evolving or ﬂuctuating spectra.

References [1] A. S. Bhushan, F. Coppinger, and B. Jalali, “Time-stretched analogue-to-digital conversion," Electronics Letters vol. 34, no. 9, pp. 839–841, April 1998. (http:/ / ieeexplore. ieee. org/ xpls/ abs_all. jsp?arnumber=682797) [2] A. Fard, S. Gupta, and B. Jalali, "Photonic time-stretch digitizer and its extension to real-time spectroscopy and imaging," Laser & Photonics Reviews vol. 7, no. 2, pp. 207-263, March 2013. (http:/ / onlinelibrary. wiley. com/ doi/ 10. 1002/ lpor. 201200015/ abstract) [3] Y. Han and B. Jalali, “Photonic Time-Stretched Analog-to-Digital Converter: Fundamental Concepts and Practical Considerations," Journal of Lightwave Technology, Vol. 21, Issue 12, pp. 3085–3103, Dec. 2003. (http:/ / www. opticsinfobase. org/ abstract. cfm?& uri=JLT-21-12-3085) [4] J. Capmany and D. Novak, “Microwave photonics combines two worlds," Nature Photonics 1, 319-330 (2007). (http:/ / www. nature. com/ nphoton/ journal/ v1/ n6/ abs/ nphoton. 2007. 89. html) [5] Yan Han, Ozdal Boyraz, Bahram Jalali, "Ultrawide-Band Photonic Time-Stretch A/D Converter Employing Phase Diversity," "IEEE TRANSACTIONS ON MICROWAVE THEORY AND TECHNIQUES" VOL. 53, NO. 4, APRIL 2005 (http:/ / ieeexplore. ieee. org/ xpls/ abs_all. jsp?arnumber=1420773) [6] J. Chou, O. Boyraz, D. Solli, and B. Jalali, “Femtosecond real-time single-shot digitizer," Applied Physics Letters 91, 161105 (2007). (http:/ / scitation. aip. org/ getabs/ servlet/ GetabsServlet?prog=normal& id=APPLAB000091000016161105000001& idtype=cvips& gifs=yes) [7] S. Gupta and B. Jalali, “Time-warp correction and calibration in photonic time-stretch analog-to-digital converter," Optics Letters 33, 2674–2676 (2008). (http:/ / www. opticsinfobase. org/ abstract. cfm?uri=ol-33-22-2674) [8] B. H. Kolner and M. Nazarathy, “Temporal imaging with a time lens," Optics Letters 14, 630-632 (1989) (http:/ / www. opticsinfobase. org/ ol/ abstract. cfm?URI=ol-14-12-630) [9] J. W. Goodman, “Introduction to Fourier Optics," McGraw-Hill (1968). [10] K. Goda, K.K. Tsia, and B. Jalali, "Serial time-encoded amplified imaging for real-time observation of fast dynamic phenomena," Nature 458, 1145–1149, 2009. (http:/ / www. nature. com/ nature/ journal/ v458/ n7242/ full/ nature07980. html) [11] D. R. Solli, J. Chou, and B. Jalali, "Amplified wavelength–time transformation for real-time spectroscopy," Nature Photonics 2, 48-51, 2008. (http:/ / www. nature. com/ nphoton/ journal/ v2/ n1/ full/ nphoton. 2007. 253. html) [12] J. Chou, D. Solli, and B. Jalali, "Real-time spectroscopy with subgigahertz resolution using amplified dispersive Fourier transformation," Applied Physics Letters 92, 111102, 2008. (http:/ / apl. aip. org/ resource/ 1/ applab/ v92/ i11/ p111102_s1)

140

Time-stretch analog-to-digital converter

Other resources • G. C. Valley, “Photonic analog-to-digital converters," Opt. Express, vol. 15, no. 5, pp. 1955–1982, March 2007. (http://www.opticsinfobase.org/oe/abstract.cfm?uri=oe-15-5-1955) • Photonic Bandwidth Compression for Instantaneous Wideband A/D Conversion (PHOBIAC) project. (http:// www.darpa.mil/MTO/Programs/phobiac/index.html) • Short time Fourier transform for time-frequency analysis of ultrawideband signals (http://www.researchgate. net/publication/3091384_Time-stretched_short-time_Fourier_transform/)

141

142

Fourier Transforms, Discrete and Fast Discrete Fourier transform Fourier transforms Continuous Fourier transform Fourier series Discrete-time Fourier transform Discrete Fourier transform Fourier analysis Related transforms

In mathematics, the discrete Fourier transform (DFT) converts a finite list of equally spaced samples of a function into the list of coefficients of a finite combination of complex sinusoids, ordered by their frequencies, that has those same sample values. It can be said to convert the sampled function from its original domain (often time or position along a line) to the frequency domain.

Relationship between the (continuous) Fourier transform and the discrete Fourier transform. Left column: A continuous function (top) and its Fourier transform (bottom). Center-left column: Periodic summation of the original function (top). Fourier transform (bottom) is zero except at discrete points. The inverse transform is a sum of sinusoids called Fourier series. Center-right column: Original function is discretized (multiplied by a Dirac comb) (top). Its Fourier transform (bottom) is a periodic summation (DTFT) of the original transform. Right column: The DFT (bottom) computes discrete samples of the continuous DTFT. The inverse DFT (top) is a periodic summation of the original samples. The FFT algorithm computes one cycle of the DFT and its inverse is one cycle of the DFT inverse.

The input samples are complex numbers (in practice, usually real numbers), and the output coefficients are complex as well. The frequencies of the output sinusoids are integer multiples of a fundamental frequency, whose corresponding period is the length of the sampling interval. The combination of sinusoids obtained through the DFT is therefore periodic with that same period. The DFT differs from the discrete-time Fourier transform (DTFT) in that its input and output sequences are both finite; it is therefore said to be the Fourier analysis of finite-domain (or periodic) discrete-time functions.

Discrete Fourier transform

143

The DFT is the most important discrete transform, used to perform Fourier analysis in many practical applications. In digital signal processing, the function is any quantity or signal that varies over time, such as the pressure of a sound wave, a radio signal, or daily temperature readings, sampled over a finite time interval (often defined by a window function). In image processing, the samples can be the values of pixels along a row or column of a raster image. The DFT is also used to efficiently solve partial differential equations, and to perform other operations such as convolutions or multiplying large integers. Illustration of using Dirac comb functions and the convolution theorem to model the Since it deals with a finite amount of effects of sampling and/or periodic summation. At lower left is a DTFT, the spectral data, it can be implemented in result of sampling s(t) at intervals of T. The spectral sequences at (a) upper right and (b) computers by numerical algorithms or lower right are respectively computed from (a) one cycle of the periodic summation of even dedicated hardware. These s(t) and (b) one cycle of the periodic summation of the s(nT) sequence. The respective formulas are (a) the Fourier series integral and (b) the DFT summation. Its similarities to implementations usually employ the original transform, S(f), and its relative computational ease are often the motivation efficient fast Fourier transform (FFT) for computing a DFT sequence. algorithms;[1] so much so that the terms "FFT" and "DFT" are often used interchangeably. Prior to its current usage, the "FFT" initialism may have also been used for the ambiguous term finite Fourier transform.

Definition The sequence of N complex numbers

is transformed into an N-periodic sequence of complex

numbers:

(integers)

Each

[2]

(Eq.1)

is a complex number that encodes both amplitude and phase of a sinusoidal component of function

.

The sinusoid's frequency is k/N cycles per sample. Its amplitude and phase are:

where atan2 is the two-argument form of the arctan function. Due to periodicity (see Periodicity), the customary domain of k actually computed is [0, N-1]. That is always the case when the DFT is implemented via the Fast Fourier transform algorithm. But other common domains are [-N/2, N/2-1] (N even) and [-(N-1)/2, (N-1)/2] (N odd), as when the left and right halves of an FFT output sequence are swapped. The transform is sometimes denoted by the symbol

, as in

Eq.1 can be interpreted or derived in various ways, for example:

or

or

.[3]

Discrete Fourier transform

144

• It completely describes the discrete-time Fourier transform (DTFT) of an N-periodic sequence, which comprises only discrete frequency components. (Discrete-time Fourier transform#Periodic data) • It can also provide uniformly spaced samples of the continuous DTFT of a finite length sequence. (Sampling the DTFT) • It is the cross correlation of the input sequence, xn, and a complex sinusoid at frequency k/N. Thus it acts like a matched filter for that frequency. • It is the discrete analogy of the formula for the coefficients of a Fourier series: (Eq.2)

which is also N-periodic. In the domain

this is the inverse transform of Eq.1.

The normalization factor multiplying the DFT and IDFT (here 1 and 1/N) and the signs of the exponents are merely conventions, and differ in some treatments. The only requirements of these conventions are that the DFT and IDFT have opposite-sign exponents and that the product of their normalization factors be 1/N. A normalization of for both the DFT and IDFT, for instance, makes the transforms unitary. In the following discussion the terms "sequence" and "vector" will be considered interchangeable.

Properties Completeness The discrete Fourier transform is an invertible, linear transformation

with

denoting the set of complex numbers. In other words, for any N > 0, an N-dimensional complex vector has a

DFT and an IDFT which are in turn N-dimensional complex vectors.

Orthogonality The vectors

form an orthogonal basis over the set of N-dimensional

complex vectors:

where

is the Kronecker delta. (In the last step, the summation is trivial if

, where it is 1+1+⋅⋅⋅=N, and

otherwise is a geometric series that can be explicitly summed to obtain zero.) This orthogonality condition can be used to derive the formula for the IDFT from the definition of the DFT, and is equivalent to the unitarity property below.

Discrete Fourier transform

145

The Plancherel theorem and Parseval's theorem If Xk and Yk are the DFTs of xn and yn respectively then the Plancherel theorem states:

where the star denotes complex conjugation. Parseval's theorem is a special case of the Plancherel theorem and states:

These theorems are also equivalent to the unitary condition below.

Periodicity The periodicity can be shown directly from the definition:

Similarly, it can be shown that the IDFT formula leads to a periodic extension.

Shift theorem Multiplying

by a linear phase

is replaced by shift of the input

for some integer m corresponds to a circular shift of the output

:

, where the subscript is interpreted modulo N (i.e., periodically). Similarly, a circular corresponds to multiplying the output

by a linear phase. Mathematically, if

represents the vector x then if then and

Circular convolution theorem and cross-correlation theorem The convolution theorem for the discrete-time Fourier transform indicates that a convolution of two infinite sequences can be obtained as the inverse transform of the product of the individual transforms. An important simplification occurs when the sequences are of finite length, N. In terms of the DFT and inverse DFT, it can be written as follows:

which is the convolution of the

sequence with a

Similarly, the cross-correlation of

and

sequence extended by periodic summation:

is given by:

When either sequence contains a string of zeros, of length L, L+1 of the circular convolution outputs are equivalent to values of Methods have also been developed to use this property as part of an efficient process that constructs

with an

or

sequence potentially much longer than the practical transform size (N). Two

Discrete Fourier transform

146

such methods are called overlap-save and overlap-add.[4] The efficiency results from the fact that a direct evaluation of either summation (above) requires operations for an output sequence of length N. An indirect method, using transforms, can take advantage of the

efficiency of the fast Fourier transform (FFT) to achieve much better

performance. Furthermore, convolutions can be used to efficiently compute DFTs via Rader's FFT algorithm and Bluestein's FFT algorithm.

Convolution theorem duality It can also be shown that:

which is the circular convolution of

and

.

Trigonometric interpolation polynomial The trigonometric interpolation polynomial for N even , for N odd, where the coefficients Xk are given by the DFT of xn above, satisfies the interpolation property

for

. For even N, notice that the Nyquist component

is handled specially.

This interpolation is not unique: aliasing implies that one could add N to any of the complex-sinusoid frequencies (e.g. changing to ) without changing the interpolation property, but giving different values in between the points. The choice above, however, is typical because it has two useful properties. First, it consists of sinusoids whose frequencies have the smallest possible magnitudes: the interpolation is bandlimited. Second, if the are real numbers, then is real as well. In contrast, the most obvious trigonometric interpolation polynomial is the one in which the frequencies range from 0 to (instead of roughly to as above), similar to the inverse DFT formula. This interpolation does not minimize the slope, and is not generally real-valued for real

; its use is a common mistake.

The unitary DFT Another way of looking at the DFT is to note that in the above discussion, the DFT can be expressed as a Vandermonde matrix:

where

is a primitive Nth root of unity. The inverse transform is then given by the inverse of the above matrix:

Discrete Fourier transform

147

With unitary normalization constants

, the DFT becomes a unitary transformation, defined by a unitary

matrix:

where det() is the determinant function. The determinant is the product of the eigenvalues, which are always

or

as described below. In a real vector space, a unitary transformation can be thought of as simply a rigid rotation of the coordinate system, and all of the properties of a rigid rotation can be found in the unitary DFT. The orthogonality of the DFT is now expressed as an orthonormality condition (which arises in many areas of mathematics as described in root of unity):

If

is defined as the unitary DFT of the vector

then

and the Plancherel theorem is expressed as:

If we view the DFT as just a coordinate transformation which simply specifies the components of a vector in a new coordinate system, then the above is just the statement that the dot product of two vectors is preserved under a unitary DFT transformation. For the special case , this implies that the length of a vector is preserved as well—this is just Parseval's theorem:

A consequence of the circular convolution theorem is that the DFT matrix

diagonalizes any circulant matrix.

Expressing the inverse DFT in terms of the DFT A useful property of the DFT is that the inverse DFT can be easily expressed in terms of the (forward) DFT, via several well-known "tricks". (For example, in computations, it is often convenient to only implement a fast Fourier transform corresponding to one transform direction and then to get the other transform direction from the first.) First, we can compute the inverse DFT by reversing the inputs (Duhamel et al., 1988):

(As usual, the subscripts are interpreted modulo N; thus, for

, we have

.)

Second, one can also conjugate the inputs and outputs:

Third, a variant of this conjugation trick, which is sometimes preferable because it requires no modification of the data values, involves swapping real and imaginary parts (which can be done on a computer simply by modifying pointers). Define swap( ) as with its real and imaginary parts swapped—that is, if then swap(

) is

. Equivalently, swap(

) equals

. Then

Discrete Fourier transform

148

That is, the inverse transform is the same as the forward transform with the real and imaginary parts swapped for both input and output, up to a normalization (Duhamel et al., 1988). The conjugation trick can also be used to define a new transform, closely related to the DFT, that is involutary—that is, which is its own inverse. In particular, is clearly its own inverse: .A closely related involutary transformation (by a factor of (1+i) /√2) is factors in

cancel the 2. For real inputs

, the real part of

, since the is none other than the

discrete Hartley transform, which is also involutary.

Eigenvalues and eigenvectors The eigenvalues of the DFT matrix are simple and well-known, whereas the eigenvectors are complicated, not unique, and are the subject of ongoing research. Consider the unitary form

defined above for the DFT of length N, where

This matrix satisfies the matrix polynomial equation:

This can be seen from the inverse properties above: operating operating

twice gives the original data in reverse order, so

four times gives back the original data and is thus the identity matrix. This means that the eigenvalues

satisfy the equation:

Therefore, the eigenvalues of

are the fourth roots of unity:

Since there are only four distinct eigenvalues for this

is +1, −1, +i, or −i. matrix, they have some multiplicity. The multiplicity

gives the number of linearly independent eigenvectors corresponding to each eigenvalue. (Note that there are N independent eigenvectors; a unitary matrix is never defective.) The problem of their multiplicity was solved by McClellan and Parks (1972), although it was later shown to have been equivalent to a problem solved by Gauss (Dickinson and Steiglitz, 1982). The multiplicity depends on the value of N modulo 4, and is given by the following table:

Multiplicities of the eigenvalues λ of the unitary DFT matrix U as a function of the transform size N (in terms of an integer m). size N λ = +1 λ = −1 λ = -i λ = +i 4m

m+1

m

m

m−1

4m + 1 m + 1

m

m

m

4m + 2 m + 1

m+1

m

m

4m + 3 m + 1

m+1

m+1 m

Otherwise stated, the characteristic polynomial of

is:

No simple analytical formula for general eigenvectors is known. Moreover, the eigenvectors are not unique because any linear combination of eigenvectors for the same eigenvalue is also an eigenvector for that eigenvalue. Various researchers have proposed different choices of eigenvectors, selected to satisfy useful properties like orthogonality and to have "simple" forms (e.g., McClellan and Parks, 1972; Dickinson and Steiglitz, 1982; Grünbaum, 1982; Atakishiyev and Wolf, 1997; Candan et al., 2000; Hanna et al., 2004; Gurevich and Hadani, 2008).

Discrete Fourier transform

149

A straightforward approach is to discretize an eigenfunction of the continuous Fourier transform, of which the most famous is the Gaussian function. Since periodic summation of the function means discretizing its frequency spectrum and discretization means periodic summation of the spectrum, the discretized and periodically summed Gaussian function yields an eigenvector of the discrete transform: •

. A closed form expression for the series is not known, but it converges rapidly.

Two other simple closed-form analytical eigenvectors for special DFT period N were found (Kong, 2008): For DFT period N = 2L + 1 = 4K +1, where K is an integer, the following is an eigenvector of DFT: • For DFT period N = 2L = 4K, where K is an integer, the following is an eigenvector of DFT: • The choice of eigenvectors of the DFT matrix has become important in recent years in order to define a discrete analogue of the fractional Fourier transform—the DFT matrix can be taken to fractional powers by exponentiating the eigenvalues (e.g., Rubio and Santhanam, 2005). For the continuous Fourier transform, the natural orthogonal eigenfunctions are the Hermite functions, so various discrete analogues of these have been employed as the eigenvectors of the DFT, such as the Kravchuk polynomials (Atakishiyev and Wolf, 1997). The "best" choice of eigenvectors to define a fractional discrete Fourier transform remains an open question, however.

Uncertainty principle If the random variable

then

is constrained by:

may be considered to represent a discrete probability mass function of n, with an associated

probability mass function constructed from the transformed variable: For the case of continuous functions P(x) and Q(k), the Heisenberg uncertainty principle states that:

where

and

are the variances of

and

respectively, with the equality attained in the case

of a suitably normalized Gaussian distribution. Although the variances may be analogously defined for the DFT, an analogous uncertainty principle is not useful, because the uncertainty will not be shift-invariant. Nevertheless, a meaningful uncertainty principle has been introduced by Massar and Spindel. However, the Hirschman uncertainty will have a useful analog for the case of the DFT. The Hirschman uncertainty principle is expressed in terms of the Shannon entropy of the two probability functions. In the discrete case, the Shannon entropies are defined as:

and

Discrete Fourier transform

150

and the Hirschman uncertainty principle becomes:

The equality is obtained for

equal to translations and modulations of a suitably normalized Kronecker comb of

period A where A is any exact integer divisor of N. The probability mass function

will then be proportional to a

suitably translated Kronecker comb of period B=N/A.

The real-input DFT If

are real numbers, as they often are in practical applications, then the DFT obeys the symmetry: where

denotes complex conjugation.

It follows that X0 and XN/2 are real-valued, and the remainder of the DFT is completely specified by just N/2-1 complex numbers.

Generalized DFT (shifted and non-linear phase) It is possible to shift the transform sampling in time and/or frequency domain by some real shifts a and b, respectively. This is sometimes known as a generalized DFT (or GDFT), also called the shifted DFT or offset DFT, and has analogous properties to the ordinary DFT:

Most often, shifts of

(half a sample) are used. While the ordinary DFT corresponds to a periodic signal in both

time and frequency domains,

produces a signal that is anti-periodic in frequency domain (

) and vice-versa for

. Thus, the specific case of

is known as an odd-time

2

odd-frequency discrete Fourier transform (or O DFT). Such shifted transforms are most often used for symmetric data, to represent different boundary symmetries, and for real-symmetric data they correspond to different forms of the discrete cosine and sine transforms. Another interesting choice is , which is called the centered DFT (or CDFT). The centered DFT has the useful property that, when N is a multiple of four, all four of its eigenvalues (see above) have equal multiplicities (Rubio and Santhanam, 2005)[5] The term GDFT is also used for the non-linear phase extensions of DFT. Hence, GDFT method provides a generalization for constant amplitude orthogonal block transforms including linear and non-linear phase types. GDFT is a framework to improve time and frequency domain properties of the traditional DFT, e.g. auto/cross-correlations, by the addition of the properly designed phase shaping function (non-linear, in general) to the original linear phase functions (Akansu and Agirman-Tosun, 2010).[6] The discrete Fourier transform can be viewed as a special case of the z-transform, evaluated on the unit circle in the complex plane; more general z-transforms correspond to complex shifts a and b above.

Multidimensional DFT The ordinary DFT transforms a one-dimensional sequence or array variable n. The multidimensional DFT of a multidimensional array variables

for

where more

in

that is a function of exactly one discrete that is a function of d discrete

is defined by:

as above and the d output indices run from compactly

expressed

in

vector

notation,

where

we

define

. This is and

Discrete Fourier transform

151 as d-dimensional vectors of indices from 0 to

where the division

is defined as

, which we define as

to be performed element-wise, and the

sum denotes the set of nested summations above. The inverse of the multi-dimensional DFT is, analogous to the one-dimensional case, given by:

As the one-dimensional DFT expresses the input as a superposition of sinusoids, the multidimensional DFT expresses the input as a superposition of plane waves, or multidimensional sinusoids. The direction of oscillation in space is . The amplitudes are . This decomposition is of great importance for everything from digital image processing (two-dimensional) to solving partial differential equations. The solution is broken up into plane waves. The multidimensional DFT can be computed by the composition of a sequence of one-dimensional DFTs along each dimension. In the two-dimensional case the independent DFTs of the rows (i.e., along ) are computed first to form a new array

. Then the

independent DFTs of y along the columns (along

) are

computed to form the final result

. Alternatively the columns can be computed first and then the rows. The

order is immaterial because the nested summations above commute. An algorithm to compute a one-dimensional DFT is thus sufficient to efficiently compute a multidimensional DFT. This approach is known as the row-column algorithm. There are also intrinsically multidimensional FFT algorithms.

The real-input multidimensional DFT For input data

consisting of real numbers, the DFT outputs have a conjugate symmetry similar to the

one-dimensional case above:

where the star again denotes complex conjugation and the

-th subscript is again interpreted modulo

(for

).

Applications The DFT has seen wide usage across a large number of fields; we only sketch a few examples below (see also the references at the end). All applications of the DFT depend crucially on the availability of a fast algorithm to compute discrete Fourier transforms and their inverses, a fast Fourier transform.

Spectral analysis When the DFT is used for spectral analysis, the time-samples of some signal

sequence usually represents a finite set of uniformly spaced

, where t represents time. The conversion from continuous time to samples

(discrete-time) changes the underlying Fourier transform of x(t) into a discrete-time Fourier transform (DTFT), which generally entails a type of distortion called aliasing. Choice of an appropriate sample-rate (see Nyquist rate) is the key to minimizing that distortion. Similarly, the conversion from a very long (or infinite) sequence to a manageable size entails a type of distortion called leakage, which is manifested as a loss of detail (aka resolution) in the DTFT. Choice of an appropriate sub-sequence length is the primary key to minimizing that effect. When the available data (and time to process it) is more than the amount needed to attain the desired frequency resolution, a standard technique is to perform multiple DFTs, for example to create a spectrogram. If the desired result is a power spectrum and noise or randomness is present in the data, averaging the magnitude components of the multiple DFTs

Discrete Fourier transform is a useful procedure to reduce the variance of the spectrum (also called a periodogram in this context); two examples of such techniques are the Welch method and the Bartlett method; the general subject of estimating the power spectrum of a noisy signal is called spectral estimation. A final source of distortion (or perhaps illusion) is the DFT itself, because it is just a discrete sampling of the DTFT, which is a function of a continuous frequency domain. That can be mitigated by increasing the resolution of the DFT. That procedure is illustrated at Sampling the DTFT. • The procedure is sometimes referred to as zero-padding, which is a particular implementation used in conjunction with the fast Fourier transform (FFT) algorithm. The inefficiency of performing multiplications and additions with zero-valued "samples" is more than offset by the inherent efficiency of the FFT. • As already noted, leakage imposes a limit on the inherent resolution of the DTFT. So there is a practical limit to the benefit that can be obtained from a fine-grained DFT.

Filter bank See FFT filter banks and Sampling the DTFT.

Data compression The field of digital signal processing relies heavily on operations in the frequency domain (i.e. on the Fourier transform). For example, several lossy image and sound compression methods employ the discrete Fourier transform: the signal is cut into short segments, each is transformed, and then the Fourier coefficients of high frequencies, which are assumed to be unnoticeable, are discarded. The decompressor computes the inverse transform based on this reduced number of Fourier coefficients. (Compression applications often use a specialized form of the DFT, the discrete cosine transform or sometimes the modified discrete cosine transform.) Some relatively recent compression algorithms, however, use wavelet transforms, which give a more uniform compromise between time and frequency domain than obtained by chopping data into segments and transforming each segment. In the case of JPEG2000, this avoids the spurious image features that appear when images are highly compressed with the original JPEG.

Partial differential equations Discrete Fourier transforms are often used to solve partial differential equations, where again the DFT is used as an approximation for the Fourier series (which is recovered in the limit of infinite N). The advantage of this approach is that it expands the signal in complex exponentials einx, which are eigenfunctions of differentiation: d/dx einx = in einx. Thus, in the Fourier representation, differentiation is simple—we just multiply by i n. (Note, however, that the choice of n is not unique due to aliasing; for the method to be convergent, a choice similar to that in the trigonometric interpolation section above should be used.) A linear differential equation with constant coefficients is transformed into an easily solvable algebraic equation. One then uses the inverse DFT to transform the result back into the ordinary spatial representation. Such an approach is called a spectral method.

152

Discrete Fourier transform

153

Polynomial multiplication Suppose we wish to compute the polynomial product c(x) = a(x) · b(x). The ordinary product expression for the coefficients of c involves a linear (acyclic) convolution, where indices do not "wrap around." This can be rewritten as a cyclic convolution by taking the coefficient vectors for a(x) and b(x) with constant term first, then appending zeros so that the resultant coefficient vectors a and b have dimension d > deg(a(x)) + deg(b(x)). Then,

Where c is the vector of coefficients for c(x), and the convolution operator

is defined so

But convolution becomes multiplication under the DFT:

Here the vector product is taken elementwise. Thus the coefficients of the product polynomial c(x) are just the terms 0, ..., deg(a(x)) + deg(b(x)) of the coefficient vector

With a fast Fourier transform, the resulting algorithm takes O (N log N) arithmetic operations. Due to its simplicity and speed, the Cooley–Tukey FFT algorithm, which is limited to composite sizes, is often chosen for the transform operation. In this case, d should be chosen as the smallest integer greater than the sum of the input polynomial degrees that is factorizable into small prime factors (e.g. 2, 3, and 5, depending upon the FFT implementation). Multiplication of large integers The fastest known algorithms for the multiplication of very large integers use the polynomial multiplication method outlined above. Integers can be treated as the value of a polynomial evaluated specifically at the number base, with the coefficients of the polynomial corresponding to the digits in that base. After polynomial multiplication, a relatively low-complexity carry-propagation step completes the multiplication. Convolution When data is convolved with a function with wide support, such as for downsampling by a large sampling ratio, because of the Convolution theorem and the FFT algorithm, it may be faster to transform it, multiply pointwise by the transform of the filter and then reverse transform it. Alternatively, a good filter is obtained by simply truncating the transformed data and re-transforming the shortened data set.

Some discrete Fourier transform pairs Some DFT pairs Note

Shift theorem

Real DFT from the geometric progression formula

from the binomial theorem

Discrete Fourier transform

154 is a rectangular window function of W points centered on n=0, where W is an odd integer, and is a sinc-like function (specifically,

is a Dirichlet

kernel) Discretization and periodic summation of the scaled Gaussian functions for

. Since either

or

is larger than one and thus warrants fast convergence of one of the two series, for large you may choose to compute the frequency spectrum and convert to the time domain using the discrete Fourier transform.

Generalizations Representation theory For more details on this topic, see Representation theory of finite groups § Discrete Fourier transform. The DFT can be interpreted as the complex-valued representation theory of the finite cyclic group. In other words, a sequence of n complex numbers can be thought of as an element of n-dimensional complex space Cn or equivalently a function f from the finite cyclic group of order n to the complex numbers, Zn → C. So f is a class function on the finite cyclic group, and thus can be expressed as a linear combination of the irreducible characters of this group, which are the roots of unity. From this point of view, one may generalize the DFT to representation theory generally, or more narrowly to the representation theory of finite groups. More narrowly still, one may generalize the DFT by either changing the target (taking values in a field other than the complex numbers), or the domain (a group other than a finite cyclic group), as detailed in the sequel.

Other fields Main articles: Discrete Fourier transform (general) and Number-theoretic transform Many of the properties of the DFT only depend on the fact that denoted

or

(so that

is a primitive root of unity, sometimes

). Such properties include the completeness, orthogonality,

Plancherel/Parseval, periodicity, shift, convolution, and unitarity properties above, as well as many FFT algorithms. For this reason, the discrete Fourier transform can be defined by using roots of unity in fields other than the complex numbers, and such generalizations are commonly called number-theoretic transforms (NTTs) in the case of finite fields. For more information, see number-theoretic transform and discrete Fourier transform (general).

Other finite groups Main article: Fourier transform on finite groups The standard DFT acts on a sequence x0, x1, …, xN−1 of complex numbers, which can be viewed as a function {0, 1, …, N − 1} → C. The multidimensional DFT acts on multidimensional sequences, which can be viewed as functions

This suggests the generalization to Fourier transforms on arbitrary finite groups, which act on functions G → C where G is a finite group. In this framework, the standard DFT is seen as the Fourier transform on a cyclic group, while the multidimensional DFT is a Fourier transform on a direct sum of cyclic groups.

Discrete Fourier transform

Alternatives Main article: Discrete wavelet transform For more details on this topic, see Discrete wavelet transform § Comparison with Fourier transform. There are various alternatives to the DFT for various applications, prominent among which are wavelets. The analog of the DFT is the discrete wavelet transform (DWT). From the point of view of time–frequency analysis, a key limitation of the Fourier transform is that it does not include location information, only frequency information, and thus has difficulty in representing transients. As wavelets have location as well as frequency, they are better able to represent location, at the expense of greater difficulty representing frequency. For details, see comparison of the discrete wavelet transform with the discrete Fourier transform.

Notes [1] Cooley et al., 1969 [2] In this context, it is common to define UNIQ-math-0-fdcc463dc40e329d-QINU to be the Nth primitive root of unity, UNIQ-math-1-fdcc463dc40e329d-QINU , to obtain the following form:

[3] As a linear transformation on a finite-dimensional vector space, the DFT expression can also be written in terms of a DFT matrix; when scaled appropriately it becomes a unitary matrix and the Xk can thus be viewed as coefficients of x in an orthonormal basis. [4] T. G. Stockham, Jr., " High-speed convolution and correlation (http:/ / dl. acm. org/ citation. cfm?id=1464209)," in 1966 Proc. AFIPS Spring Joint Computing Conf. Reprinted in Digital Signal Processing, L. R. Rabiner and C. M. Rader, editors, New York: IEEE Press, 1972. [5] Santhanam, Balu; Santhanam, Thalanayar S. "Discrete Gauss-Hermite functions and eigenvectors of the centered discrete Fourier transform" (http:/ / thamakau. usc. edu/ Proceedings/ ICASSP 2007/ pdfs/ 0301385. pdf), Proceedings of the 32nd IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2007, SPTM-P12.4), vol. III, pp. 1385-1388. [6] Akansu, Ali N.; Agirman-Tosun, Handan "Generalized Discrete Fourier Transform With Nonlinear Phase" (http:/ / web. njit. edu/ ~akansu/ PAPERS/ AkansuIEEE-TSP2010. pdf), IEEE Transactions on Signal Processing, vol. 58, no. 9, pp. 4547-4556, Sept. 2010.

Citations References • Brigham, E. Oran (1988). The fast Fourier transform and its applications. Englewood Cliffs, N.J.: Prentice Hall. ISBN 0-13-307505-2. • Oppenheim, Alan V.; Schafer, R. W.; and Buck, J. R. (1999). Discrete-time signal processing. Upper Saddle River, N.J.: Prentice Hall. ISBN 0-13-754920-2. • Smith, Steven W. (1999). "Chapter 8: The Discrete Fourier Transform" (http://www.dspguide.com/ch8/1. htm). The Scientist and Engineer's Guide to Digital Signal Processing (Second ed.). San Diego, Calif.: California Technical Publishing. ISBN 0-9660176-3-3. • Cormen, Thomas H.; Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein (2001). "Chapter 30: Polynomials and the FFT". Introduction to Algorithms (Second ed.). MIT Press and McGraw-Hill. pp. 822–848. ISBN 0-262-03293-7. esp. section 30.2: The DFT and FFT, pp. 830–838. • P. Duhamel, B. Piron, and J. M. Etcheto (1988). "On computing the inverse DFT". IEEE Trans. Acoust., Speech and Sig. Processing 36 (2): 285–286. doi: 10.1109/29.1519 (http://dx.doi.org/10.1109/29.1519). • J. H. McClellan and T. W. Parks (1972). "Eigenvalues and eigenvectors of the discrete Fourier transformation". IEEE Trans. Audio Electroacoust. 20 (1): 66–74. doi: 10.1109/TAU.1972.1162342 (http://dx.doi.org/10.1109/ TAU.1972.1162342). • Bradley W. Dickinson and Kenneth Steiglitz (1982). "Eigenvectors and functions of the discrete Fourier transform". IEEE Trans. Acoust., Speech and Sig. Processing 30 (1): 25–31. doi: 10.1109/TASSP.1982.1163843 (http://dx.doi.org/10.1109/TASSP.1982.1163843). (Note that this paper has an apparent typo in its table of the eigenvalue multiplicities: the +i/−i columns are interchanged. The correct table can be found in McClellan and

155

Discrete Fourier transform

• •

•

•

•

Parks, 1972, and is easily confirmed numerically.) F. A. Grünbaum (1982). "The eigenvectors of the discrete Fourier transform". J. Math. Anal. Appl. 88 (2): 355–363. doi: 10.1016/0022-247X(82)90199-8 (http://dx.doi.org/10.1016/0022-247X(82)90199-8). Natig M. Atakishiyev and Kurt Bernardo Wolf (1997). "Fractional Fourier-Kravchuk transform". J. Opt. Soc. Am. A 14 (7): 1467–1477. Bibcode: 1997JOSAA..14.1467A (http://adsabs.harvard.edu/abs/1997JOSAA..14. 1467A). doi: 10.1364/JOSAA.14.001467 (http://dx.doi.org/10.1364/JOSAA.14.001467). C. Candan, M. A. Kutay and H. M.Ozaktas (2000). "The discrete fractional Fourier transform". IEEE Trans. on Signal Processing 48 (5): 1329–1337. Bibcode: 2000ITSP...48.1329C (http://adsabs.harvard.edu/abs/ 2000ITSP...48.1329C). doi: 10.1109/78.839980 (http://dx.doi.org/10.1109/78.839980). Magdy Tawfik Hanna, Nabila Philip Attalla Seif, and Waleed Abd El Maguid Ahmed (2004). "Hermite-Gaussian-like eigenvectors of the discrete Fourier transform matrix based on the singular-value decomposition of its orthogonal projection matrices". IEEE Trans. Circ. Syst. I 51 (11): 2245–2254. doi: 10.1109/TCSI.2004.836850 (http://dx.doi.org/10.1109/TCSI.2004.836850). Shamgar Gurevich and Ronny Hadani (2009). "On the diagonalization of the discrete Fourier transform". Applied and Computational Harmonic Analysis 27 (1): 87–99. arXiv: 0808.3281 (http://arxiv.org/abs/0808.3281). doi: 10.1016/j.acha.2008.11.003 (http://dx.doi.org/10.1016/j.acha.2008.11.003). preprint at.

• Shamgar Gurevich, Ronny Hadani, and Nir Sochen (2008). "The finite harmonic oscillator and its applications to sequences, communication and radar". IEEE Transactions on Information Theory 54 (9): 4239–4253. arXiv: 0808.1495 (http://arxiv.org/abs/0808.1495). doi: 10.1109/TIT.2008.926440 (http://dx.doi.org/10.1109/ TIT.2008.926440). preprint at. • Juan G. Vargas-Rubio and Balu Santhanam (2005). "On the multiangle centered discrete fractional Fourier transform". IEEE Sig. Proc. Lett. 12 (4): 273–276. Bibcode: 2005ISPL...12..273V (http://adsabs.harvard.edu/ abs/2005ISPL...12..273V). doi: 10.1109/LSP.2005.843762 (http://dx.doi.org/10.1109/LSP.2005. 843762). • J. Cooley, P. Lewis, and P. Welch (1969). "The finite Fourier transform". IEEE Trans. Audio Electroacoustics 17 (2): 77–85. doi: 10.1109/TAU.1969.1162036 (http://dx.doi.org/10.1109/TAU.1969.1162036). • F.N. Kong (2008). "Analytic Expressions of Two Discrete Hermite-Gaussian Signals". IEEE Trans. Circuits and Systems –II: Express Briefs. 55 (1): 56–60. doi: 10.1109/TCSII.2007.909865 (http://dx.doi.org/10.1109/ TCSII.2007.909865).

External links • Matlab tutorial on the Discrete Fourier Transformation (http://www.nbtwiki.net/doku. php?id=tutorial:the_discrete_fourier_transformation_dft) • Interactive flash tutorial on the DFT (http://www.fourier-series.com/fourierseries2/DFT_tutorial.html) • Mathematics of the Discrete Fourier Transform by Julius O. Smith III (http://ccrma.stanford.edu/~jos/mdft/ mdft.html) • Fast implementation of the DFT - coded in C and under General Public License (GPL) (http://www.fftw.org) • The DFT “à Pied”: Mastering The Fourier Transform in One Day (http://www.dspdimension.com/admin/ dft-a-pied/) • Explained: The Discrete Fourier Transform (http://web.mit.edu/newsoffice/2009/explained-fourier.html) • wavetable Cooker (http://noisemakessound.com/blofeld-wavetable-cooker/) GPL application with graphical interface written in C, and implementing DFT IDFT to generate a wavetable set

156

Fast Fourier transform

157

Fast Fourier transform "FFT" redirects here. For other uses, see FFT (disambiguation). A fast Fourier transform (FFT) is an algorithm to compute the discrete Fourier transform (DFT) and its inverse. Fourier analysis converts time (or space) to frequency and vice versa; an FFT rapidly computes such transformations by factorizing the DFT matrix into a product of sparse (mostly zero) factors.[1] As a result, fast Fourier transforms are widely used for many applications in engineering, science, and mathematics. The basic ideas were popularized in 1965, but some FFTs had been previously known as early as 1805. Fast Fourier transforms have been described as "the most important numerical algorithm[s] of our lifetime".

Frequency and time domain for the same signal

Overview There are many different FFT algorithms involving a wide range of mathematics, from simple complex-number arithmetic to group theory and number theory; this article gives an overview of the available techniques and some of their general properties, while the specific algorithms are described in subsidiary articles linked below. The DFT is obtained by decomposing a sequence of values into components of different frequencies. This operation is useful in many fields (see discrete Fourier transform for properties and applications of the transform) but computing it directly from the definition is often too slow to be practical. An FFT is a way to compute the same result more quickly: computing the DFT of N points in the naive way, using the definition, takes O(N2) arithmetical operations, while a FFT can compute the same DFT in only O(N log N) operations. The difference in speed can be enormous, especially for long data sets where N may be in the thousands or millions. In practice, the computation time can be reduced by several orders of magnitude in such cases, and the improvement is roughly proportional to N / log(N). This huge improvement made the calculation of the DFT practical; FFTs are of great importance to a wide variety of applications, from digital signal processing and solving partial differential equations to algorithms for quick multiplication of large integers. The best-known FFT algorithms depend upon the factorization of N, but there are FFTs with O(N log N) complexity for all N, even for prime N. Many FFT algorithms only depend on the fact that is an N-th primitive root of unity, and thus can be applied to analogous transforms over any finite field, such as number-theoretic transforms. Since the inverse DFT is the same as the DFT, but with the opposite sign in the exponent and a 1/N factor, any FFT algorithm can easily be adapted for it.

Definition and speed An FFT computes the DFT and produces exactly the same result as evaluating the DFT definition directly; the most important difference is that an FFT is much faster. (In the presence of round-off error, many FFT algorithms are also much more accurate than evaluating the DFT definition directly, as discussed below.) Let x0, ...., xN-1 be complex numbers. The DFT is defined by the formula

Evaluating this definition directly requires O(N2) operations: there are N outputs Xk, and each output requires a sum of N terms. An FFT is any method to compute the same results in O(N log N) operations. More precisely, all known FFT algorithms require Θ(N log N) operations (technically, O only denotes an upper bound), although there is no

Fast Fourier transform known proof that a lower complexity score is impossible.(Johnson and Frigo, 2007) To illustrate the savings of an FFT, consider the count of complex multiplications and additions. Evaluating the DFT's sums directly involves N2 complex multiplications and N(N−1) complex additions [of which O(N) operations can be saved by eliminating trivial operations such as multiplications by 1]. The well-known radix-2 Cooley–Tukey algorithm, for N a power of 2, can compute the same result with only (N/2)log2(N) complex multiplications (again, ignoring simplifications of multiplications by 1 and similar) and Nlog2(N) complex additions. In practice, actual performance on modern computers is usually dominated by factors other than the speed of arithmetic operations and the analysis is a complicated subject (see, e.g., Frigo & Johnson, 2005), but the overall improvement from O(N2) to O(N log N) remains.

Algorithms Cooley–Tukey algorithm Main article: Cooley–Tukey FFT algorithm By far the most commonly used FFT is the Cooley–Tukey algorithm. This is a divide and conquer algorithm that recursively breaks down a DFT of any composite size N = N1N2 into many smaller DFTs of sizes N1 and N2, along with O(N) multiplications by complex roots of unity traditionally called twiddle factors (after Gentleman and Sande, 1966). This method (and the general idea of an FFT) was popularized by a publication of J. W. Cooley and J. W. Tukey in 1965, but it was later discovered (Heideman, Johnson, & Burrus, 1984) that those two authors had independently re-invented an algorithm known to Carl Friedrich Gauss around 1805 (and subsequently rediscovered several times in limited forms). The best known use of the Cooley–Tukey algorithm is to divide the transform into two pieces of size N/2 at each step, and is therefore limited to power-of-two sizes, but any factorization can be used in general (as was known to both Gauss and Cooley/Tukey). These are called the radix-2 and mixed-radix cases, respectively (and other variants such as the split-radix FFT have their own names as well). Although the basic idea is recursive, most traditional implementations rearrange the algorithm to avoid explicit recursion. Also, because the Cooley–Tukey algorithm breaks the DFT into smaller DFTs, it can be combined arbitrarily with any other algorithm for the DFT, such as those described below.

Other FFT algorithms Main articles: Prime-factor FFT algorithm, Bruun's FFT algorithm, Rader's FFT algorithm and Bluestein's FFT algorithm There are other FFT algorithms distinct from Cooley–Tukey. Cornelius Lanczos did pioneering work on the FFS and FFT with G.C. Danielson (1940). For N = N1N2 with coprime N1 and N2, one can use the Prime-Factor (Good-Thomas) algorithm (PFA), based on the Chinese Remainder Theorem, to factorize the DFT similarly to Cooley–Tukey but without the twiddle factors. The Rader-Brenner algorithm (1976) is a Cooley–Tukey-like factorization but with purely imaginary twiddle factors, reducing multiplications at the cost of increased additions and reduced numerical stability; it was later superseded by the split-radix variant of Cooley–Tukey (which achieves the same multiplication count but with fewer additions and without sacrificing accuracy). Algorithms that recursively factorize the DFT into smaller operations other than DFTs include the Bruun and QFT algorithms. (The Rader-Brenner and QFT algorithms were proposed for power-of-two sizes, but it is possible that they could be adapted to general composite n. Bruun's algorithm applies to arbitrary even composite sizes.) Bruun's algorithm, in particular, is based on interpreting the FFT as a recursive factorization of the polynomial zN−1, here into real-coefficient polynomials of the form zM−1 and z2M + azM + 1.

158

Fast Fourier transform

159

Another polynomial viewpoint is exploited by the Winograd algorithm, which factorizes zN−1 into cyclotomic polynomials—these often have coefficients of 1, 0, or −1, and therefore require few (if any) multiplications, so Winograd can be used to obtain minimal-multiplication FFTs and is often used to find efficient algorithms for small factors. Indeed, Winograd showed that the DFT can be computed with only O(N) irrational multiplications, leading to a proven achievable lower bound on the number of multiplications for power-of-two sizes; unfortunately, this comes at the cost of many more additions, a tradeoff no longer favorable on modern processors with hardware multipliers. In particular, Winograd also makes use of the PFA as well as an algorithm by Rader for FFTs of prime sizes. Rader's algorithm, exploiting the existence of a generator for the multiplicative group modulo prime N, expresses a DFT of prime size n as a cyclic convolution of (composite) size N−1, which can then be computed by a pair of ordinary FFTs via the convolution theorem (although Winograd uses other convolution methods). Another prime-size FFT is due to L. I. Bluestein, and is sometimes called the chirp-z algorithm; it also re-expresses a DFT as a convolution, but this time of the same size (which can be zero-padded to a power of two and evaluated by radix-2 Cooley–Tukey FFTs, for example), via the identity .

FFT algorithms specialized for real and/or symmetric data In many applications, the input data for the DFT are purely real, in which case the outputs satisfy the symmetry

and efficient FFT algorithms have been designed for this situation (see e.g. Sorensen, 1987). One approach consists of taking an ordinary algorithm (e.g. Cooley–Tukey) and removing the redundant parts of the computation, saving roughly a factor of two in time and memory. Alternatively, it is possible to express an even-length real-input DFT as a complex DFT of half the length (whose real and imaginary parts are the even/odd elements of the original real data), followed by O(N) post-processing operations. It was once believed that real-input DFTs could be more efficiently computed by means of the discrete Hartley transform (DHT), but it was subsequently argued that a specialized real-input DFT algorithm (FFT) can typically be found that requires fewer operations than the corresponding DHT algorithm (FHT) for the same number of inputs. Bruun's algorithm (above) is another method that was initially proposed to take advantage of real inputs, but it has not proved popular. There are further FFT specializations for the cases of real data that have even/odd symmetry, in which case one can gain another factor of (roughly) two in time and memory and the DFT becomes the discrete cosine/sine transform(s) (DCT/DST). Instead of directly modifying an FFT algorithm for these cases, DCTs/DSTs can also be computed via FFTs of real data combined with O(N) pre/post processing.

Computational issues Bounds on complexity and operation counts List of unsolved problems in computer science What is the lower bound on the complexity of fast Fourier transform algorithms? Can they be faster than Θ(N log N)?

A fundamental question of longstanding theoretical interest is to prove lower bounds on the complexity and exact operation counts of fast Fourier transforms, and many open problems remain. It is not even rigorously proved whether DFTs truly require Ω(N log(N)) (i.e., order N log(N) or greater) operations, even for the simple case of power of two sizes, although no algorithms with lower complexity are known. In particular, the count of arithmetic operations is usually the focus of such questions, although actual performance on modern-day computers is determined by many other factors such as cache or CPU pipeline optimization.

Fast Fourier transform

160

Following pioneering work by Winograd (1978), a tight Θ(N) lower bound is known for the number of real multiplications required by an FFT. It can be shown that only irrational real multiplications are required to compute a DFT of power-of-two length

. Moreover, explicit algorithms

that achieve this count are known (Heideman & Burrus, 1986; Duhamel, 1990). Unfortunately, these algorithms require too many additions to be practical, at least on modern computers with hardware multipliers.Wikipedia:Citation needed A tight lower bound is not known on the number of required additions, although lower bounds have been proved under some restrictive assumptions on the algorithms. In 1973, Morgenstern proved an Ω(N log(N)) lower bound on the addition count for algorithms where the multiplicative constants have bounded magnitudes (which is true for most but not all FFT algorithms). Pan (1986) proved an Ω(N log(N)) lower bound assuming a bound on a measure of the FFT algorithm's "asynchronicity", but the generality of this assumption is unclear. For the case of power-of-two N, Papadimitriou (1979) argued that the number of complex-number additions achieved by Cooley–Tukey algorithms is optimal under certain assumptions on the graph of the algorithm (his assumptions imply, among other things, that no additive identities in the roots of unity are exploited). (This argument would imply that at least real additions are required, although this is not a tight bound because extra additions are required as part of complex-number multiplications.) Thus far, no published FFT algorithm has achieved fewer than complex-number additions (or their equivalent) for power-of-two N. A third problem is to minimize the total number of real multiplications and additions, sometimes called the "arithmetic complexity" (although in this context it is the exact count and not the asymptotic complexity that is being considered). Again, no tight lower bound has been proven. Since 1968, however, the lowest published count for power-of-two N was long achieved by the split-radix FFT algorithm, which requires real multiplications and additions for N > 1. This was recently reduced to

(Johnson and Frigo, 2007;

Lundy and Van Buskirk, 2007). A slightly larger count (but still better than split radix for N≥256) was shown to be provably optimal for N≤512 under additional restrictions on the possible algorithms (split-radix-like flowgraphs with unit-modulus multiplicative factors), by reduction to a Satisfiability Modulo Theories problem solvable by brute force (Haynal & Haynal, 2011). Most of the attempts to lower or prove the complexity of FFT algorithms have focused on the ordinary complex-data case, because it is the simplest. However, complex-data FFTs are so closely related to algorithms for related problems such as real-data FFTs, discrete cosine transforms, discrete Hartley transforms, and so on, that any improvement in one of these would immediately lead to improvements in the others (Duhamel & Vetterli, 1990).

Accuracy and approximations All of the FFT algorithms discussed above compute the DFT exactly (in exact arithmetic, i.e. neglecting floating-point errors). A few "FFT" algorithms have been proposed, however, that compute the DFT approximately, with an error that can be made arbitrarily small at the expense of increased computations. Such algorithms trade the approximation error for increased speed or other properties. For example, an approximate FFT algorithm by Edelman et al. (1999) achieves lower communication requirements for parallel computing with the help of a fast multipole method. A wavelet-based approximate FFT by Guo and Burrus (1996) takes sparse inputs/outputs (time/frequency localization) into account more efficiently than is possibleWikipedia:Citation needed with an exact FFT. Another algorithm for approximate computation of a subset of the DFT outputs is due to Shentov et al. (1995). The Edelman algorithm works equally well for sparse and non-sparse data, since it is based on the compressibility (rank deficiency) of the Fourier matrix itself rather than the compressibility (sparsity) of the data. Conversely, if the data are sparse—that is, if only K out of N Fourier coefficients are nonzero—then the complexity can be reduced to O(Klog(N)log(N/K)), and this has been demonstrated to lead to practical speedups compared to an ordinary FFT for N/K>32 in a large-N example (N=222) using a probabilistic approximate algorithm (which estimates the largest K coefficients to several decimal places).[2]

Fast Fourier transform

161

Even the "exact" FFT algorithms have errors when finite-precision floating-point arithmetic is used, but these errors are typically quite small; most FFT algorithms, e.g. Cooley–Tukey, have excellent numerical properties as a consequence of the pairwise summation structure of the algorithms. The upper bound on the relative error for the Cooley–Tukey algorithm is O(ε log N), compared to O(εN3/2) for the naïve DFT formula (Gentleman and Sande, 1966), where ε is the machine floating-point relative precision. In fact, the root mean square (rms) errors are much better than these upper bounds, being only O(ε √log N) for Cooley–Tukey and O(ε √N) for the naïve DFT (Schatzman, 1996). These results, however, are very sensitive to the accuracy of the twiddle factors used in the FFT (i.e. the trigonometric function values), and it is not unusual for incautious FFT implementations to have much worse accuracy, e.g. if they use inaccurate trigonometric recurrence formulas. Some FFTs other than Cooley–Tukey, such as the Rader-Brenner algorithm, are intrinsically less stable. In fixed-point arithmetic, the finite-precision errors accumulated by FFT algorithms are worse, with rms errors growing as O(√N) for the Cooley–Tukey algorithm (Welch, 1969). Moreover, even achieving this accuracy requires careful attention to scaling in order to minimize the loss of precision, and fixed-point FFT algorithms involve rescaling at each intermediate stage of decompositions like Cooley–Tukey. To verify the correctness of an FFT implementation, rigorous guarantees can be obtained in O(Nlog(N)) time by a simple procedure checking the linearity, impulse-response, and time-shift properties of the transform on random inputs (Ergün, 1995).

Multidimensional FFTs As defined in the multidimensional DFT article, the multidimensional DFT

transforms an array xn with a d-dimensional vector of indices (over

for each j), where the division n/N, defined as

by a set of d nested summations , is

performed element-wise. Equivalently, it is the composition of a sequence of d sets of one-dimensional DFTs, performed along one dimension at a time (in any order). This compositional viewpoint immediately provides the simplest and most common multidimensional DFT algorithm, known as the row-column algorithm (after the two-dimensional case, below). That is, one simply performs a sequence of d one-dimensional FFTs (by any of the above algorithms): first you transform along the n1 dimension, then along the n2 dimension, and so on (or actually, any ordering will work). This method is easily shown to have the usual O(Nlog(N)) complexity, where is the total number of data points transformed. In particular, there are N/N1 transforms of size N1, etcetera, so the complexity of the sequence of FFTs is:

In two dimensions, the xk can be viewed as an

matrix, and this algorithm corresponds to first performing

the FFT of all the rows (resp. columns), grouping the resulting transformed rows (resp. columns) together as another matrix, and then performing the FFT on each of the columns (resp. rows) of this second matrix, and similarly grouping the results into the final result matrix. In more than two dimensions, it is often advantageous for cache locality to group the dimensions recursively. For example, a three-dimensional FFT might first perform two-dimensional FFTs of each planar "slice" for each fixed n1, and then perform the one-dimensional FFTs along the n1 direction. More generally, an asymptotically optimal cache-oblivious algorithm consists of recursively dividing the dimensions into two groups and

Fast Fourier transform

162

that are transformed recursively (rounding if d is not even) (see Frigo and Johnson, 2005). Still, this remain straightforward variation of the row-column algorithm that ultimately requires only a one-dimensional FFT algorithm as the base case, and still has O(Nlog(N)) complexity. Yet another variation is to perform matrix transpositions in between transforming subsequent dimensions, so that the transforms operate on contiguous data; this is especially important for out-of-core and distributed memory situations where accessing non-contiguous data is extremely time-consuming. There are other multidimensional FFT algorithms that are distinct from the row-column algorithm, although all of them have O(Nlog(N)) complexity. Perhaps the simplest non-row-column FFT is the vector-radix FFT algorithm, which is a generalization of the ordinary Cooley–Tukey algorithm where one divides the transform dimensions by a vector of radices at each step. (This may also have cache benefits.) The simplest case of vector-radix is where all of the radices are equal (e.g. vector-radix-2 divides all of the dimensions by two), but this is not necessary. Vector radix with only a single non-unit radix at a time, i.e. , is essentially a row-column algorithm. Other, more complicated, methods include polynomial transform algorithms due to Nussbaumer (1977), which view the transform in terms of convolutions and polynomial products. See Duhamel and Vetterli (1990) for more information and references.

Other generalizations An O(N5/2log(N)) generalization to spherical harmonics on the sphere S2 with N2 nodes was described by Mohlenkamp (1999), along with an algorithm conjectured (but not proven) to have O(N2 log2(N)) complexity; Mohlenkamp also provides an implementation in the libftsh library [3]. A spherical-harmonic algorithm with O(N2log(N)) complexity is described by Rokhlin and Tygert (2006). The Fast Folding Algorithm is analogous to the FFT, except that it operates on a series of binned waveforms rather than a series of real or complex scalar values. Rotation (which in the FFT is multiplication by a complex phasor) is a circular shift of the component waveform. Various groups have also published "FFT" algorithms for non-equispaced data, as reviewed in Potts et al. (2001). Such algorithms do not strictly compute the DFT (which is only defined for equispaced data), but rather some approximation thereof (a non-uniform discrete Fourier transform, or NDFT, which itself is often computed only approximately). More generally there are various other methods of spectral estimation.

References [1] Charles Van Loan, Computational Frameworks for the Fast Fourier Transform (SIAM, 1992). [2] Haitham Hassanieh, Piotr Indyk, Dina Katabi, and Eric Price, "Simple and Practical Algorithm for Sparse Fourier Transform" (http:/ / www. mit. edu/ ~ecprice/ papers/ sparse-fft-soda. pdf) (PDF), ACM-SIAM Symposium On Discrete Algorithms (SODA), Kyoto, January 2012. See also the sFFT Web Page (http:/ / groups. csail. mit. edu/ netmit/ sFFT/ ). [3] http:/ / www. math. ohiou. edu/ ~mjm/ research/ libftsh. html

• Brenner, N.; Rader, C. (1976). "A New Principle for Fast Fourier Transformation". IEEE Acoustics, Speech & Signal Processing 24 (3): 264–266. doi: 10.1109/TASSP.1976.1162805 (http://dx.doi.org/10.1109/TASSP. 1976.1162805). • Brigham, E. O. (2002). The Fast Fourier Transform. New York: Prentice-Hall • Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein, 2001. Introduction to Algorithms, 2nd. ed. MIT Press and McGraw-Hill. ISBN 0-262-03293-7. Especially chapter 30, "Polynomials and the FFT." • Duhamel, Pierre (1990). "Algorithms meeting the lower bounds on the multiplicative complexity of lengthDFTs and their connection with practical algorithms". IEEE Trans. Acoust. Speech. Sig. Proc. 38 (9): 1504–151. doi: 10.1109/29.60070 (http://dx.doi.org/10.1109/29.60070). • P. Duhamel and M. Vetterli, 1990, Fast Fourier transforms: a tutorial review and a state of the art (http://dx.doi. org/10.1016/0165-1684(90)90158-U), Signal Processing 19: 259–299.

Fast Fourier transform • A. Edelman, P. McCorquodale, and S. Toledo, 1999, The Future Fast Fourier Transform? (http://dx.doi.org/10. 1137/S1064827597316266), SIAM J. Sci. Computing 20: 1094–1114. • D. F. Elliott, & K. R. Rao, 1982, Fast transforms: Algorithms, analyses, applications. New York: Academic Press. • Funda Ergün, 1995, Testing multivariate linear functions: Overcoming the generator bottleneck (http://dx.doi. org/10.1145/225058.225167), Proc. 27th ACM Symposium on the Theory of Computing: 407–416. • M. Frigo and S. G. Johnson, 2005, " The Design and Implementation of FFTW3 (http://fftw.org/ fftw-paper-ieee.pdf)," Proceedings of the IEEE 93: 216–231. • Carl Friedrich Gauss, 1866. " Theoria interpolationis methodo nova tractata (http://lseet.univ-tln.fr/~iaroslav/ Gauss_Theoria_interpolationis_methodo_nova_tractata.php)," Werke band 3, 265–327. Göttingen: Königliche Gesellschaft der Wissenschaften. • W. M. Gentleman and G. Sande, 1966, "Fast Fourier transforms—for fun and profit," Proc. AFIPS 29: 563–578. doi: 10.1145/1464291.1464352 (http://dx.doi.org/10.1145/1464291.1464352) • H. Guo and C. S. Burrus, 1996, Fast approximate Fourier transform via wavelets transform (http://dx.doi.org/ 10.1117/12.255236), Proc. SPIE Intl. Soc. Opt. Eng. 2825: 250–259. • H. Guo, G. A. Sitton, C. S. Burrus, 1994, The Quick Discrete Fourier Transform (http://dx.doi.org/10.1109/ ICASSP.1994.389994), Proc. IEEE Conf. Acoust. Speech and Sig. Processing (ICASSP) 3: 445–448. • Steve Haynal and Heidi Haynal, " Generating and Searching Families of FFT Algorithms (http://jsat.ewi. tudelft.nl/content/volume7/JSAT7_13_Haynal.pdf)", Journal on Satisfiability, Boolean Modeling and Computation vol. 7, pp. 145–187 (2011). • Heideman, M. T.; Johnson, D. H.; Burrus, C. S. (1984). "Gauss and the history of the fast Fourier transform". IEEE ASSP Magazine 1 (4): 14–21. doi: 10.1109/MASSP.1984.1162257 (http://dx.doi.org/10.1109/MASSP. 1984.1162257). • Heideman, Michael T.; Burrus, C. Sidney (1986). "On the number of multiplications necessary to compute a lengthDFT". IEEE Trans. Acoust. Speech. Sig. Proc. 34 (1): 91–95. doi: 10.1109/TASSP.1986.1164785 • • • • •

• • •

(http://dx.doi.org/10.1109/TASSP.1986.1164785). S. G. Johnson and M. Frigo, 2007. " A modified split-radix FFT with fewer arithmetic operations (http://www. fftw.org/newsplit.pdf)," IEEE Trans. Signal Processing 55 (1): 111–119. T. Lundy and J. Van Buskirk, 2007. "A new matrix approach to real FFTs and convolutions of length 2k," Computing 80 (1): 23-45. Kent, Ray D. and Read, Charles (2002). Acoustic Analysis of Speech. ISBN 0-7693-0112-6. Cites Strang, G. (1994)/May–June). Wavelets. American Scientist, 82, 250-255. Morgenstern, Jacques (1973). "Note on a lower bound of the linear complexity of the fast Fourier transform". J. ACM 20 (2): 305–306. doi: 10.1145/321752.321761 (http://dx.doi.org/10.1145/321752.321761). Mohlenkamp, M. J. (1999). "A fast transform for spherical harmonics" (http://www.math.ohiou.edu/~mjm/ research/MOHLEN1999P.pdf). J. Fourier Anal. Appl. 5 (2-3): 159–184. doi: 10.1007/BF01261607 (http://dx. doi.org/10.1007/BF01261607). Nussbaumer, H. J. (1977). "Digital filtering using polynomial transforms". Electronics Lett. 13 (13): 386–387. doi: 10.1049/el:19770280 (http://dx.doi.org/10.1049/el:19770280). V. Pan, 1986, The trade-off between the additive complexity and the asyncronicity of linear and bilinear algorithms (http://dx.doi.org/10.1016/0020-0190(86)90035-9), Information Proc. Lett. 22: 11-14. Christos H. Papadimitriou, 1979, Optimality of the fast Fourier transform (http://dx.doi.org/10.1145/322108. 322118), J. ACM 26: 95-102.

• D. Potts, G. Steidl, and M. Tasche, 2001. " Fast Fourier transforms for nonequispaced data: A tutorial (http:// www.tu-chemnitz.de/~potts/paper/ndft.pdf)", in: J.J. Benedetto and P. Ferreira (Eds.), Modern Sampling Theory: Mathematics and Applications (Birkhauser).

163

Fast Fourier transform • Press, WH; Teukolsky, SA; Vetterling, WT; Flannery, BP (2007), "Chapter 12. Fast Fourier Transform" (http:// apps.nrbook.com/empanel/index.html#pg=600), Numerical Recipes: The Art of Scientific Computing (3rd ed.), New York: Cambridge University Press, ISBN 978-0-521-88068-8 • Rokhlin, Vladimir; Tygert, Mark (2006). "Fast algorithms for spherical harmonic expansions". SIAM J. Sci. Computing 27 (6): 1903–1928. doi: 10.1137/050623073 (http://dx.doi.org/10.1137/050623073). • James C. Schatzman, 1996, Accuracy of the discrete Fourier transform and the fast Fourier transform (http:// portal.acm.org/citation.cfm?id=240432), SIAM J. Sci. Comput. 17: 1150–1166. • Shentov, O. V.; Mitra, S. K.; Heute, U.; Hossen, A. N. (1995). "Subband DFT. I. Definition, interpretations and extensions". Signal Processing 41 (3): 261–277. doi: 10.1016/0165-1684(94)00103-7 (http://dx.doi.org/10. 1016/0165-1684(94)00103-7). • Sorensen, H. V.; Jones, D. L.; Heideman, M. T.; Burrus, C. S. (1987). "Real-valued fast Fourier transform algorithms". IEEE Trans. Acoust. Speech Sig. Processing 35 (35): 849–863. doi: 10.1109/TASSP.1987.1165220 (http://dx.doi.org/10.1109/TASSP.1987.1165220). See also Sorensen, H.; Jones, D.; Heideman, M.; Burrus, C. (1987). "Corrections to "Real-valued fast Fourier transform algorithms"". IEEE Transactions on Acoustics, Speech, and Signal Processing 35 (9): 1353–1353. doi: 10.1109/TASSP.1987.1165284 (http://dx.doi.org/10. 1109/TASSP.1987.1165284). • Welch, Peter D. (1969). "A fixed-point fast Fourier transform error analysis". IEEE Trans. Audio Electroacoustics 17 (2): 151–157. doi: 10.1109/TAU.1969.1162035 (http://dx.doi.org/10.1109/TAU.1969.1162035). • Winograd, S. (1978). "On computing the discrete Fourier transform". Math. Computation 32 (141): 175–199. doi: 10.1090/S0025-5718-1978-0468306-4 (http://dx.doi.org/10.1090/S0025-5718-1978-0468306-4). JSTOR 2006266 (http://www.jstor.org/stable/2006266).

External links • Fast Fourier Algorithm (http://www.cs.pitt.edu/~kirk/cs1501/animations/FFT.html) • Fast Fourier Transforms (http://cnx.org/content/col10550/), Connexions online book edited by C. Sidney Burrus, with chapters by C. Sidney Burrus, Ivan Selesnick, Markus Pueschel, Matteo Frigo, and Steven G. Johnson (2008). • Links to FFT code and information online. (http://www.fftw.org/links.html) • National Taiwan University – FFT (http://www.cmlab.csie.ntu.edu.tw/cml/dsp/training/coding/transform/ fft.html) • FFT programming in C++ — Cooley–Tukey algorithm. (http://www.librow.com/articles/article-10) • Online documentation, links, book, and code. (http://www.jjj.de/fxt/) • Using FFT to construct aggregate probability distributions (http://www.vosesoftware.com/ModelRiskHelp/ index.htm#Aggregate_distributions/Aggregate_modeling_-_Fast_Fourier_Transform_FFT_method.htm) • Sri Welaratna, " Thirty years of FFT analyzers (http://www.dataphysics.com/ 30_Years_of_FFT_Analyzers_by_Sri_Welaratna.pdf)", Sound and Vibration (January 1997, 30th anniversary issue). A historical review of hardware FFT devices. • FFT Basics and Case Study Using Multi-Instrument (http://www.virtins.com/doc/D1002/ FFT_Basics_and_Case_Study_using_Multi-Instrument_D1002.pdf) • FFT Textbook notes, PPTs, Videos (http://numericalmethods.eng.usf.edu/topics/fft.html) at Holistic Numerical Methods Institute. • ALGLIB FFT Code (http://www.alglib.net/fasttransforms/fft.php) GPL Licensed multilanguage (VBA, C++, Pascal, etc.) numerical analysis and data processing library. • MIT's sFFT (http://groups.csail.mit.edu/netmit/sFFT/) MIT Sparse FFT algorithm and implementation. • VB6 FFT (http://www.borgdesign.ro/fft.zip) VB6 optimized library implementation with source code.

164

Cooley-Tukey FFT algorithm

165

Cooley-Tukey FFT algorithm The Cooley–Tukey algorithm, named after J.W. Cooley and John Tukey, is the most common fast Fourier transform (FFT) algorithm. It re-expresses the discrete Fourier transform (DFT) of an arbitrary composite size N = N1N2 in terms of smaller DFTs of sizes N1 and N2, recursively, in order to reduce the computation time to O(N log N) for highly-composite N (smooth numbers). Because of the algorithm's importance, specific variants and implementation styles have become known by their own names, as described below. Because the Cooley-Tukey algorithm breaks the DFT into smaller DFTs, it can be combined arbitrarily with any other algorithm for the DFT. For example, Rader's or Bluestein's algorithm can be used to handle large prime factors that cannot be decomposed by Cooley–Tukey, or the prime-factor algorithm can be exploited for greater efficiency in separating out relatively prime factors. See also the fast Fourier transform for information on other FFT algorithms, specializations for real and/or symmetric data, and accuracy in the face of finite floating-point precision.

History This algorithm, including its recursive application, was invented around 1805 by Carl Friedrich Gauss, who used it to interpolate the trajectories of the asteroids Pallas and Juno, but his work was not widely recognized (being published only posthumously and in neo-Latin).[1][2] Gauss did not analyze the asymptotic computational time, however. Various limited forms were also rediscovered several times throughout the 19th and early 20th centuries. FFTs became popular after James Cooley of IBM and John Tukey of Princeton published a paper in 1965 reinventing the algorithm and describing how to perform it conveniently on a computer. Tukey reportedly came up with the idea during a meeting of a US presidential advisory committee discussing ways to detect nuclear-weapon tests in the Soviet Union.[3] Another participant at that meeting, Richard Garwin of IBM, recognized the potential of the method and put Tukey in touch with Cooley, who implemented it for a different (and less-classified) problem: analyzing 3d crystallographic data (see also: multidimensional FFTs). Cooley and Tukey subsequently published their joint paper, and wide adoption quickly followed. The fact that Gauss had described the same algorithm (albeit without analyzing its asymptotic cost) was not realized until several years after Cooley and Tukey's 1965 paper. Their paper cited as inspiration only work by I. J. Good on what is now called the prime-factor FFT algorithm (PFA); although Good's algorithm was initially mistakenly thought to be equivalent to the Cooley–Tukey algorithm, it was quickly realized that PFA is a quite different algorithm (only working for sizes that have relatively prime factors and relying on the Chinese Remainder Theorem, unlike the support for any composite size in Cooley–Tukey).[4]

The radix-2 DIT case A radix-2 decimation-in-time (DIT) FFT is the simplest and most common form of the Cooley–Tukey algorithm, although highly optimized Cooley–Tukey implementations typically use other forms of the algorithm as described below. Radix-2 DIT divides a DFT of size N into two interleaved DFTs (hence the name "radix-2") of size N/2 with each recursive stage. The discrete Fourier transform (DFT) is defined by the formula:

where

is an integer ranging from

to

.

Radix-2 DIT first computes the DFTs of the even-indexed inputs odd-indexed inputs

and of the

, and then combines those two results to produce the DFT of

Cooley-Tukey FFT algorithm

166

the whole sequence. This idea can then be performed recursively to reduce the overall runtime to O(N log N). This simplified form assumes that N is a power of two; since the number of sample points N can usually be chosen freely by the application, this is often not an important restriction. The Radix-2 DIT algorithm rearranges the DFT of the function indices and a sum over the odd-numbered indices

One can factor a common multiplier

into two parts: a sum over the even-numbered :

out of the second sum, as shown in the equation below. It is then clear

that the two sums are the DFT of the even-indexed part

and the DFT of odd-indexed part

function

by

by

. Denote the DFT of the Even-indexed inputs

of the

and the DFT of the Odd-indexed inputs

and we obtain:

Thanks to the periodicity of the DFT, we know that and

.

Therefore, we can rewrite the above equation as

We also know that the twiddle factor

obeys the following relation:

This allows us to cut the number of "twiddle factor" calculations in half also. For

, we have

This result, expressing the DFT of length N recursively in terms of two DFTs of size N/2, is the core of the radix-2 DIT fast Fourier transform. The algorithm gains its speed by re-using the results of intermediate computations to compute multiple DFT outputs. Note that final outputs are obtained by a +/− combination of and , which is simply a size-2 DFT (sometimes called a butterfly in this context); when this is generalized to larger radices below, the size-2 DFT is replaced by a larger DFT (which itself can be evaluated with an FFT).

Cooley-Tukey FFT algorithm

167

This process is an example of the general technique of divide and conquer algorithms; in many traditional implementations, however, the explicit recursion is avoided, and instead one traverses the computational tree in breadth-first fashion. The above re-expression of a size-N DFT as two size-N/2 DFTs is sometimes called the Danielson–Lanczos lemma, since the identity was noted by those two authors in 1942[5] (influenced by Runge's 1903 work). They applied their lemma in a "backwards" recursive fashion, repeatedly doubling the DFT size until the transform spectrum converged (although they apparently didn't Data flow diagram for N=8: a decimation-in-time radix-2 FFT breaks a length-N realize the linearithmic [i.e., order N log N] DFT into two length-N/2 DFTs followed by a combining stage consisting of many asymptotic complexity they had achieved). size-2 DFTs called "butterfly" operations (so-called because of the shape of the The Danielson–Lanczos work predated data-flow diagrams). widespread availability of computers and required hand calculation (possibly with mechanical aids such as adding machines); they reported a computation time of 140 minutes for a size-64 DFT operating on real inputs to 3–5 significant digits. Cooley and Tukey's 1965 paper reported a running time of 0.02 minutes for a size-2048 complex DFT on an IBM 7094 (probably in 36-bit single precision, ~8 digits). Rescaling the time by the number of operations, this corresponds roughly to a speedup factor of around 800,000. (To put the time for the hand calculation in perspective, 140 minutes for size 64 corresponds to an average of at most 16 seconds per floating-point operation, around 20% of which are multiplications.)

Pseudocode In pseudocode, the above procedure could be written: X0,...,N−1 ← ditfft2(x, N, s):

DFT of (x0, xs, x2s, ..., x(N-1)s):

if N = 1 then X0 ← x0

trivial size-1 DFT base case

X0,...,N/2−1 ← ditfft2(x, N/2, 2s)

DFT of (x0, x2s, x4s, ...)

XN/2,...,N−1 ← ditfft2(x+s, N/2, 2s)

DFT of (xs, xs+2s, xs+4s, ...)

for k = 0 to N/2−1

combine DFTs of two halves into full DFT:

else

t ← Xk Xk ← t + exp(−2πi k/N) Xk+N/2 Xk+N/2 ← t − exp(−2πi k/N) Xk+N/2 endfor endif

Here, ditfft2(x,N,1), computes X=DFT(x) out-of-place by a radix-2 DIT FFT, where N is an integer power of 2 and s=1 is the stride of the input x array. x+s denotes the array starting with xs. (The results are in the correct order in X and no further bit-reversal permutation is required; the often-mentioned necessity of a separate bit-reversal stage only arises for certain in-place algorithms, as described below.)

Cooley-Tukey FFT algorithm High-performance FFT implementations make many modifications to the implementation of such an algorithm compared to this simple pseudocode. For example, one can use a larger base case than N=1 to amortize the overhead of recursion, the twiddle factors can be precomputed, and larger radices are often used for cache reasons; these and other optimizations together can improve the performance by an order of magnitude or more.[] (In many textbook implementations the depth-first recursion is eliminated entirely in favor of a nonrecursive breadth-first approach, although depth-first recursion has been argued to have better memory locality.) Several of these ideas are described in further detail below.

General factorizations More generally, Cooley–Tukey algorithms recursively re-express a DFT of a composite size N = N1N2 as:[6] 1. Perform N1 DFTs of size N2. 2. Multiply by complex roots of unity called twiddle factors. 3. Perform N2 DFTs of size N1. Typically, either N1 or N2 is a small factor (not necessarily prime), called the radix (which can differ between stages of the recursion). If N1 is the radix, it is called a decimation in time The basic step of the Cooley–Tukey FFT for general factorizations can be viewed as (DIT) algorithm, whereas if N2 is the re-interpreting a 1d DFT as something like a 2d DFT. The 1d input array of length N = radix, it is decimation in frequency N1N2 is reinterpreted as a 2d N1×N2 matrix stored in column-major order. One performs (DIF, also called the Sande-Tukey smaller 1d DFTs along the N2 direction (the non-contiguous direction), then multiplies by phase factors (twiddle factors), and finally performs 1d DFTs along the N1 direction. The algorithm). The version presented transposition step can be performed in the middle, as shown here, or at the beginning or above was a radix-2 DIT algorithm; in end. This is done recursively for the smaller transforms. the final expression, the phase multiplying the odd transform is the twiddle factor, and the +/- combination (butterfly) of the even and odd transforms is a size-2 DFT. (The radix's small DFT is sometimes known as a butterfly, so-called because of the shape of the dataflow diagram for the radix-2 case.) There are many other variations on the Cooley–Tukey algorithm. Mixed-radix implementations handle composite sizes with a variety of (typically small) factors in addition to two, usually (but not always) employing the O(N2) algorithm for the prime base cases of the recursion (it is also possible to employ an N log N algorithm for the prime base cases, such as Rader's or Bluestein's algorithm). Split radix merges radices 2 and 4, exploiting the fact that the first transform of radix 2 requires no twiddle factor, in order to achieve what was long the lowest known arithmetic operation count for power-of-two sizes, although recent variations achieve an even lower count.[7][8] (On present-day computers, performance is determined more by cache and CPU pipeline considerations than by strict operation counts; well-optimized FFT implementations often employ larger radices and/or hard-coded base-case transforms of significant size.) Another way of looking at the Cooley–Tukey algorithm is that it re-expresses a size N one-dimensional DFT as an N1 by N2 two-dimensional DFT (plus twiddles), where the output matrix is transposed. The net result of all of these transpositions, for a radix-2 algorithm, corresponds to a bit reversal of the input (DIF) or output (DIT) indices. If, instead of using a small radix, one employs a radix of roughly √N and explicit input/output matrix transpositions, it is called a four-step algorithm (or six-step, depending on the number of transpositions), initially proposed to improve memory locality,[9][10] e.g. for cache optimization or out-of-core operation, and was

168

Cooley-Tukey FFT algorithm

169

later shown to be an optimal cache-oblivious algorithm.[11] The general Cooley–Tukey factorization rewrites the indices k and n as

and

,

respectively, where the indices ka and na run from 0..Na-1 (for a of 1 or 2). That is, it re-indexes the input (n) and output (k) as N1 by N2 two-dimensional arrays in column-major and row-major order, respectively; the difference between these indexings is a transposition, as mentioned above. When this re-indexing is substituted into the DFT formula for nk, the cross term vanishes (its exponential is unity), and the remaining terms give

where each inner sum is a DFT of size N2, each outer sum is a DFT of size N1, and the [...] bracketed term is the twiddle factor. An arbitrary radix r (as well as mixed radices) can be employed, as was shown by both Cooley and Tukey as well as Gauss (who gave examples of radix-3 and radix-6 steps). Cooley and Tukey originally assumed that the radix butterfly required O(r2) work and hence reckoned the complexity for a radix r to be O(r2 N/r logrN) = O(N log2(N) r/log2r); from calculation of values of r/log2r for integer values of r from 2 to 12 the optimal radix is found to be 3 (the closest integer to e, which minimizes r/log2r).[12] This analysis was erroneous, however: the radix-butterfly is also a DFT and can be performed via an FFT algorithm in O(r log r) operations, hence the radix r actually cancels in the complexity O(r log(r) N/r logrN), and the optimal r is determined by more complicated considerations. In practice, quite large r (32 or 64) are important in order to effectively exploit e.g. the large number of processor registers on modern processors, and even an unbounded radix r=√N also achieves O(N log N) complexity and has theoretical and practical advantages for large N as mentioned above.

Data reordering, bit reversal, and in-place algorithms Although the abstract Cooley–Tukey factorization of the DFT, above, applies in some form to all implementations of the algorithm, much greater diversity exists in the techniques for ordering and accessing the data at each stage of the FFT. Of special interest is the problem of devising an in-place algorithm that overwrites its input with its output data using only O(1) auxiliary storage. The most well-known reordering technique involves explicit bit reversal for in-place radix-2 algorithms. Bit reversal is the permutation where the data at an index n, written in binary with digits b4b3b2b1b0 (e.g. 5 digits for N=32 inputs), is transferred to the index with reversed digits b0b1b2b3b4 . Consider the last stage of a radix-2 DIT algorithm like the one presented above, where the output is written in-place over the input: when and are combined with a size-2 DFT, those two values are overwritten by the outputs. However, the two output values should go in the first and second halves of the output array, corresponding to the most significant bit b4 (for N=32); whereas the two inputs and are interleaved in the even and odd elements, corresponding to the least significant bit b0. Thus, in order to get the output in the correct place, these two bits must be swapped. If you include all of the recursive stages of a radix-2 DIT algorithm, all the bits must be swapped and thus one must pre-process the input (or post-process the output) with a bit reversal to get in-order output. (If each size-N/2 subtransform is to operate on contiguous data, the DIT input is pre-processed by bit-reversal.) Correspondingly, if you perform all of the steps in reverse order, you obtain a radix-2 DIF algorithm with bit reversal in post-processing (or pre-processing, respectively). Alternatively, some applications (such as convolution) work equally well on bit-reversed data, so one can perform forward transforms, processing, and then inverse transforms all without bit reversal to produce final results in the natural order. Many FFT users, however, prefer natural-order outputs, and a separate, explicit bit-reversal stage can have a non-negligible impact on the computation time, even though bit reversal can be done in O(N) time and has been the

Cooley-Tukey FFT algorithm subject of much research. Also, while the permutation is a bit reversal in the radix-2 case, it is more generally an arbitrary (mixed-base) digit reversal for the mixed-radix case, and the permutation algorithms become more complicated to implement. Moreover, it is desirable on many hardware architectures to re-order intermediate stages of the FFT algorithm so that they operate on consecutive (or at least more localized) data elements. To these ends, a number of alternative implementation schemes have been devised for the Cooley–Tukey algorithm that do not require separate bit reversal and/or involve additional permutations at intermediate stages. The problem is greatly simplified if it is out-of-place: the output array is distinct from the input array or, equivalently, an equal-size auxiliary array is available. The Stockham auto-sort algorithm[13] performs every stage of the FFT out-of-place, typically writing back and forth between two arrays, transposing one "digit" of the indices with each stage, and has been especially popular on SIMD architectures.[] Even greater potential SIMD advantages (more consecutive accesses) have been proposed for the Pease algorithm, which also reorders out-of-place with each stage, but this method requires separate bit/digit reversal and O(N log N) storage. One can also directly apply the Cooley–Tukey factorization definition with explicit (depth-first) recursion and small radices, which produces natural-order out-of-place output with no separate permutation step (as in the pseudocode above) and can be argued to have cache-oblivious locality benefits on systems with hierarchical memory.[14] A typical strategy for in-place algorithms without auxiliary storage and without separate digit-reversal passes involves small matrix transpositions (which swap individual pairs of digits) at intermediate stages, which can be combined with the radix butterflies to reduce the number of passes over the data.

References [1] Gauss, Carl Friedrich, " Theoria interpolationis methodo nova tractata (http:/ / lseet. univ-tln. fr/ ~iaroslav/ Gauss_Theoria_interpolationis_methodo_nova_tractata. php)", Werke, Band 3, 265–327 (Königliche Gesellschaft der Wissenschaften, Göttingen, 1866) [2] Heideman, M. T., D. H. Johnson, and C. S. Burrus, " Gauss and the history of the fast Fourier transform (http:/ / ieeexplore. ieee. org/ xpls/ abs_all. jsp?arnumber=1162257)," IEEE ASSP Magazine, 1, (4), 14–21 (1984) [3] Rockmore, Daniel N. , Comput. Sci. Eng. 2 (1), 60 (2000). The FFT — an algorithm the whole family can use (http:/ / www. cs. dartmouth. edu/ ~rockmore/ cse-fft. pdf) Special issue on "top ten algorithms of the century " (http:/ / amath. colorado. edu/ resources/ archive/ topten. pdf) [4] James W. Cooley, Peter A. W. Lewis, and Peter W. Welch, "Historical notes on the fast Fourier transform," Proc. IEEE, vol. 55 (no. 10), p. 1675–1677 (1967). [5] Danielson, G. C., and C. Lanczos, "Some improvements in practical Fourier analysis and their application to X-ray scattering from liquids," J. Franklin Inst. 233, 365–380 and 435–452 (1942). [6] Duhamel, P., and M. Vetterli, "Fast Fourier transforms: a tutorial review and a state of the art," Signal Processing 19, 259–299 (1990) [7] Lundy, T., and J. Van Buskirk, "A new matrix approach to real FFTs and convolutions of length 2k," Computing 80, 23-45 (2007). [8] Johnson, S. G., and M. Frigo, " A modified split-radix FFT with fewer arithmetic operations (http:/ / www. fftw. org/ newsplit. pdf)," IEEE Trans. Signal Processing 55 (1), 111–119 (2007). [9] Gentleman W. M., and G. Sande, "Fast Fourier transforms—for fun and profit," Proc. AFIPS 29, 563–578 (1966). [10] Bailey, David H., "FFTs in external or hierarchical memory," J. Supercomputing 4 (1), 23–35 (1990) [11] M. Frigo, C.E. Leiserson, H. Prokop, and S. Ramachandran. Cache-oblivious algorithms. In Proceedings of the 40th IEEE Symposium on Foundations of Computer Science (FOCS 99), p.285-297. 1999. Extended abstract at IEEE (http:/ / ieeexplore. ieee. org/ iel5/ 6604/ 17631/ 00814600. pdf?arnumber=814600), at Citeseer (http:/ / citeseer. ist. psu. edu/ 307799. html). [12] Cooley, J. W., P. Lewis and P. Welch, "The Fast Fourier Transform and its Applications", IEEE Trans on Education 12, 1, 28-34 (1969) [13] Originally attributed to Stockham in W. T. Cochran et al., What is the fast Fourier transform? (http:/ / dx. doi. org/ 10. 1109/ PROC. 1967. 5957), Proc. IEEE vol. 55, 1664–1674 (1967). [14] A free (GPL) C library for computing discrete Fourier transforms in one or more dimensions, of arbitrary size, using the Cooley–Tukey algorithm

170

Cooley-Tukey FFT algorithm

171

External links • a simple, pedagogical radix-2 Cooley–Tukey FFT algorithm in C++. (http://www.librow.com/articles/ article-10) • KISSFFT (http://sourceforge.net/projects/kissfft/): a simple mixed-radix Cooley–Tukey implementation in C (open source)

Butterfly diagram This article is about butterfly diagrams in FFT algorithms; for the sunspot diagrams of the same name, see Solar cycle. In the context of fast Fourier transform algorithms, a butterfly is a portion of the computation that combines the results of smaller discrete Fourier transforms (DFTs) into a larger DFT, or vice versa (breaking a larger DFT up into subtransforms). The name "butterfly" comes from the shape of the data-flow diagram in the radix-2 case, as described below.[1] The same structure can also be found in the Viterbi algorithm, used for finding the most likely sequence of hidden states. Most commonly, the term "butterfly" appears in the context of the Cooley–Tukey FFT algorithm, which recursively breaks down a DFT of composite size n = rm into r smaller transforms of size m where r is the "radix" of the transform. These smaller DFTs are then combined via size-r butterflies, which themselves are DFTs of size r (performed m times on corresponding outputs of the sub-transforms) pre-multiplied by roots of unity (known as twiddle factors). (This is the "decimation in time" case; one can also perform the steps in reverse, known as "decimation in frequency", where the butterflies come first and are post-multiplied by twiddle factors. See also the Cooley–Tukey FFT article.)

Data flow diagram connecting the inputs x (left) to the outputs y that depend on them (right) for a "butterfly" step of a radix-2 Cooley–Tukey FFT. This diagram resembles a butterfly (as in the Morpho butterfly shown for comparison), hence the name.

Radix-2 butterfly diagram In the case of the radix-2 Cooley–Tukey algorithm, the butterfly is simply a DFT of size-2 that takes two inputs (x0, x1) (corresponding outputs of the two sub-transforms) and gives two outputs (y0, y1) by the formula (not including twiddle factors):

If one draws the data-flow diagram for this pair of operations, the (x0, x1) to (y0, y1) lines cross and resemble the wings of a butterfly, hence the name (see also the illustration at right).

Butterfly diagram

172

More specifically, a decimation-in-time FFT algorithm on n = 2 p inputs with respect to a primitive n-th root of unity relies on O(n log n) butterflies of the form:

where k is an integer depending on the part of the transform being computed. Whereas the corresponding inverse transform can mathematically be performed by replacing ω with ω−1 (and possibly multiplying by an overall scale factor, depending on the normalization convention), one may also directly invert the butterflies: A decimation-in-time radix-2 FFT breaks a length-N DFT into two length-N/2 DFTs followed by a combining stage consisting of many butterfly operations.

corresponding to a decimation-in-frequency FFT algorithm.

Other uses The butterfly can also be used to improve the randomness of large arrays of partially random numbers, by bringing every 32 or 64 bit word into causal contact with every other word through a desired hashing algorithm, so that a change in any one bit has the possibility of changing all the bits in the large array.

References [1] Alan V. Oppenheim, Ronald W. Schafer, and John R. Buck, Discrete-Time Signal Processing, 2nd edition (Upper Saddle River, NJ: Prentice Hall, 1989)

External links • explanation of the FFT and butterfly diagrams (http://www.relisoft.com/Science/Physics/fft.html). • butterfly diagrams of various FFT implementations (Radix-2, Radix-4, Split-Radix) (http://www.cmlab.csie. ntu.edu.tw/cml/dsp/training/coding/transform/fft.html).

Codec

Codec This article is about encoding and decoding a digital data stream. For other uses, see Codec (disambiguation). Further information: List of codecs and Video codecs A codec is a device or computer program capable of encoding or decoding a digital data stream or signal. The word codec is a portmanteau of "coder-decoder" or, less commonly, "compressor-decompressor". A codec (the program) should not be confused with a coding or compression format or standard – a format is a document (the standard), a way of storing data, while a codec is a program (an implementation) which can read or write such files. In practice, however, "codec" is sometimes used loosely to refer to formats. A codec encodes a data stream or signal for transmission, storage or encryption, or decodes it for playback or editing. Codecs are used in videoconferencing, streaming media and video editing applications. A video camera's analog-to-digital converter (ADC) converts its analog signals into digital signals, which are then passed through a video compressor for digital transmission or storage. A receiving device then runs the signal through a video decompressor, then a digital-to-analog converter (DAC) for analog display. The term codec is also used as a generic name for a videoconferencing unit.

Related concepts An endec (encoder/decoder) is a similar yet different concept mainly used for hardware. In the mid 20th century, a "codec" was hardware that coded analog signals into pulse-code modulation (PCM) and decoded them back. Late in the century the name came to be applied to a class of software for converting among digital signal formats, and including compander functions. A modem is a contraction of modulator/demodulator (although they were referred to as "datasets" by telcos) and converts digital data from computers to analog for phone line transmission. On the receiving end the analog is converted back to digital. Codecs do the opposite (convert audio analog to digital and then computer digital sound back to audio). An audio codec converts analog audio signals into digital signals for transmission or storage. A receiving device then converts the digital signals back to analog using an audio decompressor, for playback. An example of this is the codecs used in the sound cards of personal computers. A video codec accomplishes the same task for video signals.

Compression quality • Lossy codecs: Many of the more popular codecs in the software world are lossy, meaning that they reduce quality by some amount in order to achieve compression. Often, this type of compression is virtually indistinguishable from the original uncompressed sound or images, depending on the codec and the settings used. Smaller data sets ease the strain on relatively expensive storage sub-systems such as non-volatile memory and hard disk, as well as write-once-read-many formats such as CD-ROM, DVD and Blu-ray Disc. Lower data rates also reduce cost and improve performance when the data is transmitted. • Lossless codecs: There are also many lossless codecs which are typically used for archiving data in a compressed form while retaining all of the information present in the original stream. If preserving the original quality of the stream is more important than eliminating the correspondingly larger data sizes, lossless codecs are preferred. This is especially true if the data is to undergo further processing (for example editing) in which case the repeated application of processing (encoding and decoding) on lossy codecs will degrade the quality of the resulting data such that it is no longer identifiable (visually, audibly or both). Using more than one codec or encoding scheme successively can also degrade quality significantly. The decreasing cost of storage capacity and network bandwidth has a tendency to reduce the need for lossy codecs for some media.

173

Codec

Media codecs Two principal techniques are used in codecs, pulse-code modulation and delta modulation. Codecs are often designed to emphasize certain aspects of the media to be encoded. For example, a digital video (using a DV codec) of a sports event needs to encode motion well but not necessarily exact colors, while a video of an art exhibit needs to encode color and surface texture well. Audio codecs for cell phones need to have very low latency between source encoding and playback. In contrast, audio codecs for recording or broadcast can use high-latency audio compression techniques to achieve higher fidelity at a lower bit-rate. There are thousands of audio and video codecs, ranging in cost from free to hundreds of dollars or more. This variety of codecs can create compatibility and obsolescence issues. The impact is lessened for older formats, for which free or nearly-free codecs have existed for a long time. The older formats are often ill-suited to modern applications, however, such as playback in small portable devices. For example, raw uncompressed PCM audio (44.1 kHz, 16 bit stereo, as represented on an audio CD or in a .wav or .aiff file) has long been a standard across multiple platforms, but its transmission over networks is slow and expensive compared with more modern compressed formats, such as MP3. Many multimedia data streams contain both audio and video, and often some metadata that permit synchronization of audio and video. Each of these three streams may be handled by different programs, processes, or hardware; but for the multimedia data streams to be useful in stored or transmitted form, they must be encapsulated together in a container format. Lower bitrate codecs allow more users, but they also have more distortion. Beyond the initial increase in distortion, lower bit rate codecs also achieve their lower bit rates by using more complex algorithms that make certain assumptions, such as those about the media and the packet loss rate. Other codecs may not make those same assumptions. When a user with a low bitrate codec talks to a user with another codec, additional distortion is introduced by each transcoding. AVI is sometimes erroneously described as a codec, but AVI is actually a container format, while a codec is a software or hardware tool that encodes or decodes audio or video into or from some audio or video format. Audio and video encoded with many codecs might be put into an AVI container, although AVI is not an ISO standard. There are also other well-known container formats, such as Ogg, ASF, QuickTime, RealMedia, Matroska, and DivX Media Format. Some container formats which are ISO standards are MPEG transport stream, MPEG program stream, MP4 and ISO base media file format.

References

174

FFTW

175

FFTW FFTW Developer(s)

Matteo Frigo and Steven G. Johnson

Initial release 24 March 1997 Stable release 3.3.4 / 16 March 2014 Written in

C, OCaml

Type

Numerical software

License

GPL, commercial

Website

www.fftw.org

[1]

The Fastest Fourier Transform in the West (FFTW) is a software library for computing discrete Fourier transforms (DFTs) developed by Matteo Frigo and Steven G. Johnson at the Massachusetts Institute of Technology. FFTW is known as the fastest free software implementation of the Fast Fourier transform (FFT) algorithm (upheld by regular benchmarks[2]). It can compute transforms of real and complex-valued arrays of arbitrary size and dimension in O(n log n) time. It does this by supporting a variety of algorithms and choosing the one (a particular decomposition of the transform into smaller transforms) it estimates or measures to be preferable in the particular circumstances. It works best on arrays of sizes with small prime factors, with powers of two being optimal and large primes being worst case (but still O(n log n)). To decompose transforms of composite sizes into smaller transforms, it chooses among several variants of the Cooley–Tukey FFT algorithm (corresponding to different factorizations and/or different memory-access patterns), while for prime sizes it uses either Rader's or Bluestein's FFT algorithm. Once the transform has been broken up into subtransforms of sufficiently small sizes, FFTW uses hard-coded unrolled FFTs for these small sizes that were produced (at compile time, not at run time) by code generation; these routines use a variety of algorithms including Cooley–Tukey variants, Rader's algorithm, and prime-factor FFT algorithms. For a sufficiently large number of repeated transforms it is advantageous to measure the performance of some or all of the supported algorithms on the given array size and platform. These measurements, which the authors refer to as "wisdom", can be stored in a file or string for later use. FFTW has a "guru interface" that intends "to expose as much as possible of the flexibility in the underlying FFTW architecture". This allows, among other things, multi-dimensional transforms and multiple transforms in a single call (e.g., where the data is interleaved in memory). FFTW has limited support for out-of-order transforms (using the MPI version). The data reordering incurs an overhead, which for in-place transforms of arbitrary size and dimension is non-trivial to avoid. It is undocumented for which transforms this overhead is significant. FFTW is licensed under the GNU General Public License. It is also licensed commercially by MIT and is used in the commercial MATLAB[3] matrix package for calculating FFTs. FFTW is written in the C language, but Fortran and Ada interfaces exist, as well as interfaces for a few other languages. While the library itself is C, the code is actually generated from a program called 'genfft', which is written in OCaml.[4] In 1999, FFTW won the J. H. Wilkinson Prize for Numerical Software.

FFTW

References [1] http:/ / www. fftw. org/ [2] Homepage, second paragraph (http:/ / www. fftw. org/ ), and benchmarks page (http:/ / www. fftw. org/ benchfft/ ) [3] Faster Finite Fourier Transforms: MATLAB 6 incorporates FFTW (http:/ / www. mathworks. com/ company/ newsletters/ articles/ faster-finite-fourier-transforms-matlab. html) [4] "FFTW FAQ" (http:/ / www. fftw. org/ faq/ section2. html#languages)

External links • Official website (http://www.fftw.org/)

176

177

Wavelets Wavelet A wavelet is a wave-like oscillation with an amplitude that begins at zero, increases, and then decreases back to zero. It can typically be visualized as a "brief oscillation" like one might see recorded by a seismograph or heart monitor. Generally, wavelets are purposefully crafted to have specific properties that make them useful for signal processing. Wavelets can be combined, using a "reverse, shift, multiply and integrate" technique called convolution, with portions of a known signal to extract information from the unknown signal. For example, a wavelet could be created to have a frequency of Middle C and a short duration of roughly a 32nd note. If this wavelet was to be convolved with a signal created from the recording of a song, then the resulting signal would be useful for determining when the Middle C note was being played in the song. Mathematically, the wavelet will correlate with the signal if the unknown signal contains information of similar frequency. This concept of correlation is at the core of many practical applications of wavelet theory. As a mathematical tool, wavelets can be used to extract information from many different kinds of data, including – but certainly not limited to – audio signals and images. Sets of wavelets are generally needed to analyze data fully. A set of "complementary" wavelets will decompose data without gaps or overlap so that the decomposition process is mathematically reversible. Thus, sets of complementary wavelets are useful in wavelet based compression/decompression algorithms where it is desirable to recover the original information with minimal loss. In formal terms, this representation is a wavelet series representation of a square-integrable function with respect to either a complete, orthonormal set of basis functions, or an overcomplete set or frame of a vector space, for the Hilbert space of square integrable functions.

Seismic wavelet

Name The word wavelet has been used for decades in digital signal processing and exploration geophysics. The equivalent French word ondelette meaning "small wave" was used by Morlet and Grossmann in the early 1980s.

Wavelet theory Wavelet theory is applicable to several subjects. All wavelet transforms may be considered forms of time-frequency representation for continuous-time (analog) signals and so are related to harmonic analysis. Almost all practically useful discrete wavelet transforms use discrete-time filterbanks. These filter banks are called the wavelet and scaling coefficients in wavelets nomenclature. These filterbanks may contain either finite impulse response (FIR) or infinite impulse response (IIR) filters. The wavelets forming a continuous wavelet transform (CWT) are subject to the

Wavelet

178

uncertainty principle of Fourier analysis respective sampling theory: Given a signal with some event in it, one cannot assign simultaneously an exact time and frequency response scale to that event. The product of the uncertainties of time and frequency response scale has a lower bound. Thus, in the scaleogram of a continuous wavelet transform of this signal, such an event marks an entire region in the time-scale plane, instead of just one point. Also, discrete wavelet bases may be considered in the context of other forms of the uncertainty principle. Wavelet transforms are broadly divided into three classes: continuous, discrete and multiresolution-based.

Continuous wavelet transforms (continuous shift and scale parameters) In continuous wavelet transforms, a given signal of finite energy is projected on a continuous family of frequency bands (or similar subspaces of the Lp function space L2(R) ). For instance the signal may be represented on every frequency band of the form [f, 2f] for all positive frequencies f > 0. Then, the original signal can be reconstructed by a suitable integration over all the resulting frequency components. The frequency bands or subspaces (sub-bands) are scaled versions of a subspace at scale 1. This subspace in turn is in most situations generated by the shifts of one generating function ψ in L2(R), the mother wavelet. For the example of the scale one frequency band [1, 2] this function is

with the (normalized) sinc function. That, Meyer's, and two other examples of mother wavelets are:

Meyer

Morlet

Mexican Hat

The subspace of scale a or frequency band [1/a, 2/a] is generated by the functions (sometimes called child wavelets)

where a is positive and defines the scale and b is any real number and defines the shift. The pair (a, b) defines a point in the right halfplane R+ × R. The projection of a function x onto the subspace of scale a then has the form

with wavelet coefficients

See a list of some Continuous wavelets. For the analysis of the signal x, one can assemble the wavelet coefficients into a scaleogram of the signal.

Wavelet

179

Discrete wavelet transforms (discrete shift and scale parameters) It is computationally impossible to analyze a signal using all wavelet coefficients, so one may wonder if it is sufficient to pick a discrete subset of the upper halfplane to be able to reconstruct a signal from the corresponding wavelet coefficients. One such system is the affine system for some real parameters a > 1, b > 0. The corresponding discrete subset of the halfplane consists of all the points (am, namb) with m, n in Z. The corresponding baby wavelets are now given as

A sufficient condition for the reconstruction of any signal x of finite energy by the formula

form a tight frame of L2(R).

is that the functions

Multiresolution based discrete wavelet transforms In any discretised wavelet transform, there are only a finite number of wavelet coefficients for each bounded rectangular region in the upper halfplane. Still, each coefficient requires the evaluation of an integral. In special situations this numerical complexity can be avoided if the scaled and shifted wavelets form a multiresolution analysis. This means that there has to exist an auxiliary function, the father wavelet φ in L2(R), and that a is an integer. A typical choice is a = 2 and b = 1. The most famous pair of father and mother wavelets is the Daubechies 4-tap wavelet. Note that not every orthonormal discrete wavelet basis can be associated to a multiresolution analysis; for example, the Journe wavelet admits no multiresolution analysis.

D4 wavelet

From the mother and father wavelets one constructs the subspaces

The mother wavelet

keeps the time domain properties, while the father wavelets

keeps the frequency

domain properties. From these it is required that the sequence forms a multiresolution analysis of L2 and that the subspaces

are the orthogonal

"differences" of the above sequence, that is, Wm is the orthogonal complement of Vm inside the subspace Vm−1, In analogy to the sampling theorem one may conclude that the space Vm with sampling distance 2m more or less covers the frequency baseband from 0 to 2−m-1. As orthogonal complement, Wm roughly covers the band [2−m−1, 2−m]. From those inclusions and orthogonality relations, especially sequences

and so that so that

, follows the existence of

that satisfy the identities and

Wavelet

180

The second identity of the first pair is a refinement equation for the father wavelet φ. Both pairs of identities form the basis for the algorithm of the fast wavelet transform. From the multiresolution analysis derives the orthogonal decomposition of the space L2 as

For any signal or function

this gives a representation in basis functions of the corresponding subspaces as

where the coefficients are and .

Mother wavelet For practical applications, and for efficiency reasons, one prefers continuously differentiable functions with compact support as mother (prototype) wavelet (functions). However, to satisfy analytical requirements (in the continuous WT) and in general for theoretical reasons, one chooses the wavelet functions from a subspace of the space This is the space of measurable functions that are absolutely and square integrable: and Being in this space ensures that one can formulate the conditions of zero mean and square norm one: is the condition for zero mean, and is the condition for square norm one. For ψ to be a wavelet for the continuous wavelet transform (see there for exact statement), the mother wavelet must satisfy an admissibility criterion (loosely speaking, a kind of half-differentiability) in order to get a stably invertible transform. For the discrete wavelet transform, one needs at least the condition that the wavelet series is a representation of the identity in the space L2(R). Most constructions of discrete WT make use of the multiresolution analysis, which defines the wavelet by a scaling function. This scaling function itself is solution to a functional equation. In most situations it is useful to restrict ψ to be a continuous function with a higher number M of vanishing moments, i.e. for all integer m < M

The mother wavelet is scaled (or dilated) by a factor of a and translated (or shifted) by a factor of b to give (under Morlet's original formulation):

For the continuous WT, the pair (a,b) varies over the full half-plane R+ × R; for the discrete WT this pair varies over a discrete subset of it, which is also called affine group. These functions are often incorrectly referred to as the basis functions of the (continuous) transform. In fact, as in the continuous Fourier transform, there is no basis in the continuous wavelet transform. Time-frequency interpretation uses a subtly different formulation (after Delprat). Restriction：

Wavelet

181

(1) (2)

when a1 = a and b1 = b, has a finite time interval

Comparisons with Fourier transform (continuous-time) The wavelet transform is often compared with the Fourier transform, in which signals are represented as a sum of sinusoids. In fact, the Fourier transform can be viewed as a special case of the continuous wavelet transform with the choice of the mother wavelet . The main difference in general is that wavelets are localized in both time and frequency whereas the standard Fourier transform is only localized in frequency. The Short-time Fourier transform (STFT) is similar to the wavelet transform, in that it is also time and frequency localized, but there are issues with the frequency/time resolution trade-off. In particular, assuming a rectangular window region, one may think of the STFT as a transform with a slightly different kernel

where

can often be written as

, where

and u respectively denote the length and

temporal offset of the windowing function. Using Parseval’s theorem, one may define the wavelet’s energy as = From this, the square of the temporal support of the window offset by time u is given by

and the square of the spectral support of the window acting on a frequency

As stated by the Heisenberg uncertainty principle, the product of the temporal and spectral supports for any given time-frequency atom, or resolution cell. The STFT windows restrict the resolution cells to spectral and temporal supports determined by . Multiplication with a rectangular window in the time domain corresponds to convolution with a function in the frequency domain, resulting in spurious ringing artifacts for short/localized temporal windows. With the continuous-time Fourier Transform, and this convolution is with a delta function in Fourier space, resulting in the true Fourier transform of the signal

. The window function may be some other apodizing filter,

such as a Gaussian. The choice of windowing function will affect the approximation error relative to the true Fourier transform. A given resolution cell’s time-bandwidth product may not be exceeded with the STFT. All STFT basis elements maintain a uniform spectral and temporal support for all temporal shifts or offsets, thereby attaining an equal resolution in time for lower and higher frequencies. The resolution is purely determined by the sampling width. In contrast, the wavelet transform’s multiresolutional properties enables large temporal supports for lower frequencies while maintaining short temporal widths for higher frequencies by the scaling properties of the wavelet transform. This property extends conventional time-frequency analysis into time-scale analysis.[1]

Wavelet

The discrete wavelet transform is less computationally complex, taking O(N) time as compared to O(N log N) for the fast Fourier transform. This computational advantage is not inherent to the transform, but reflects the choice of a logarithmic division of frequency, in contrast to the equally spaced frequency divisions of the FFT (Fast Fourier Transform) which uses the same basis functions as DFT (Discrete Fourier Transform).[2] It is also important to note that this complexity only applies when the filter size has no relation to the signal size. A wavelet without compact support such as the Shannon wavelet would require O(N2). (For instance, a logarithmic Fourier Transform also exists with O(N) complexity, but the original signal must be sampled logarithmically in time, which is only useful for certain types of signals.[3])

182

STFT time-frequency atoms (left) and DWT time-scale atoms (right). The time-frequency atoms are four different basis functions used for the STFT (i.e. four separate Fourier transforms required). The time-scale atoms of the DWT achieve small temporal widths for high frequencies and good temporal widths for low frequencies with a single transform basis set.

Definition of a wavelet There are a number of ways of defining a wavelet (or a wavelet family).

Scaling filter An orthogonal wavelet is entirely defined by the scaling filter – a low-pass finite impulse response (FIR) filter of length 2N and sum 1. In biorthogonal wavelets, separate decomposition and reconstruction filters are defined. For analysis with orthogonal wavelets the high pass filter is calculated as the quadrature mirror filter of the low pass, and reconstruction filters are the time reverse of the decomposition filters. Daubechies and Symlet wavelets can be defined by the scaling filter.

Scaling function Wavelets are defined by the wavelet function ψ(t) (i.e. the mother wavelet) and scaling function φ(t) (also called father wavelet) in the time domain. The wavelet function is in effect a band-pass filter and scaling it for each level halves its bandwidth. This creates the problem that in order to cover the entire spectrum, an infinite number of levels would be required. The scaling function filters the lowest level of the transform and ensures all the spectrum is covered. See [4] for a detailed explanation. For a wavelet with compact support, φ(t) can be considered finite in length and is equivalent to the scaling filter g. Meyer wavelets can be defined by scaling functions

Wavelet function The wavelet only has a time domain representation as the wavelet function ψ(t). For instance, Mexican hat wavelets can be defined by a wavelet function. See a list of a few Continuous wavelets.

History The development of wavelets can be linked to several separate trains of thought, starting with Haar's work in the early 20th century. Later work by Dennis Gabor yielded Gabor atoms (1946), which are constructed similarly to wavelets, and applied to similar purposes. Notable contributions to wavelet theory can be attributed to Zweig’s discovery of the continuous wavelet transform in 1975 (originally called the cochlear transform and discovered while

Wavelet studying the reaction of the ear to sound),[5] Pierre Goupillaud, Grossmann and Morlet's formulation of what is now known as the CWT (1982), Jan-Olov Strömberg's early work on discrete wavelets (1983), Daubechies' orthogonal wavelets with compact support (1988), Mallat's multiresolution framework (1989), Akansu's Binomial QMF (1990), Nathalie Delprat's time-frequency interpretation of the CWT (1991), Newland's harmonic wavelet transform (1993) and many others since.

Timeline • First wavelet (Haar wavelet) by Alfréd Haar (1909) • Since the 1970s: George Zweig, Jean Morlet, Alex Grossmann • Since the 1980s: Yves Meyer, Stéphane Mallat, Ingrid Daubechies, Ronald Coifman, Ali Akansu, Victor Wickerhauser,

Wavelet transforms A wavelet is a mathematical function used to divide a given function or continuous-time signal into different scale components. Usually one can assign a frequency range to each scale component. Each scale component can then be studied with a resolution that matches its scale. A wavelet transform is the representation of a function by wavelets. The wavelets are scaled and translated copies (known as "daughter wavelets") of a finite-length or fast-decaying oscillating waveform (known as the "mother wavelet"). Wavelet transforms have advantages over traditional Fourier transforms for representing functions that have discontinuities and sharp peaks, and for accurately deconstructing and reconstructing finite, non-periodic and/or non-stationary signals. Wavelet transforms are classified into discrete wavelet transforms (DWTs) and continuous wavelet transforms (CWTs). Note that both DWT and CWT are continuous-time (analog) transforms. They can be used to represent continuous-time (analog) signals. CWTs operate over every possible scale and translation whereas DWTs use a specific subset of scale and translation values or representation grid. There are a large number of wavelet transforms each suitable for different applications. For a full list see list of wavelet-related transforms but the common ones are listed below: • • • • • • • •

Continuous wavelet transform (CWT) Discrete wavelet transform (DWT) Fast wavelet transform (FWT) Lifting scheme & Generalized Lifting Scheme Wavelet packet decomposition (WPD) Stationary wavelet transform (SWT) Fractional Fourier transform (FRFT) Fractional wavelet transform (FRWT)

Generalized transforms There are a number of generalized transforms of which the wavelet transform is a special case. For example, Joseph Segman introduced scale into the Heisenberg group, giving rise to a continuous transform space that is a function of time, scale, and frequency. The CWT is a two-dimensional slice through the resulting 3d time-scale-frequency volume. Another example of a generalized transform is the chirplet transform in which the CWT is also a two dimensional slice through the chirplet transform. An important application area for generalized transforms involves systems in which high frequency resolution is crucial. For example, darkfield electron optical transforms intermediate between direct and reciprocal space have been widely used in the harmonic analysis of atom clustering, i.e. in the study of crystals and crystal defects.[6] Now

183

Wavelet that transmission electron microscopes are capable of providing digital images with picometer-scale information on atomic periodicity in nanostructure of all sorts, the range of pattern recognition[7] and strain[8]/metrology[9] applications for intermediate transforms with high frequency resolution (like brushlets[10] and ridgelets[11]) is growing rapidly. Fractional wavelet transform (FRWT) is a generalization of the classical wavelet transform in the fractional Fourier transform domains. This transform is capable of providing the time- and fractional-domain information simultaneously and representing signals in the time-fractional-frequency plane.[12]

Applications of Wavelet Transform Generally, an approximation to DWT is used for data compression if a signal is already sampled, and the CWT for signal analysis.[13] Thus, DWT approximation is commonly used in engineering and computer science, and the CWT in scientific research. Like some other transforms, wavelet transforms can be used to transform data, then encode the transformed data, resulting in effective compression. For example, JPEG 2000 is an image compression standard that uses biorthogonal wavelets. This means that although the frame is overcomplete, it is a tight frame (see types of Frame of a vector space), and the same frame functions (except for conjugation in the case of complex wavelets) are used for both analysis and synthesis, i.e., in both the forward and inverse transform. For details see wavelet compression. A related use is for smoothing/denoising data based on wavelet coefficient thresholding, also called wavelet shrinkage. By adaptively thresholding the wavelet coefficients that correspond to undesired frequency components smoothing and/or denoising operations can be performed. Wavelet transforms are also starting to be used for communication applications. Wavelet OFDM is the basic modulation scheme used in HD-PLC (a power line communications technology developed by Panasonic), and in one of the optional modes included in the IEEE 1901 standard. Wavelet OFDM can achieve deeper notches than traditional FFT OFDM, and wavelet OFDM does not require a guard interval (which usually represents significant overhead in FFT OFDM systems).[14]

As a representation of a signal Often, signals can be represented well as a sum of sinusoids. However, consider a non-continuous signal with an abrupt discontinuity; this signal can still be represented as a sum of sinusoids, but requires an infinite number, which is an observation known as Gibbs phenomenon. This, then, requires an infinite number of Fourier coefficients, which is not practical for many applications, such as compression. Wavelets are more useful for describing these signals with discontinuities because of their time-localized behavior (both Fourier and wavelet transforms are frequency-localized, but wavelets have an additional time-localization property). Because of this, many types of signals in practice may be non-sparse in the Fourier domain, but very sparse in the wavelet domain. This is particularly useful in signal reconstruction, especially in the recently popular field of compressed sensing. (Note that the Short-time Fourier transform (STFT) is also localized in time and frequency, but there are often problems with the frequency-time resolution trade-off. Wavelets are better signal representations because of multiresolution analysis.) This motivates why wavelet transforms are now being adopted for a vast number of applications, often replacing the conventional Fourier Transform. Many areas of physics have seen this paradigm shift, including molecular dynamics, ab initio calculations, astrophysics, density-matrix localisation, seismology, optics, turbulence and quantum mechanics. This change has also occurred in image processing, EEG, EMG,[15] ECG analyses, brain rhythms, DNA analysis, protein analysis, climatology, human sexual response analysis,[16] general signal processing, speech recognition, acoustics, vibration signals,[17] computer graphics, multifractal analysis, and sparse coding. In computer vision and image processing, the notion of scale space representation and Gaussian derivative operators is

184

Wavelet

185

regarded as a canonical multi-scale representation.

Wavelet Denoising Suppose we measure a noisy signal

. Assume s has a sparse representation in a certain wavelet bases,

and So

.

Most elements in p are 0 or close to 0, and Since W is orthogonal, the estimation problem amounts to recovery of a signal in iid Gaussian noise. As p is sparse, one method is to apply a gaussian mixture model for p. Assume a prior

,

is the variance of "significant" coefficients, and

is the variance of "insignificant" coefficients. Then

,

is called the shrinkage factor, which depends on the prior variances

and

. The effect of the shrinkage factor is that small coefficients are set early to 0, and large coefficients are unaltered. Small coefficients are mostly noises, and large coefficients contain actual signal. At last, apply the inverse wavelet transform to obtain

List of wavelets Discrete wavelets • • • • • • • • • • •

Beylkin (18) BNC wavelets Coiflet (6, 12, 18, 24, 30) Cohen-Daubechies-Feauveau wavelet (Sometimes referred to as CDF N/P or Daubechies biorthogonal wavelets) Daubechies wavelet (2, 4, 6, 8, 10, 12, 14, 16, 18, 20, etc.) Binomial-QMF (Also referred to as Daubechies wavelet) Haar wavelet Mathieu wavelet Legendre wavelet Villasenor wavelet Symlet[18]

Continuous wavelets Real-valued • • • • • •

Beta wavelet Hermitian wavelet Hermitian hat wavelet Meyer wavelet Mexican hat wavelet Shannon wavelet

Wavelet Complex-valued • • • • •

Complex Mexican hat wavelet fbsp wavelet Morlet wavelet Shannon wavelet Modified Morlet wavelet

Notes [1] Mallat, Stephane. "A wavelet tour of signal processing. 1998." 250-252. [2] The Scientist and Engineer's Guide to Digital Signal Processing By Steven W. Smith, Ph.D. chapter 8 equation 8-1: http:/ / www. dspguide. com/ ch8/ 4. htm [3] http:/ / homepages. dias. ie/ ~ajones/ publications/ 28. pdf [4] http:/ / www. polyvalens. com/ blog/ ?page_id=15#7. + The+ scaling+ function+ %5B7%5D [5] http:/ / scienceworld. wolfram. com/ biography/ Zweig. html Zweig, George Biography on Scienceworld.wolfram.com [6] P. Hirsch, A. Howie, R. Nicholson, D. W. Pashley and M. J. Whelan (1965/1977) Electron microscopy of thin crystals (Butterworths, London/Krieger, Malabar FLA) ISBN 0-88275-376-2 [7] P. Fraundorf, J. Wang, E. Mandell and M. Rose (2006) Digital darkfield tableaus, Microscopy and Microanalysis 12:S2, 1010–1011 (cf. arXiv:cond-mat/0403017 (http:/ / arxiv. org/ abs/ cond-mat/ 0403017)) [8] M. J. Hÿtch, E. Snoeck and R. Kilaas (1998) Quantitative measurement of displacement and strain fields from HRTEM micrographs, Ultramicroscopy 74:131-146. [9] Martin Rose (2006) Spacing measurements of lattice fringes in HRTEM image using digital darkfield decomposition (M.S. Thesis in Physics, U. Missouri – St. Louis) [10] F. G. Meyer and R. R. Coifman (1997) Applied and Computational Harmonic Analysis 4:147. [11] A. G. Flesia, H. Hel-Or, A. Averbuch, E. J. Candes, R. R. Coifman and D. L. Donoho (2001) Digital implementation of ridgelet packets (Academic Press, New York). [12] J. Shi, N.-T. Zhang, and X.-P. Liu, "A novel fractional wavelet transform and its applications," Sci. China Inf. Sci., vol. 55, no. 6, pp. 1270–1279, June 2012. URL: http:/ / www. springerlink. com/ content/ q01np2848m388647/ [13] A.N. Akansu, W.A. Serdijn and I.W. Selesnick, Emerging applications of wavelets: A review (http:/ / web. njit. edu/ ~akansu/ PAPERS/ ANA-IWS-WAS-ELSEVIER PHYSCOM 2010. pdf), Physical Communication, Elsevier, vol. 3, issue 1, pp. 1-18, March 2010. [14] An overview of P1901 PHY/MAC proposal. [15] J. Rafiee et al. Feature extraction of forearm EMG signals for prosthetics, Expert Systems with Applications 38 (2011) 4058–67. [16] J. Rafiee et al. Female sexual responses using signal processing techniques, The Journal of Sexual Medicine 6 (2009) 3086–96. (pdf) (http:/ / rafiee. us/ files/ JSM_2009. pdf) [17] J. Rafiee and Peter W. Tse, Use of autocorrelation in wavelet coefficients for fault diagnosis, Mechanical Systems and Signal Processing 23 (2009) 1554–72. [18] Matlab Toolbox – URL: http:/ / matlab. izmiran. ru/ help/ toolbox/ wavelet/ ch06_a32. html

References • Paul S. Addison, The Illustrated Wavelet Transform Handbook, Institute of Physics, 2002, ISBN 0-7503-0692-0 • Ali Akansu and Richard Haddad, Multiresolution Signal Decomposition: Transforms, Subbands, Wavelets, Academic Press, 1992, ISBN 0-12-047140-X • B. Boashash, editor, "Time-Frequency Signal Analysis and Processing – A Comprehensive Reference", Elsevier Science, Oxford, 2003, ISBN 0-08-044335-4. • Tony F. Chan and Jackie (Jianhong) Shen, Image Processing and Analysis – Variational, PDE, Wavelet, and Stochastic Methods, Society of Applied Mathematics, ISBN 0-89871-589-X (2005) • Ingrid Daubechies, Ten Lectures on Wavelets, Society for Industrial and Applied Mathematics, 1992, ISBN 0-89871-274-2 • Ramazan Gençay, Faruk Selçuk and Brandon Whitcher, An Introduction to Wavelets and Other Filtering Methods in Finance and Economics, Academic Press, 2001, ISBN 0-12-279670-5 • Haar A., Zur Theorie der orthogonalen Funktionensysteme, Mathematische Annalen, 69, pp 331–371, 1910. • Barbara Burke Hubbard, "The World According to Wavelets: The Story of a Mathematical Technique in the Making", AK Peters Ltd, 1998, ISBN 1-56881-072-5, ISBN 978-1-56881-072-0

186

Wavelet • Gerald Kaiser, A Friendly Guide to Wavelets, Birkhauser, 1994, ISBN 0-8176-3711-7 • Stéphane Mallat, "A wavelet tour of signal processing" 2nd Edition, Academic Press, 1999, ISBN 0-12-466606-X • Donald B. Percival and Andrew T. Walden, Wavelet Methods for Time Series Analysis, Cambridge University Press, 2000, ISBN 0-521-68508-7 • Press, WH; Teukolsky, SA; Vetterling, WT; Flannery, BP (2007), "Section 13.10. Wavelet Transforms" (http:// apps.nrbook.com/empanel/index.html#pg=699), Numerical Recipes: The Art of Scientific Computing (3rd ed.), New York: Cambridge University Press, ISBN 978-0-521-88068-8 • P. P. Vaidyanathan, Multirate Systems and Filter Banks, Prentice Hall, 1993, ISBN 0-13-605718-7 • Mladen Victor Wickerhauser, Adapted Wavelet Analysis From Theory to Software, A K Peters Ltd, 1994, ISBN 1-56881-041-5 • Martin Vetterli and Jelena Kovačević, "Wavelets and Subband Coding", Prentice Hall, 1995, ISBN 0-13-097080-8

External links • Hazewinkel, Michiel, ed. (2001), "Wavelet analysis" (http://www.encyclopediaofmath.org/index.php?title=p/ w097160), Encyclopedia of Mathematics, Springer, ISBN 978-1-55608-010-4 • OpenSource Wavelet C# Code (http://www.waveletstudio.net/) • JWave – Open source Java implementation of several orthogonal and non-orthogonal wavelets (https://code. google.com/p/jwave/) • Wavelet Analysis in Mathematica (http://reference.wolfram.com/mathematica/guide/Wavelets.html) (A very comprehensive set of wavelet analysis tools) • 1st NJIT Symposium on Wavelets (April 30, 1990) (First Wavelets Conference in USA) (http://web.njit.edu/ ~ali/s1.htm) • Binomial-QMF Daubechies Wavelets (http://web.njit.edu/~ali/NJITSYMP1990/ AkansuNJIT1STWAVELETSSYMPAPRIL301990.pdf) • Wavelets (http://www-math.mit.edu/~gs/papers/amsci.pdf) by Gilbert Strang, American Scientist 82 (1994) 250–255. (A very short and excellent introduction) • Wavelet Digest (http://www.wavelet.org) • NASA Signal Processor featuring Wavelet methods (http://www.grc.nasa.gov/WWW/OptInstr/ NDE_Wave_Image_ProcessorLab.html) Description of NASA Signal & Image Processing Software and Link to Download • Course on Wavelets given at UC Santa Barbara, 2004 (http://wavelets.ens.fr/ENSEIGNEMENT/COURS/ UCSB/index.html) • The Wavelet Tutorial by Polikar (http://users.rowan.edu/~polikar/WAVELETS/WTtutorial.html) (Easy to understand when you have some background with fourier transforms!) • OpenSource Wavelet C++ Code (http://herbert.the-little-red-haired-girl.org/en/software/wavelet/) • Wavelets for Kids (PDF file) (http://www.isye.gatech.edu/~brani/wp/kidsA.pdf) (Introductory (for very smart kids!)) • Link collection about wavelets (http://www.cosy.sbg.ac.at/~uhl/wav.html) • Gerald Kaiser's acoustic and electromagnetic wavelets (http://wavelets.com/pages/center.html) • A really friendly guide to wavelets (http://perso.wanadoo.fr/polyvalens/clemens/wavelets/wavelets.html) • Wavelet-based image annotation and retrieval (http://www.alipr.com) • Very basic explanation of Wavelets and how FFT relates to it (http://www.relisoft.com/Science/Physics/ sampling.html) • A Practical Guide to Wavelet Analysis (http://paos.colorado.edu/research/wavelets/) is very helpful, and the wavelet software in FORTRAN, IDL and MATLAB are freely available online. Note that the biased wavelet power spectrum needs to be rectified (http://ocgweb.marine.usf.edu/~liu/wavelet.html).

187

Wavelet

188

• WITS: Where Is The Starlet? (http://www.laurent-duval.eu/siva-wits-where-is-the-starlet.html) A dictionary of tens of wavelets and wavelet-related terms ending in -let, from activelets to x-lets through bandlets, contourlets, curvelets, noiselets, wedgelets. • Python Wavelet Transforms Package (http://www.pybytes.com/pywavelets/) OpenSource code for computing 1D and 2D Discrete wavelet transform, Stationary wavelet transform and Wavelet packet transform. • Wavelet Library (http://pages.cs.wisc.edu/~kline/wvlib) GNU/GPL library for n-dimensional discrete wavelet/framelet transforms. • The Fractional Spline Wavelet Transform (http://bigwww.epfl.ch/publications/blu0001.pdf) describes a fractional wavelet transform based on fractional b-Splines. • A Panorama on Multiscale Geometric Representations, Intertwining Spatial, Directional and Frequency Selectivity (http://dx.doi.org/10.1016/j.sigpro.2011.04.025) provides a tutorial on two-dimensional oriented wavelets and related geometric multiscale transforms. • HD-PLC Alliance (http://www.hd-plc.org/) • Signal Denoising using Wavelets (http://tx.technion.ac.il/~rc/SignalDenoisingUsingWavelets_RamiCohen. pdf) • A Concise Introduction to Wavelets (http://www.docstoc.com/docs/160022503/ A-Concise-Introduction-to-Wavelets) by René Puchinger.

Discrete wavelet transform In numerical analysis and functional analysis, a discrete wavelet transform (DWT) is any wavelet transform for which the wavelets are discretely sampled. As with other wavelet transforms, a key advantage it has over Fourier transforms is temporal resolution: it captures both frequency and location information (location in time).

Examples Haar wavelets Main article: Haar wavelet The first DWT was invented by the Hungarian mathematician Alfréd Haar. For an input represented by a list of numbers, the Haar wavelet transform may be considered to simply pair up input values, storing the difference and passing the sum. This process is repeated recursively, pairing up the sums to provide the next scale: finally resulting in differences and one final sum.

An example of the 2D discrete wavelet transform that is used in JPEG2000. The original image is high-pass filtered, yielding the three large images, each describing local changes in brightness (details) in the original image. It is then low-pass filtered and downscaled, yielding an approximation image; this image is high-pass filtered to produce the three smaller detail images, and low-pass filtered to produce the final approximation image in the upper-left.

Discrete wavelet transform

Daubechies wavelets Main article: Daubechies wavelet The most commonly used set of discrete wavelet transforms was formulated by the Belgian mathematician Ingrid Daubechies in 1988. This formulation is based on the use of recurrence relations to generate progressively finer discrete samplings of an implicit mother wavelet function; each resolution is twice that of the previous scale. In her seminal paper, Daubechies derives a family of wavelets, the first of which is the Haar wavelet. Interest in this field has exploded since then, and many variations of Daubechies' original wavelets were developed.[1]

The Dual-Tree Complex Wavelet Transform (ℂWT) The Dual-Tree Complex Wavelet Transform (ℂWT) is a relatively recent enhancement to the discrete wavelet transform (DWT), with important additional properties: It is nearly shift invariant and directionally selective in two and higher dimensions. It achieves this with a redundancy factor of only substantially lower than the undecimated DWT. The multidimensional (M-D) dual-tree ℂWT is nonseparable but is based on a computationally efficient, separable filter bank (FB).[2]

Others Other forms of discrete wavelet transform include the non- or undecimated wavelet transform (where downsampling is omitted), the Newland transform (where an orthonormal basis of wavelets is formed from appropriately constructed top-hat filters in frequency space). Wavelet packet transforms are also related to the discrete wavelet transform. Complex wavelet transform is another form.

Properties The Haar DWT illustrates the desirable properties of wavelets in general. First, it can be performed in operations; second, it captures not only a notion of the frequency content of the input, by examining it at different scales, but also temporal content, i.e. the times at which these frequencies occur. Combined, these two properties make the Fast wavelet transform (FWT) an alternative to the conventional Fast Fourier Transform (FFT).

Time Issues Due to the rate-change operators in the filter bank, the discrete WT is not time-invariant but actually very sensitive to the alignment of the signal in time. To address the time-varying problem of wavelet transforms, Mallat and Zhong proposed a new algorithm for wavelet representation of a signal, which is invariant to time shifts.[3] According to this algorithm, which is called a TI-DWT, only the scale parameter is sampled along the dyadic sequence 2^j (j∈Z) and the wavelet transform is calculated for each point in time.[4][5]

Applications The discrete wavelet transform has a huge number of applications in science, engineering, mathematics and computer science. Most notably, it is used for signal coding, to represent a discrete signal in a more redundant form, often as a preconditioning for data compression. Practical applications can also be found in signal processing of accelerations for gait analysis,[6] in digital communications and many others.[7] [8][9] It is shown that discrete wavelet transform (discrete in scale and shift, and continuous in time) is successfully implemented as analog filter bank in biomedical signal processing for design of low-power pacemakers and also in ultra-wideband (UWB) wireless communications.[10]

189

Discrete wavelet transform

Comparison with Fourier transform See also: Discrete Fourier transform To illustrate the differences and similarities between the discrete wavelet transform with the discrete Fourier transform, consider the DWT and DFT of the following sequence: (1,0,0,0), a unit impulse. The DFT has orthogonal basis (DFT matrix):

while the DWT with Haar wavelets for length 4 data has orthogonal basis in the rows of:

(To simplify notation, whole numbers are used, so the bases are orthogonal but not orthonormal.) Preliminary observations include: • Wavelets have location – the (1,1,–1,–1) wavelet corresponds to “left side” versus “right side”, while the last two wavelets have support on the left side or the right side, and one is a translation of the other. • Sinusoidal waves do not have location – they spread across the whole space – but do have phase – the second and third waves are translations of each other, corresponding to being 90° out of phase, like cosine and sine, of which these are discrete versions. Decomposing the sequence with respect to these bases yields:

The DWT demonstrates the localization: the (1,1,1,1) term gives the average signal value, the (1,1,–1,–1) places the signal in the left side of the domain, and the (1,–1,0,0) places it at the left side of the left side, and truncating at any stage yields a downsampled version of the signal:

190

Discrete wavelet transform

191

The DFT, by contrast, expresses the sequence by the interference of waves of various frequencies – thus truncating the series yields a low-pass filtered version of the series:

The sinc function, showing the time domain artifacts (undershoot and ringing) of truncating a Fourier series.

Notably, the middle approximation (2-term) differs. From the frequency domain perspective, this is a better approximation, but from the time domain perspective it has drawbacks – it exhibits undershoot – one of the values is negative, though the original series is non-negative everywhere – and ringing, where the right side is non-zero, unlike in the wavelet transform. On the other hand, the Fourier approximation correctly shows a peak, and all points are within of their correct value, though all points have error. The wavelet approximation, by contrast, places a peak on the left half, but has no peak at the first point, and while it is exactly correct for half the values (reflecting location), it has an error of

for the other values.

This illustrates the kinds of trade-offs between these transforms, and how in some respects the DWT provides preferable behavior, particularly for the modeling of transients.

Definition One level of the transform The DWT of a signal is calculated by passing it through a series of filters. First the samples are passed through a low pass filter with impulse response resulting in a convolution of the two:

The signal is also decomposed simultaneously using a high-pass filter

. The outputs giving the detail coefficients

(from the high-pass filter) and approximation coefficients (from the low-pass). It is important that the two filters are related to each other and they are known as a quadrature mirror filter. However, since half the frequencies of the signal have now been removed, half the samples can be discarded according to Nyquist’s rule. The filter outputs are then subsampled by 2 (Mallat's and the common notation is the opposite, g- high pass and h- low pass):

Discrete wavelet transform

192

This decomposition has halved the time resolution since only half of each filter output characterises the signal. However, each output has half the frequency band of the input so the frequency resolution has been doubled.

Block diagram of filter analysis

With the subsampling operator

the above summation can be written more concisely.

However computing a complete convolution

with subsequent downsampling would waste computation time.

The Lifting scheme is an optimization where these two computations are interleaved.

Cascading and Filter banks This decomposition is repeated to further increase the frequency resolution and the approximation coefficients decomposed with high and low pass filters and then down-sampled. This is represented as a binary tree with nodes representing a sub-space with a different time-frequency localisation. The tree is known as a filter bank.

A 3 level filter bank

At each level in the above diagram the signal is decomposed into low and high frequencies. Due to the decomposition process the input signal must be a multiple of where is the number of levels. For example a signal with 32 samples, frequency range 0 to

and 3 levels of decomposition, 4 output scales are

produced:

Frequency domain representation of the DWT

Level

Frequencies

Samples

3

to

4 to

4

2

to

8

1

to

16

Discrete wavelet transform

193

Relationship to the Mother Wavelet The filterbank implementation of wavelets can be interpreted as computing the wavelet coefficients of a discrete set of child wavelets for a given mother wavelet . In the case of the discrete wavelet transform, the mother wavelet is shifted and scaled by powers of two

where

is the scale parameter and

Recall that the wavelet coefficient signal of length

Now fix

is the shift parameter, both which are integers. of a signal

is the projection of

onto a wavelet, and let

be a

. In the case of a child wavelet in the discrete family above,

at a particular scale, so that

viewed as a convolution of

is a function of

can be

with a dilated, reflected, and normalized version of the mother wavelet,

, sampled at the points coefficients give at level

only. In light of the above equation,

. But this is precisely what the detail

of the discrete wavelet transform. Therefore, for an appropriate choice of

and

, the detail coefficients of the filter bank correspond exactly to a wavelet coefficient of a discrete set of child wavelets for a given mother wavelet . As an example, consider the discrete Haar wavelet, whose mother wavelet is reflected, and normalized version of this wavelet is

. Then the dilated,

, which is, indeed, the highpass

decomposition filter for the discrete Haar wavelet transform

Time Complexity The filterbank implementation of the Discrete Wavelet Transform takes only O(N) in certain cases, as compared to O(N log N) for the fast Fourier transform. Note that if

and

are both a constant length (i.e. their length is independent of N), then

and

each take O(N) time. The wavelet filterbank does each of these two O(N) convolutions, then splits the signal into two branches of size N/2. But it only recursively splits the upper branch convolved with (as contrasted with the FFT, which recursively splits both the upper branch and the lower branch). This leads to the following recurrence relation which leads to an O(N) time for the entire operation, as can be shown by a geometric series expansion of the above relation. As an example, the Discrete Haar Wavelet Transform is linear, since in that case 2.

and

are constant length

Discrete wavelet transform

Other transforms See also: Adam7 algorithm The Adam7 algorithm, used for interlacing in the Portable Network Graphics (PNG) format, is a multiscale model of the data which is similar to a DWT with Haar wavelets. Unlike the DWT, it has a specific scale – it starts from an 8×8 block, and it downsamples the image, rather than decimating (low-pass filtering, then downsampling). It thus offers worse frequency behavior, showing artifacts (pixelation) at the early stages, in return for simpler implementation.

Code example In its simplest form, the DWT is remarkably easy to compute. The Haar wavelet in Java: public static int[] discreteHaarWaveletTransform(int[] input) { // This function assumes that input.length=2^n, n>1 int[] output = new int[input.length]; for (int length = input.length >> 1; ; length >>= 1) { // length = input.length / 2^n, WITH n INCREASING to log(input.length) / log(2) for (int i = 0; i < length; ++i) { int sum = input[i * 2] + input[i * 2 + 1]; int difference = input[i * 2] - input[i * 2 + 1]; output[i] = sum; output[length + i] = difference; } if (length == 1) { return output; } //Swap arrays to do next iteration System.arraycopy(output, 0, input, 0, length

Xem thm...
PDF generated using the open source mwlib toolkit. See http://code.pediapress.com/ for more information. PDF generated at: Sat, 07 Jun 2014 14:28:26 UTC

Contents Articles Digital signal processing

1

Discrete signal

6

Sampling Sampling (signal processing)

8 8

Sample and hold

13

Digital-to-analog converter

15

Analog-to-digital converter

21

Window function

32

Quantization (signal processing)

53

Quantization error

63

ENOB

74

Sampling rate

75

Nyquist–Shannon sampling theorem

80

Nyquist frequency

91

Nyquist rate

93

Oversampling

95

Undersampling

97

Delta-sigma modulation

99

Jitter

112

Aliasing

117

Anti-aliasing filter

122

Flash ADC

124

Successive approximation ADC

126

Integrating ADC

129

Time-stretch analog-to-digital converter

137

Fourier Transforms, Discrete and Fast

142

Discrete Fourier transform

142

Fast Fourier transform

157

Cooley-Tukey FFT algorithm

165

Butterfly diagram

171

Codec

173

FFTW

175

Wavelets

177

Wavelet

177

Discrete wavelet transform

188

Fast wavelet transform

196

Haar wavelet

198

Filtering

205

Digital filter

205

Finite impulse response

211

Infinite impulse response

218

Nyquist ISI criterion

221

Pulse shaping

223

Raised-cosine filter

225

Root-raised-cosine filter

228

Adaptive filter

229

Kalman filter

234

Wiener filter

255

Receivers

260

References Article Sources and Contributors

261

Image Sources, Licenses and Contributors

265

Article Licenses License

268

Digital signal processing

Digital signal processing Digital signal processing (DSP) is the mathematical manipulation of an information signal to modify or improve it in some way. It is characterized by the representation of discrete time, discrete frequency, or other discrete domain signals by a sequence of numbers or symbols and the processing of these signals. The goal of DSP is usually to measure, filter and/or compress continuous real-world analog signals. The first step is usually to convert the signal from an analog to a digital form, by sampling and then digitizing it using an analog-to-digital converter (ADC), which turns the analog signal into a stream of numbers. However, often, the required output signal is another analog output signal, which requires a digital-to-analog converter (DAC). Even if this process is more complex than analog processing and has a discrete value range, the application of computational power to digital signal processing allows for many advantages over analog processing in many applications, such as error detection and correction in transmission as well as data compression. Digital signal processing and analog signal processing are subfields of signal processing. DSP applications include: audio and speech signal processing, sonar and radar signal processing, sensor array processing, spectral estimation, statistical signal processing, digital image processing, signal processing for communications, control of systems, biomedical signal processing, seismic data processing, etc. DSP algorithms have long been run on standard computers, as well as on specialized processors called digital signal processor and on purpose-built hardware such as application-specific integrated circuit (ASICs). Today there are additional technologies used for digital signal processing including more powerful general purpose microprocessors, field-programmable gate arrays (FPGAs), digital signal controllers (mostly for industrial apps such as motor control), and stream processors, among others. Digital signal processing can involve linear or nonlinear operations. Nonlinear signal processing is closely related to nonlinear system identification [1] and can be implemented in the time, frequency, and spatio-temporal domains.

Signal sampling Main article: Sampling (signal processing) With the increasing use of computers the usage of and need for digital signal processing has increased. To use an analog signal on a computer, it must be digitized with an analog-to-digital converter. Sampling is usually carried out in two stages, discretization and quantization. In the discretization stage, the space of signals is partitioned into equivalence classes and quantization is carried out by replacing the signal with representative signal of the corresponding equivalence class. In the quantization stage the representative signal values are approximated by values from a finite set. The Nyquist–Shannon sampling theorem states that a signal can be exactly reconstructed from its samples if the sampling frequency is greater than twice the highest frequency of the signal; but requires an infinite number of samples. In practice, the sampling frequency is often significantly more than twice that required by the signal's limited bandwidth. Some (continuous-time) periodic signals become non-periodic after sampling, and some non-periodic signals become periodic after sampling. In general, for a periodic signal with period T to be periodic (with period N) after sampling with sampling interval Ts, the following must be satisfied: where k is an integer.

1

Digital signal processing

DSP domains In DSP, engineers usually study digital signals in one of the following domains: time domain (one-dimensional signals), spatial domain (multidimensional signals), frequency domain, and wavelet domains. They choose the domain in which to process a signal by making an informed guess (or by trying different possibilities) as to which domain best represents the essential characteristics of the signal. A sequence of samples from a measuring device produces a time or spatial domain representation, whereas a discrete Fourier transform produces the frequency domain information, that is the frequency spectrum. Autocorrelation is defined as the cross-correlation of the signal with itself over varying intervals of time or space.

Time and space domains Main article: Time domain The most common processing approach in the time or space domain is enhancement of the input signal through a method called filtering. Digital filtering generally consists of some linear transformation of a number of surrounding samples around the current sample of the input or output signal. There are various ways to characterize filters; for example: • A "linear" filter is a linear transformation of input samples; other filters are "non-linear". Linear filters satisfy the superposition condition, i.e. if an input is a weighted linear combination of different signals, the output is an equally weighted linear combination of the corresponding output signals. • A "causal" filter uses only previous samples of the input or output signals; while a "non-causal" filter uses future input samples. A non-causal filter can usually be changed into a causal filter by adding a delay to it. • A "time-invariant" filter has constant properties over time; other filters such as adaptive filters change in time. • A "stable" filter produces an output that converges to a constant value with time, or remains bounded within a finite interval. An "unstable" filter can produce an output that grows without bounds, with bounded or even zero input. • A "finite impulse response" (FIR) filter uses only the input signals, while an "infinite impulse response" filter (IIR) uses both the input signal and previous samples of the output signal. FIR filters are always stable, while IIR filters may be unstable. A filter can be represented by a block diagram, which can then be used to derive a sample processing algorithm to implement the filter with hardware instructions. A filter may also be described as a difference equation, a collection of zeroes and poles or, if it is an FIR filter, an impulse response or step response. The output of a linear digital filter to any given input may be calculated by convolving the input signal with the impulse response.

Frequency domain Main article: Frequency domain Signals are converted from time or space domain to the frequency domain usually through the Fourier transform. The Fourier transform converts the signal information to a magnitude and phase component of each frequency. Often the Fourier transform is converted to the power spectrum, which is the magnitude of each frequency component squared. The most common purpose for analysis of signals in the frequency domain is analysis of signal properties. The engineer can study the spectrum to determine which frequencies are present in the input signal and which are missing. In addition to frequency information, phase information is often needed. This can be obtained from the Fourier transform. With some applications, how the phase varies with frequency can be a significant consideration.

2

Digital signal processing Filtering, particularly in non-realtime work can also be achieved by converting to the frequency domain, applying the filter and then converting back to the time domain. This is a fast, O(n log n) operation, and can give essentially any filter shape including excellent approximations to brickwall filters. There are some commonly used frequency domain transformations. For example, the cepstrum converts a signal to the frequency domain through Fourier transform, takes the logarithm, then applies another Fourier transform. This emphasizes the harmonic structure of the original spectrum. Frequency domain analysis is also called spectrum- or spectral analysis.

Z-plane analysis Main article: Z-transform Whereas analog filters are usually analyzed in terms of transfer functions in the s plane using Laplace transforms, digital filters are analyzed in the z plane in terms of Z-transforms. A digital filter may be described in the z plane by its characteristic collection of zeroes and poles. The z plane provides a means for mapping digital frequency (samples/second) to real and imaginary z components, where for continuous periodic signals and (

is the digital frequency). This is useful for providing a visualization of the frequency response of a

digital system or signal.

Wavelet Main article: Discrete wavelet transform In numerical analysis and functional analysis, a discrete wavelet transform (DWT) is any wavelet transform for which the wavelets are discretely sampled. As with other wavelet transforms, a key advantage it has over Fourier transforms is temporal resolution: it captures both frequency and location information (location in time).

Applications The main applications of DSP are audio signal processing, audio compression, digital image processing, video compression, speech processing, speech recognition, digital communications, RADAR, SONAR, Financial signal processing, seismology and biomedicine. Specific examples are speech compression An example of the 2D discrete wavelet transform that is used in JPEG2000. The and transmission in digital mobile phones, original image is high-pass filtered, yielding the three large images, each room correction of sound in hi-fi and sound describing local changes in brightness (details) in the original image. It is then low-pass filtered and downscaled, yielding an approximation image; this image is reinforcement applications, weather high-pass filtered to produce the three smaller detail images, and low-pass filtered forecasting, economic forecasting, seismic to produce the final approximation image in the upper-left. data processing, analysis and control of industrial processes, medical imaging such as CAT scans and MRI, MP3 compression, computer graphics, image manipulation, hi-fi loudspeaker crossovers and equalization, and audio effects for use with electric guitar amplifiers.

3

Digital signal processing

Implementation Depending on the requirements of the application, digital signal processing tasks can be implemented on general purpose computers (e.g. supercomputers, mainframe computers, or personal computers) or with embedded processors that may or may not include specialized microprocessors called digital signal processors. Often when the processing requirement is not real-time, processing is economically done with an existing general-purpose computer and the signal data (either input or output) exists in data files. This is essentially no different from any other data processing, except DSP mathematical techniques (such as the FFT) are used, and the sampled data is usually assumed to be uniformly sampled in time or space. For example: processing digital photographs with software such as Photoshop. However, when the application requirement is real-time, DSP is often implemented using specialized microprocessors such as the DSP56000, the TMS320, or the SHARC. These often process data using fixed-point arithmetic, though some more powerful versions use floating point arithmetic. For faster applications FPGAs might be used. Beginning in 2007, multicore implementations of DSPs have started to emerge from companies including Freescale and Stream Processors, Inc. For faster applications with vast usage, ASICs might be designed specifically. For slow applications, a traditional slower processor such as a microcontroller may be adequate. Also a growing number of DSP applications are now being implemented on Embedded Systems using powerful PCs with a Multi-core processor.

Techniques • • • • • • • • • •

Bilinear transform Discrete Fourier transform Discrete-time Fourier transform Filter design LTI system theory Minimum phase Transfer function Z-transform Goertzel algorithm s-plane

Related fields • • • • • • • • • • • •

Analog signal processing Automatic control Computer Engineering Computer Science Data compression Dataflow programming Electrical engineering Fourier Analysis Information theory Machine Learning Real-time computing Stream processing

• Telecommunication • Time series

4

Digital signal processing • Wavelet

References [1] Billings S.A. "Nonlinear System Identification: NARMAX Methods in the Time, Frequency, and Spatio-Temporal Domains". Wiley, 2013

Further reading • Alan V. Oppenheim, Ronald W. Schafer, John R. Buck : Discrete-Time Signal Processing, Prentice Hall, ISBN 0-13-754920-2 • Boaz Porat: A Course in Digital Signal Processing, Wiley, ISBN 0-471-14961-6 • Richard G. Lyons: Understanding Digital Signal Processing, Prentice Hall, ISBN 0-13-108989-7 • Jonathan Yaakov Stein, Digital Signal Processing, a Computer Science Perspective, Wiley, ISBN 0-471-29546-9 • Sen M. Kuo, Woon-Seng Gan: Digital Signal Processors: Architectures, Implementations, and Applications, Prentice Hall, ISBN 0-13-035214-4 • Bernard Mulgrew, Peter Grant, John Thompson: Digital Signal Processing - Concepts and Applications, Palgrave Macmillan, ISBN 0-333-96356-3 • Steven W. Smith (2002). Digital Signal Processing: A Practical Guide for Engineers and Scientists (http://www. dspguide.com). Newnes. ISBN 0-7506-7444-X. • Paul A. Lynn, Wolfgang Fuerst: Introductory Digital Signal Processing with Computer Applications, John Wiley & Sons, ISBN 0-471-97984-8 • James D. Broesch: Digital Signal Processing Demystified, Newnes, ISBN 1-878707-16-7 • John G. Proakis, Dimitris Manolakis: Digital Signal Processing: Principles, Algorithms and Applications, 4th ed, Pearson, April 2006, ISBN 978-0131873742 • Hari Krishna Garg: Digital Signal Processing Algorithms, CRC Press, ISBN 0-8493-7178-3 • P. Gaydecki: Foundations Of Digital Signal Processing: Theory, Algorithms And Hardware Design, Institution of Electrical Engineers, ISBN 0-85296-431-5 • Paul M. Embree, Damon Danieli: C++ Algorithms for Digital Signal Processing, Prentice Hall, ISBN 0-13-179144-3 • Vijay Madisetti, Douglas B. Williams: The Digital Signal Processing Handbook, CRC Press, ISBN 0-8493-8572-5 • Stergios Stergiopoulos: Advanced Signal Processing Handbook: Theory and Implementation for Radar, Sonar, and Medical Imaging Real-Time Systems, CRC Press, ISBN 0-8493-3691-0 • Joyce Van De Vegte: Fundamentals of Digital Signal Processing, Prentice Hall, ISBN 0-13-016077-6 • Ashfaq Khan: Digital Signal Processing Fundamentals, Charles River Media, ISBN 1-58450-281-9 • Jonathan M. Blackledge, Martin Turner: Digital Signal Processing: Mathematical and Computational Methods, Software Development and Applications, Horwood Publishing, ISBN 1-898563-48-9 • Doug Smith: Digital Signal Processing Technology: Essentials of the Communications Revolution, American Radio Relay League, ISBN 0-87259-819-5 • Charles A. Schuler: Digital Signal Processing: A Hands-On Approach, McGraw-Hill, ISBN 0-07-829744-3 • James H. McClellan, Ronald W. Schafer, Mark A. Yoder: Signal Processing First, Prentice Hall, ISBN 0-13-090999-8 • John G. Proakis: A Self-Study Guide for Digital Signal Processing, Prentice Hall, ISBN 0-13-143239-7 • N. Ahmed and K.R. Rao (1975). Orthogonal Transforms for Digital Signal Processing. Springer-Verlag (Berlin – Heidelberg – New York), ISBN 3-540-06556-3.

5

Discrete signal

6

Discrete signal A discrete signal or discrete-time signal is a time series consisting of a sequence of quantities. In other words, it is a time series that is a function over a domain of integers. Unlike a continuous-time signal, a discrete-time signal is not a function of a continuous argument; however, it may have been obtained by sampling from a continuous-time signal, and then each value in the sequence is called a sample. When a discrete-time signal obtained by sampling a sequence corresponding to uniformly spaced times, it has an associated sampling rate; the sampling rate is not apparent in the data sequence, and so needs to be associated as a characteristic unit of the system.

Discrete sampled signal

Acquisition Discrete signals may have several origins, but can usually be classified into one of two groups:[1] • By acquiring values of an analog signal at constant or variable rate. This process is called sampling.[2]

Digital signal

• By recording the number of events of a given kind over finite time periods. For example, this could be the number of people taking a certain elevator every day.

Digital signals A digital signal is a discrete-time signal for which not only the time but also the amplitude has been made discrete; in other words, its samples take on only values from a discrete set (a countable set that can be mapped one-to-one to a subset of integers). If that discrete set is finite, the discrete values can be represented with digital words of a finite width. Most commonly, these discrete values are represented as fixed-point words (either proportional to the waveform values or companded) or floating-point words.

Discrete cosine waveform with frequency of 50 Hz and a sampling rate of 1000 samples/sec, easily satisfying the sampling theorem for reconstruction of the original cosine function from samples.

The process of converting a continuous-valued discrete-time signal to a digital (discrete-valued discrete-time) signal is known as analog-to-digital conversion. It usually proceeds by replacing each original sample value by an approximation selected from a given discrete set (for example by truncating or rounding, but much more sophisticated methods exist), a process known as quantization. This process loses information, and so discrete-valued signals are only an

Discrete signal approximation of the converted continuous-valued discrete-time signal, itself only an approximation of the original continuous-valued continuous-time signal. Common practical digital signals are represented as 8-bit (256 levels), 16-bit (65,536 levels), 32-bit (4.3 billion levels), and so on, though any number of quantization levels is possible, not just powers of two.

References [1] "Digital Signal Processing" Prentice Hall - Pages 11-12 [2] "Digital Signal Processing: Instant access." Butterworth-Heinemann - Page 8

• Gershenfeld, Neil A. (1999). The Nature of mathematical Modeling. Cambridge University Press. ISBN 0-521-57095-6. • Wagner, Thomas Charles Gordon (1959). Analytical transients. Wiley.

7

8

Sampling Sampling (signal processing) In signal processing, sampling is the reduction of a continuous signal to a discrete signal. A common example is the conversion of a sound wave (a continuous signal) to a sequence of samples (a discrete-time signal). A sample refers to a value or set of values at a point in time and/or space. A sampler is a subsystem or operation that extracts samples from a continuous signal. A theoretical ideal sampler produces samples equivalent to the instantaneous value of the continuous signal at the desired points.

Signal sampling representation. The continuous signal is represented with a green colored line while the discrete samples are indicated by the blue vertical lines.

Theory See also: Nyquist–Shannon sampling theorem Sampling can be done for functions varying in space, time, or any other dimension, and similar results are obtained in two or more dimensions. For functions that vary with time, let s(t) be a continuous function (or "signal") to be sampled, and let sampling be performed by measuring the value of the continuous function every T seconds, which is called the sampling interval. Then the sampled function is given by the sequence: s(nT), for integer values of n. The sampling frequency or sampling rate, fs, is defined as the number of samples obtained in one second (samples per second), thus fs = 1/T. Reconstructing a continuous function from samples is done by interpolation algorithms. The Whittaker–Shannon interpolation formula is mathematically equivalent to an ideal lowpass filter whose input is a sequence of Dirac delta functions that are modulated (multiplied) by the sample values. When the time interval between adjacent samples is a constant (T), the sequence of delta functions is called a Dirac comb. Mathematically, the modulated Dirac comb is equivalent to the product of the comb function with s(t). That purely mathematical abstraction is sometimes referred to as impulse sampling. Most sampled signals are not simply stored and reconstructed. But the fidelity of a theoretical reconstruction is a customary measure of the effectiveness of sampling. That fidelity is reduced when s(t) contains frequency components higher than fs/2 Hz, which is known as the Nyquist frequency of the sampler. Therefore s(t) is usually the output of a lowpass filter, functionally known as an anti-aliasing filter. Without an anti-aliasing filter, frequencies higher than the Nyquist frequency will influence the samples in a way that is misinterpreted by the interpolation process.[1] For details, see Aliasing.

Sampling (signal processing)

Practical considerations In practice, the continuous signal is sampled using an analog-to-digital converter (ADC), a device with various physical limitations. This results in deviations from the theoretically perfect reconstruction, collectively referred to as distortion. Various types of distortion can occur, including: • Aliasing. Some amount of aliasing is inevitable because only theoretical, infinitely long, functions can have no frequency content above the Nyquist frequency. Aliasing can be made arbitrarily small by using a sufficiently large order of the anti-aliasing filter. • Aperture error results from the fact that the sample is obtained as a time average within a sampling region, rather than just being equal to the signal value at the sampling instant. In a capacitor-based sample and hold circuit, aperture error is introduced because the capacitor cannot instantly change voltage thus requiring the sample to have non-zero width. • Jitter or deviation from the precise sample timing intervals. • Noise, including thermal sensor noise, analog circuit noise, etc. • Slew rate limit error, caused by the inability of the ADC input value to change sufficiently rapidly. • Quantization as a consequence of the finite precision of words that represent the converted values. • Error due to other non-linear effects of the mapping of input voltage to converted output value (in addition to the effects of quantization). Although the use of oversampling can completely eliminate aperture error and aliasing by shifting them out of the pass band, this technique cannot be practically used above a few GHz, and may be prohibitively expensive at much lower frequencies. Furthermore, while oversampling can reduce quantization error and non-linearity, it cannot eliminate these entirely. Consequently, practical ADCs at audio frequencies typically do not exhibit aliasing, aperture error, and are not limited by quantization error. Instead, analog noise dominates. At RF and microwave frequencies where oversampling is impractical and filters are expensive, aperture error, quantization error and aliasing can be significant limitations. Jitter, noise, and quantization are often analyzed by modeling them as random errors added to the sample values. Integration and zero-order hold effects can be analyzed as a form of low-pass filtering. The non-linearities of either ADC or DAC are analyzed by replacing the ideal linear function mapping with a proposed nonlinear function.

Applications Audio sampling Digital audio uses pulse-code modulation and digital signals for sound reproduction. This includes analog-to-digital conversion (ADC), digital-to-analog conversion (DAC), storage, and transmission. In effect, the system commonly referred to as digital is in fact a discrete-time, discrete-level analog of a previous electrical analog. While modern systems can be quite subtle in their methods, the primary usefulness of a digital system is the ability to store, retrieve and transmit signals without any loss of quality. Sampling rate When it is necessary to capture audio covering the entire 20–20,000 Hz range of human hearing, such as when recording music or many types of acoustic events, audio waveforms are typically sampled at 44.1 kHz (CD), 48 kHz (professional audio), 88.2 kHz, or 96 kHz. The approximately double-rate requirement is a consequence of the Nyquist theorem. Sampling rates higher than about 50 kHz to 60 kHz cannot supply more usable information for human listeners. Early professional audio equipment manufacturers chose sampling rates in the region of 50 kHz for this reason.

9

Sampling (signal processing)

10

There has been an industry trend towards sampling rates well beyond the basic requirements: such as 96 kHz and even 192 kHz This is in contrast with laboratory experiments, which have failed to show that ultrasonic frequencies are audible to human observers; however in some cases ultrasonic sounds do interact with and modulate the audible part of the frequency spectrum (intermodulation distortion). It is noteworthy that intermodulation distortion is not present in the live audio and so it represents an artificial coloration to the live sound. One advantage of higher sampling rates is that they can relax the low-pass filter design requirements for ADCs and DACs, but with modern oversampling sigma-delta converters this advantage is less important. The Audio Engineering Society recommends 48 kHz sample rate for most applications but gives recognition to 44.1 kHz for Compact Disc and other consumer uses, 32 kHz for transmission-related application, and 96 kHz for higher bandwidth or relaxed anti-aliasing filtering. A more complete list of common audio sample rates is: Sampling rate

Use

8,000 Hz

[2] Telephone and encrypted walkie-talkie, wireless intercom and wireless microphone transmission; adequate for human speech but without sibilance; ess sounds like eff (/s/, /f/).

11,025 Hz

One quarter the sampling rate of audio CDs; used for lower-quality PCM, MPEG audio and for audio analysis of subwoofer bandpasses.Wikipedia:Citation needed

16,000 Hz

Wideband frequency extension over standard telephone narrowband 8,000 Hz. Used in most modern VoIP and VVoIP [3] communication products.

22,050 Hz

One half the sampling rate of audio CDs; used for lower-quality PCM and MPEG audio and for audio analysis of low frequency energy. Suitable for digitizing early 20th century audio formats such as 78s.

32,000 Hz

miniDV digital video camcorder, video tapes with extra channels of audio (e.g. DVCAM with 4 Channels of audio), DAT (LP mode), Germany's Digitales Satellitenradio, NICAM digital audio, used alongside analogue television sound in some countries. High-quality digital wireless microphones. Suitable for digitizing FM radio.Wikipedia:Citation needed

44,056 Hz

Used by digital audio locked to NTSC color video signals (245 lines by 3 samples by 59.94 fields per second = 29.97 frames per second).

44,100 Hz

Audio CD, also most commonly used with MPEG-1 audio (VCD, SVCD, MP3). Originally chosen by Sony because it could be recorded on modified video equipment running at either 25 frames per second (PAL) or 30 frame/s (using an NTSC monochrome video recorder) and cover the 20 kHz bandwidth thought necessary to match professional analog recording equipment of the time. A PCM adaptor would fit digital audio samples into the analog video channel of, for example, PAL video tapes using 588 lines by 3 samples by 25 frames per second.

47,250 Hz

world's first commercial PCM sound recorder by Nippon Columbia (Denon)

48,000 Hz

The standard audio sampling rate used by professional digital video equipment such as tape recorders, video servers, vision mixers and so on. This rate was chosen because it could deliver a 22 kHz frequency response and work with 29.97 frames per second NTSC video - as well as 25 frame/s, 30 frame/s and 24 frame/s systems. With 29.97 frame/s systems it is necessary to handle 1601.6 audio samples per frame delivering an integer number of audio samples only every fifth video frame. Also used for sound with consumer video formats like DV, digital TV, DVD, and films. The professional Serial Digital Interface (SDI) and High-definition Serial Digital Interface (HD-SDI) used to connect broadcast television equipment together uses this audio sampling frequency. Most professional audio gear uses 48 kHz sampling, including mixing consoles, and digital recording devices.

50,000 Hz

First commercial digital audio recorders from the late 70s from 3M and Soundstream.

50,400 Hz

Sampling rate used by the Mitsubishi X-80 digital audio recorder.

88,200 Hz

Sampling rate used by some professional recording equipment when the destination is CD (multiples of 44,100 Hz). Some pro audio gear uses (or is able to select) 88.2 kHz sampling, including mixers, EQs, compressors, reverb, crossovers and recording devices.

96,000 Hz

DVD-Audio, some LPCM DVD tracks, BD-ROM (Blu-ray Disc) audio tracks, HD DVD (High-Definition DVD) audio tracks. Some professional recording and production equipment is able to select 96 kHz sampling. This sampling frequency is twice the 48 kHz standard commonly used with audio on professional equipment.

176,400 Hz

Sampling rate used by HDCD recorders and other professional applications for CD production.

Sampling (signal processing)

192,000 Hz

DVD-Audio, some LPCM DVD tracks, BD-ROM (Blu-ray Disc) audio tracks, and HD DVD (High-Definition DVD) audio tracks, High-Definition audio recording devices and audio editing software. This sampling frequency is four times the 48 kHz standard commonly used with audio on professional video equipment.

352,800 Hz

Digital eXtreme Definition, used for recording and editing Super Audio CDs, as 1-bit DSD is not suited for editing. Eight times the frequency of 44.1 kHz.

2,822,400 Hz SACD, 1-bit delta-sigma modulation process known as Direct Stream Digital, co-developed by Sony and Philips. 5,644,800 Hz Double-Rate DSD, 1-bit Direct Stream Digital at 2x the rate of the SACD. Used in some professional DSD recorders.

Bit depth See also: Audio bit depth Audio is typically recorded at 8-, 16-, and 20-bit depth, which yield a theoretical maximum Signal-to-quantization-noise ratio (SQNR) for a pure sine wave of, approximately, 49.93 dB, 98.09 dB and 122.17 dB. CD quality audio uses 16-bit samples. Thermal noise limits the true number of bits that can be used in quantization. Few analog systems have signal to noise ratios (SNR) exceeding 120 dB. However, digital signal processing operations can have very high dynamic range, consequently it is common to perform mixing and mastering operations at 32-bit precision and then convert to 16 or 24 bit for distribution. Speech sampling Speech signals, i.e., signals intended to carry only human speech, can usually be sampled at a much lower rate. For most phonemes, almost all of the energy is contained in the 5Hz-4 kHz range, allowing a sampling rate of 8 kHz. This is the sampling rate used by nearly all telephony systems, which use the G.711 sampling and quantization specifications.

Video sampling Standard-definition television (SDTV) uses either 720 by 480 pixels (US NTSC 525-line) or 704 by 576 pixels (UK PAL 625-line) for the visible picture area. High-definition television (HDTV) uses 720p (progressive), 1080i (interlaced), and 1080p (progressive, also known as Full-HD). In digital video, the temporal sampling rate is defined the frame rate – or rather the field rate – rather than the notional pixel clock. The image sampling frequency is the repetition rate of the sensor integration period. Since the integration period may be significantly shorter than the time between repetitions, the sampling frequency can be different from the inverse of the sample time: • 50 Hz – PAL video • 60 / 1.001 Hz ~= 59.94 Hz – NTSC video Video digital-to-analog converters operate in the megahertz range (from ~3 MHz for low quality composite video scalers in early games consoles, to 250 MHz or more for the highest-resolution VGA output). When analog video is converted to digital video, a different sampling process occurs, this time at the pixel frequency, corresponding to a spatial sampling rate along scan lines. A common pixel sampling rate is: • 13.5 MHz – CCIR 601, D1 video Spatial sampling in the other direction is determined by the spacing of scan lines in the raster. The sampling rates and resolutions in both spatial directions can be measured in units of lines per picture height. Spatial aliasing of high-frequency luma or chroma video components shows up as a moiré pattern.

11

Sampling (signal processing)

12

3D sampling • X-ray computed tomography uses 3 dimensional space • Voxel

Undersampling Main article: Undersampling When a bandpass signal is sampled slower than its Nyquist rate, the samples are indistinguishable from samples of a low-frequency alias of the high-frequency signal. That is often done purposefully in such a way that the lowest-frequency alias satisfies the Nyquist criterion, because the bandpass signal is still uniquely represented and recoverable. Such undersampling is also known as bandpass sampling, harmonic sampling, IF sampling, and direct IF to digital conversion.

The top 2 graphs depict Fourier transforms of 2 different functions that produce the same results when sampled at a particular rate. The baseband function is sampled faster than its Nyquist rate, and the bandpass function is undersampled, effectively converting it to baseband. The lower graphs indicate how identical spectral results are created by the aliases of the sampling process.

Oversampling Main article: Oversampling Oversampling is used in most modern analog-to-digital converters to reduce the distortion introduced by practical digital-to-analog converters, such as a zero-order hold instead of idealizations like the Whittaker–Shannon interpolation formula.

Complex sampling Complex sampling (I/Q sampling) refers to the simultaneous sampling of two different, but related, waveforms, resulting in pairs of samples that are subsequently treated as complex numbers.[4] When one waveform is the Hilbert transform of the other waveform

the complex-valued function,

is called an analytic signal, whose Fourier transform is zero for all negative values of frequency. In that case, the Nyquist rate for a waveform with no frequencies ≥ B can be reduced to just B (complex samples/sec), instead of 2B (real samples/sec).[5] More apparently, the equivalent baseband waveform, also has a Nyquist rate of B, because all of its non-zero frequency content is shifted into the interval [-B/2, B/2). Although complex-valued samples can be obtained as described above, they are also created by manipulating samples of a real-valued waveform. For instance, the equivalent baseband waveform can be created without [6] explicitly computing by processing the product sequence through a digital lowpass filter whose cutoff frequency is B/2.[7] Computing only every other sample of the output sequence reduces the sample-rate commensurate with the reduced Nyquist rate. The result is half as many complex-valued samples as the original number of real samples. No information is lost, and the original s(t) waveform can be recovered, if necessary.

Sampling (signal processing)

13

Notes [1] C. E. Shannon, "Communication in the presence of noise", Proc. Institute of Radio Engineers, vol. 37, no.1, pp. 10–21, Jan. 1949. Reprint as classic paper in: Proc. IEEE, Vol. 86, No. 2, (Feb 1998) (http:/ / www. stanford. edu/ class/ ee104/ shannonpaper. pdf) [2] HME DX200 encrypted wireless intercom (http:/ / www. hme. com/ proDX200. cfm) [3] http:/ / www. voipsupply. com/ cisco-hd-voice [4] Sample-pairs are also sometimes viewed as points on a constellation diagram. [5] When the complex sample-rate is B, a frequency component at 0.6 B, for instance, will have an alias at −0.4 B, which is unambiguous because of the constraint that the pre-sampled signal was analytic. Also see Aliasing#Complex_sinusoids [6] When s(t) is sampled at the Nyquist frequency (1/T = 2B), the product sequence simplifies to UNIQ-math-0-fdcc463dc40e329d-QINU [7] The sequence of complex numbers is convolved with the impulse response of a filter with real-valued coefficients. That is equivalent to separately filtering the sequences of real parts and imaginary parts and reforming complex pairs at the outputs.

Citations Further reading • Matt Pharr and Greg Humphreys, Physically Based Rendering: From Theory to Implementation, Morgan Kaufmann, July 2004. ISBN 0-12-553180-X. The chapter on sampling ( available online (http://graphics. stanford.edu/~mmp/chapters/pbrt_chapter7.pdf)) is nicely written with diagrams, core theory and code sample.

External links • Journal devoted to Sampling Theory (http://www.stsip.org) • I/Q Data for Dummies (http://whiteboard.ping.se/SDR/IQ) A page trying to answer the question Why I/Q Data?

Sample and hold For Neil Young song, see Trans (album). For remix album by Simian Mobile Disco, see Sample and Hold. In electronics, a sample and hold (S/H, also "follow-and-hold"[1]) circuit is an analog device that samples (captures, grabs) the voltage of a continuously varying analog signal and holds (locks, freezes) its value at a constant level for a specified minimum period of time. Sample and hold circuits and related peak detectors are the elementary analog memory devices. They are typically used in analog-to-digital converters to eliminate variations in input signal that can corrupt the conversion process.[2] A typical sample and hold circuit stores electric charge in a capacitor and contains at least one fast FET switch and at least one operational amplifier. To sample the input signal the switch connects the capacitor to the output of a buffer amplifier. The buffer amplifier charges or discharges the capacitor so that the voltage across the capacitor is practically equal, or proportional

A simplified sample and hold circuit diagram. AI is an analog input, AO — an analog output, C — a control signal.

Sample times.

Sample and hold

to, input voltage. In hold mode the switch disconnects the capacitor from the buffer. The capacitor is invariably discharged by its own leakage currents and useful load currents, which makes the circuit inherently volatile, but the loss of voltage (voltage drop) within a specified hold time remains within an acceptable error margin. For practically all commercial liquid crystal active matrix displays based on TN, IPS or VA electro-optic LC cells (excluding bi-stable Sample and hold. phenomena), each pixel represents a small capacitor, which has to be periodically charged to a level corresponding to the greyscale value (contrast) desired for a picture element. In order to maintain the level during a scanning cycle (frame period), an additional electric capacitor is attached in parallel to each LC pixel to better hold the voltage. A thin-film FET switch is addressed to select a particular LC pixel and charge the picture information for it. In contrast to an S/H in general electronics, there is no output operational amplifier and no electrical signal AO. Instead, the charge on the hold capacitors controls the deformation of the LC molecules and thereby the optical effect as its output. The invention of this concept and its implementation in thin-film technology have been honored with the IEEE Jun-ichi Nishizawa Medal in 2011.[3] During a scanning cycle, the picture doesn’t follow the input signal. This does not allow the eye to refresh and can lead to blurring during motion sequences, also the transition is visible between frames because the backlight is constantly illuminated, adding to display motion blur.[4][5]

Purpose Sample and hold circuits are used in linear systems. In some kinds of analog-to-digital converters, the input is compared to a voltage generated internally from a digital-to-analog converter (DAC). The circuit tries a series of values and stops converting once the voltages are equal, within some defined error margin. If the input value was permitted to change during this comparison process, the resulting conversion would be inaccurate and possibly completely unrelated to the true input value. Such successive approximation converters will often incorporate internal sample and hold circuitry. In addition, sample and hold circuits are often used when multiple samples need to be measured at the same time. Each value is sampled and held, using a common sample clock.

Implementation To keep the input voltage as stable as possible, it is essential that the capacitor have very low leakage, and that it not be loaded to any significant degree which calls for a very high input impedance.

Notes [1] [2] [3] [4]

Horowitz and Hill, p. 220. Kefauver and Patschke, p. 37. Press release IEEE, Aug. 2011 (http:/ / www. ieee. org/ about/ news/ 2011/ honors_ceremony/ releases_nishizawa. html) Charles Poynton is an authority on artifacts related to HDTV, and discusses motion artifacts succinctly and specifically (http:/ / www. poynton. com/ PDFs/ Motion_portrayal. pdf) [5] Eye-tracking based motion blur on LCD (http:/ / ieeexplore. ieee. org/ xpls/ abs_all. jsp?arnumber=5583881& tag=1)

14

Sample and hold

References • Paul Horowitz, Winfield Hill (2001 ed.). The Art of Electronics (http://books.google.com/ books?id=bkOMDgwFA28C&pg=PA220&dq=sample+and+hold&cd=1#v=onepage&q=sample and hold& f=false). Cambridge University Press. ISBN 0-521-37095-7. • Alan P. Kefauver, David Patschke (2007). Fundamentals of digital audio (http://books.google.com/ books?id=UpzqCrj7QxYC&pg=PA60&dq=sample+and+hold&cd=7#v=onepage&q=sample and hold& f=false). A-R Editions, Inc. ISBN 0-89579-611-2. • Analog Devices 21 page Tutorial "Sample and Hold Amplifiers" http://www.analog.com/static/imported-files/ tutorials/MT-090.pdf • Ndjountche, Tertulien (2011). CMOS Analog Integrated Circuits: High-Speed and Power-Efficient Design (http:/ /www.crcpress.com/ecommerce_product/product_detail.jsf?isbn=0&catno=k12557). Boca Raton, FL, USA: CRC Press. p. 925. ISBN 978-1-4398-5491-4. • Applications of Monolithic Sample and hold Amplifiers-Intersil (http://www.intersil.com/data/an/an517.pdf)

Digital-to-analog converter For digital television converter boxes, see digital television adapter. In electronics, a digital-to-analog converter (DAC, D/A, D2A or D-to-A) is a function that converts digital data (usually binary) into an analog signal (current, voltage, or electric charge). An analog-to-digital converter (ADC) performs the reverse function. Unlike analog signals, digital data can be transmitted, manipulated, and stored without degradation, albeit with more complex equipment. But a DAC is needed to convert the digital signal to analog to drive an earphone or loudspeaker amplifier in order to produce sound (analog air pressure waves). DACs and their inverse, ADCs, are part of an enabling technology 8-channel digital-to-analog converter Cirrus Logic CS4382 as used in a soundcard. that has contributed greatly to the digital revolution. To illustrate, consider a typical long-distance telephone call. The caller's voice is converted into an analog electrical signal by a microphone, then the analog signal is converted to a digital stream by an ADC. The digital stream is then divided into packets where it may be mixed with other digital data, not necessarily audio. The digital packets are then sent to the destination, but each packet may take a completely different route and may not even arrive at the destination in the correct time order. The digital voice data is then extracted from the packets and assembled into a digital data stream. A DAC converts this into an analog electrical signal, which drives an audio amplifier, which in turn drives a loudspeaker, which finally produces sound. There are several DAC architectures; the suitability of a DAC for a particular application is determined by six main parameters: physical size, power consumption, resolution, speed, accuracy, cost. Due to the complexity and the need for precisely matched components, all but the most specialist DACs are implemented as integrated circuits (ICs). Digital-to-analog conversion can degrade a signal, so a DAC should be specified that that has insignificant errors in terms of the application. DACs are commonly used in music players to convert digital data streams into analog audio signals. They are also used in televisions and mobile phones to convert digital video data into analog video signals which connect to the screen drivers to display monochrome or color images. These two applications use DACs at opposite ends of the speed/resolution trade-off. The audio DAC is a low speed high resolution type while the video DAC is a high speed low to medium resolution type. Discrete DACs would typically be extremely high speed low resolution power

15

Digital-to-analog converter

16

hungry types, as used in military radar systems. Very high speed test equipment, especially sampling oscilloscopes, may also use discrete DACS.

Overview A DAC converts an abstract finite-precision number (usually a fixed-point binary number) into a physical quantity (e.g., a voltage or a pressure). In particular, DACs are often used to convert finite-precision time series data to a continually varying physical signal. A typical DAC converts the abstract numbers into a concrete sequence of impulses that are then processed by a reconstruction filter using some form of interpolation to fill in data between the impulses. Other DAC methods (e.g., methods based on delta-sigma modulation) produce a pulse-density modulated signal that can then be filtered in a similar way to produce a smoothly varying signal.

Ideally sampled signal.

As per the Nyquist–Shannon sampling theorem, a DAC can reconstruct the original signal from the sampled data provided that its bandwidth meets certain requirements (e.g., a baseband signal with bandwidth less than the Nyquist frequency). Digital sampling introduces quantization error that manifests as low-level noise added to the reconstructed signal.

Practical operation Instead of impulses, usually the sequence of numbers update the analog voltage at uniform sampling intervals, which are then often interpolated via a reconstruction filter to continuously varied levels. These numbers are written to the DAC, typically with a clock signal that causes each number to be latched in sequence, at which time the DAC output voltage changes rapidly from the previous value to the value represented by the currently latched number. The effect of this is that the output voltage is held in time at the current value until the next input number is latched, resulting in a piecewise constant or staircase-shaped output. This is equivalent to a zero-order hold operation and has an effect on the frequency response of the reconstructed signal.

Piecewise constant output of an idealized DAC lacking a reconstruction filter. In a practical DAC, a filter or the finite bandwidth of the device smooths out the step response into a continuous curve.

The fact that DACs output a sequence of piecewise constant values (known as zero-order hold in sample data textbooks) or rectangular pulses causes multiple harmonics above the Nyquist frequency. Usually, these are removed with a low pass filter acting as a reconstruction filter in applications that require it.

Digital-to-analog converter

17

Applications Audio Most modern audio signals are stored in digital form (for example MP3s and CDs) and in order to be heard through speakers they must be converted into an analog signal. DACs are therefore found in CD players, digital music players, and PC sound cards.

A simplified functional diagram of an 8-bit DAC

Specialist standalone DACs can also be found in high-end hi-fi systems. These normally take the digital output of a compatible CD player or dedicated transport (which is basically a CD player with no internal DAC) and convert the signal into an analog line-level output that can then be fed into an amplifier to drive speakers. Similar digital-to-analog converters can be found in digital speakers such as USB speakers, and in sound cards. In VoIP (Voice over IP) applications, the source must first be digitized for transmission, so it undergoes conversion via an analog-to-digital converter, and is then reconstructed into analog using a DAC on the receiving party's end.

Video Video sampling tends to work on a completely different scale altogether thanks to the highly nonlinear response both of cathode ray tubes (for which the vast majority of digital video foundation work was targeted) and the human eye, using a "gamma curve" to provide an appearance of evenly distributed brightness steps across the display's full dynamic range - hence the need to use RAMDACs in computer video applications with deep enough colour resolution to make engineering a hardcoded value into the DAC for each output level of Top-loading CD player and external each channel impractical (e.g. an Atari ST or Sega Genesis would digital-to-analog converter. require 24 such values; a 24-bit video card would need 768...). Given this inherent distortion, it is not unusual for a television or video projector to truthfully claim a linear contrast ratio (difference between darkest and brightest output levels) of 1000:1 or greater, equivalent to 10 bits of audio precision even though it may only accept signals with 8-bit precision and use an LCD panel that only represents 6 or 7 bits per channel. Video signals from a digital source, such as a computer, must be converted to analog form if they are to be displayed on an analog monitor. As of 2007, analog inputs were more commonly used than digital, but this changed as flat panel displays with DVI and/or HDMI connections became more widespread.Wikipedia:Citation needed A video DAC is, however, incorporated in any digital video player with analog outputs. The DAC is usually integrated with some memory (RAM), which contains conversion tables for gamma correction, contrast and brightness, to make a device called a RAMDAC. A device that is distantly related to the DAC is the digitally controlled potentiometer, used to control an analog signal digitally.

Digital-to-analog converter

Mechanical An unusual application of digital-to-analog conversion was the whiffletree electromechanical digital-to-analog converter linkage in the IBM Selectric typewriter.Wikipedia:Citation needed

DAC types The most common types of electronic DACs are: • The pulse-width modulator, the simplest DAC type. A stable current or voltage is switched into a low-pass analog filter with a duration determined by the digital input code. This technique is often used for electric motor speed control, but has many other applications as well. • Oversampling DACs or interpolating DACs such as the delta-sigma DAC, use a pulse density conversion technique. The oversampling technique allows for the use of a lower resolution DAC internally. A simple 1-bit DAC is often chosen because the oversampled result is inherently linear. The DAC is driven with a pulse-density modulated signal, created with the use of a low-pass filter, step nonlinearity (the actual 1-bit DAC), and negative feedback loop, in a technique called delta-sigma modulation. This results in an effective high-pass filter acting on the quantization (signal processing) noise, thus steering this noise out of the low frequencies of interest into the megahertz frequencies of little interest, which is called noise shaping. The quantization noise at these high frequencies is removed or greatly attenuated by use of an analog low-pass filter at the output (sometimes a simple RC low-pass circuit is sufficient). Most very high resolution DACs (greater than 16 bits) are of this type due to its high linearity and low cost. Higher oversampling rates can relax the specifications of the output low-pass filter and enable further suppression of quantization noise. Speeds of greater than 100 thousand samples per second (for example, 192 kHz) and resolutions of 24 bits are attainable with delta-sigma DACs. A short comparison with pulse-width modulation shows that a 1-bit DAC with a simple first-order integrator would have to run at 3 THz (which is physically unrealizable) to achieve 24 meaningful bits of resolution, requiring a higher-order low-pass filter in the noise-shaping loop. A single integrator is a low-pass filter with a frequency response inversely proportional to frequency and using one such integrator in the noise-shaping loop is a first order delta-sigma modulator. Multiple higher order topologies (such as MASH) are used to achieve higher degrees of noise-shaping with a stable topology. • The binary-weighted DAC, which contains individual electrical components for each bit of the DAC connected to a summing point. These precise voltages or currents sum to the correct output value. This is one of the fastest conversion methods but suffers from poor accuracy because of the high precision required for each individual voltage or current. Such high-precision components are expensive, so this type of converter is usually limited to 8-bit resolution or less.Wikipedia:Citation needed • Switched resistor DAC contains of a parallel resistor network. Individual resistors are enabled or bypassed in the network based on the digital input. • Switched current source DAC, from which different current sources are selected based on the digital input. • Switched capacitor DAC contains a parallel capacitor network. Individual capacitors are connected or disconnected with switches based on the input. • The R-2R ladder DAC which is a binary-weighted DAC that uses a repeating cascaded structure of resistor values R and 2R. This improves the precision due to the relative ease of producing equal valued-matched resistors (or current sources). However, wide converters perform slowly due to increasingly large RC-constants for each added R-2R link. • The Successive-Approximation or Cyclic DAC, which successively constructs the output during each cycle. Individual bits of the digital input are processed each cycle until the entire input is accounted for. • The thermometer-coded DAC, which contains an equal resistor or current-source segment for each possible value of DAC output. An 8-bit thermometer DAC would have 255 segments, and a 16-bit thermometer DAC would have 65,535 segments. This is perhaps the fastest and highest precision DAC architecture but at the expense of

18

Digital-to-analog converter high cost. Conversion speeds of >1 billion samples per second have been reached with this type of DAC. • Hybrid DACs, which use a combination of the above techniques in a single converter. Most DAC integrated circuits are of this type due to the difficulty of getting low cost, high speed and high precision in one device. • The segmented DAC, which combines the thermometer-coded principle for the most significant bits and the binary-weighted principle for the least significant bits. In this way, a compromise is obtained between precision (by the use of the thermometer-coded principle) and number of resistors or current sources (by the use of the binary-weighted principle). The full binary-weighted design means 0% segmentation, the full thermometer-coded design means 100% segmentation. • Most DACs, shown earlier in this list, rely on a constant reference voltage to create their output value. Alternatively, a multiplying DAC takes a variable input voltage for their conversion. This puts additional design constraints on the bandwidth of the conversion circuit.

DAC performance DACs are very important to system performance. The most important characteristics of these devices are: Resolution The number of possible output levels the DAC is designed to reproduce. This is usually stated as the number of bits it uses, which is the base two logarithm of the number of levels. For instance a 1 bit DAC is designed to reproduce 2 (21) levels while an 8 bit DAC is designed for 256 (28) levels. Resolution is related to the effective number of bits which is a measurement of the actual resolution attained by the DAC. Resolution determines color depth in video applications and audio bit depth in audio applications. Maximum sampling rate A measurement of the maximum speed at which the DACs circuitry can operate and still produce the correct output. As stated in the Nyquist–Shannon sampling theorem defines a relationship between the sampling frequency and bandwidth of the sampled signal. Monotonicity The ability of a DAC's analog output to move only in the direction that the digital input moves (i.e., if the input increases, the output doesn't dip before asserting the correct output.) This characteristic is very important for DACs used as a low frequency signal source or as a digitally programmable trim element. Total harmonic distortion and noise (THD+N) A measurement of the distortion and noise introduced to the signal by the DAC. It is expressed as a percentage of the total power of unwanted harmonic distortion and noise that accompany the desired signal. This is a very important DAC characteristic for dynamic and small signal DAC applications. Dynamic range A measurement of the difference between the largest and smallest signals the DAC can reproduce expressed in decibels. This is usually related to resolution and noise floor. Other measurements, such as phase distortion and jitter, can also be very important for some applications, some of which (e.g. wireless data transmission, composite video) may even rely on accurate production of phase-adjusted signals. Linear PCM audio sampling usually works on the basis of each bit of resolution being equivalent to 6 decibels of amplitude (a 2x increase in volume or precision). Non-linear PCM encodings (A-law / μ-law, ADPCM, NICAM) attempt to improve their effective dynamic ranges by a variety of methods - logarithmic step sizes between the output signal strengths represented by each data bit (trading greater quantisation distortion of loud signals for better performance of quiet signals)

19

Digital-to-analog converter

DAC figures of merit • Static performance: • Differential nonlinearity (DNL) shows how much two adjacent code analog values deviate from the ideal 1 LSB step.[1] • Integral nonlinearity (INL) shows how much the DAC transfer characteristic deviates from an ideal one. That is, the ideal characteristic is usually a straight line; INL shows how much the actual voltage at a given code value differs from that line, in LSBs (1 LSB steps). • Gain • Offset • Noise is ultimately limited by the thermal noise generated by passive components such as resistors. For audio applications and in room temperatures, such noise is usually a little less than 1 μV (microvolt) of white noise. This limits performance to less than 20~21 bits even in 24-bit DACs. • Frequency domain performance • Spurious-free dynamic range (SFDR) indicates in dB the ratio between the powers of the converted main signal and the greatest undesired spur. • Signal-to-noise and distortion ratio (SNDR) indicates in dB the ratio between the powers of the converted main signal and the sum of the noise and the generated harmonic spurs • i-th harmonic distortion (HDi) indicates the power of the i-th harmonic of the converted main signal • Total harmonic distortion (THD) is the sum of the powers of all HDi • If the maximum DNL error is less than 1 LSB, then the D/A converter is guaranteed to be monotonic. However, many monotonic converters may have a maximum DNL greater than 1 LSB. • Time domain performance: • Glitch impulse area (glitch energy) • Response uncertainty • Time nonlinearity (TNL)

References [1] ADC and DAC Glossary - Maxim (http:/ / www. maxim-ic. com/ appnotes. cfm/ appnote_number/ 641/ )

Further reading • Kester, Walt, The Data Conversion Handbook (http://www.analog.com/library/analogDialogue/archives/ 39-06/data_conversion_handbook.html), ISBN 0-7506-7841-0 • S. Norsworthy, Richard Schreier, Gabor C. Temes, Delta-Sigma Data Converters. ISBN 0-7803-1045-4. • Mingliang Liu, Demystifying Switched-Capacitor Circuits. ISBN 0-7506-7907-7. • Behzad Razavi, Principles of Data Conversion System Design. ISBN 0-7803-1093-4. • Phillip E. Allen, Douglas R. Holberg, CMOS Analog Circuit Design. ISBN 0-19-511644-5. • Robert F. Coughlin, Frederick F. Driscoll, Operational Amplifiers and Linear Integrated Circuits. ISBN 0-13-014991-8. • A Anand Kumar, Fundamentals of Digital Circuits. ISBN 81-203-1745-9, ISBN 978-81-203-1745-1.

20

Digital-to-analog converter

21

External links • ADC and DAC Glossary (http://www.maxim-ic.com/appnotes.cfm/an_pk/641/CMP/WP-36)

Analog-to-digital converter An analog-to-digital converter (abbreviated ADC, A/D or A to D) is a device that converts a continuous physical quantity (usually voltage) to a digital number that represents the quantity's amplitude. The conversion involves quantization of the input, so it necessarily introduces a small amount of error. Instead of doing a single conversion, an ADC often performs the conversions ("samples" the input) periodically. The result is a sequence of digital values that have been converted from a continuous-time and continuous-amplitude analog signal to a discrete-time and discrete-amplitude digital signal.

4-channel stereo multiplexed analog-to-digital converter WM8775SEDS made by Wolfson Microelectronics placed on an X-Fi Fatal1ty Pro sound card.

An ADC is defined by its bandwidth (the range of frequencies it can measure) and its signal to noise ratio (how accurately it can measure a signal relative to the noise it introduces). The actual bandwidth of an ADC is characterized primarily by its sampling rate, and to a lesser extent by how it handles errors such as aliasing. The dynamic range of an ADC is influenced by many factors, including the resolution (the number of output levels it can quantize a signal to), linearity and accuracy (how well the quantization levels match the true analog signal) and jitter (small timing errors that introduce additional noise). The dynamic range of an ADC is often summarized in terms of its effective number of bits (ENOB), the number of bits of each measure it returns that are on average not noise. An ideal ADC has an ENOB equal to its resolution. ADCs are chosen to match the bandwidth and required signal to noise ratio of the signal to be quantized. If an ADC operates at a sampling rate greater than twice the bandwidth of the signal, then perfect reconstruction is possible given an ideal ADC and neglecting quantization error. The presence of quantization error limits the dynamic range of even an ideal ADC, however, if the dynamic range of the ADC exceeds that of the input signal, its effects may be neglected resulting in an essentially perfect digital representation of the input signal. An ADC may also provide an isolated measurement such as an electronic device that converts an input analog voltage or current to a digital number proportional to the magnitude of the voltage or current. However, some non-electronic or only partially electronic devices, such as rotary encoders, can also be considered ADCs. The digital output may use different coding schemes. Typically the digital output will be a two's complement binary number that is proportional to the input, but there are other possibilities. An encoder, for example, might output a Gray code. The inverse operation is performed by a digital-to-analog converter (DAC).

Analog-to-digital converter

22

Concepts Resolution The resolution of the converter indicates the number of discrete values it can produce over the range of analog values. The resolution determines the magnitude of the quantization error and therefore determines the maximum possible average signal to noise ratio for an ideal ADC without the use of oversampling. The values are usually stored electronically in binary form, so the resolution is usually expressed in bits. In consequence, the number of discrete values available, or "levels", is assumed to be a power of two. For example, an ADC with a resolution of 8 bits can encode an analog input to one in 256 different levels, since 28 = 256. The values can represent the ranges from 0 to 255 (i.e. unsigned integer) or from −128 to 127 (i.e. signed integer), depending on the application.

Fig. 1. An 8-level ADC coding scheme.

Resolution can also be defined electrically, and expressed in volts. The minimum change in voltage required to guarantee a change in the output code level is called the least significant bit (LSB) voltage. The resolution Q of the ADC is equal to the LSB voltage. The voltage resolution of an ADC is equal to its overall voltage measurement range divided by the number of discrete values:

where M is the ADC's resolution in bits and EFSR is the full scale voltage range (also called 'span'). EFSR is given by where VRefHi and VRefLow are the upper and lower extremes, respectively, of the voltages that can be coded. Normally, the number of voltage intervals is given by

where M is the ADC's resolution in bits. That is, one voltage interval is assigned in between two consecutive code levels. Example: • • • •

Coding scheme as in figure 1 (assume input signal x(t) = Acos(t), A = 5V) Full scale measurement range = -5 to 5 volts ADC resolution is 8 bits: 28 = 256 quantization levels (codes) ADC voltage resolution, Q = (10 V − 0 V) / 256 = 10 V / 256 ≈ 0.039 V ≈ 39 mV.

In practice, the useful resolution of a converter is limited by the best signal-to-noise ratio (SNR) that can be achieved for a digitized signal. An ADC can resolve a signal to only a certain number of bits of resolution, called the effective number of bits (ENOB). One effective bit of resolution changes the signal-to-noise ratio of the digitized signal by 6 dB, if the resolution is limited by the ADC. If a preamplifier has been used prior to A/D conversion, the noise introduced by the amplifier can be an important contributing factor towards the overall SNR.

Analog-to-digital converter

23

Quantization error Main article: Quantization error Quantization error is the noise introduced by quantization in an ideal ADC. It is a rounding error between the analog input voltage to the ADC and the output digitized value. The noise is non-linear and signal-dependent. In an ideal analog-to-digital converter, where the quantization error is uniformly distributed between −1/2 LSB and +1/2 LSB, and the signal has a uniform distribution covering all quantization levels, the Signal-to-quantization-noise ratio (SQNR) can be calculated from

Comparison of quantizing a sinusoid to 64 levels (6 bits) and 256 levels (8 bits). The additive noise created by 6-bit quantization is 12 dB greater than the noise created by 8-bit quantization. When the spectral distribution is flat, as in this example, the 12 dB difference manifests as a measurable difference in the noise floors.

Where Q is the number of quantization bits. For example, a 16-bit ADC has a maximum signal-to-noise ratio of 6.02 × 16 = 96.3 dB, and therefore the quantization error is 96.3 dB below the maximum level. Quantization error is distributed from DC to the Nyquist frequency, consequently if part of the ADC's bandwidth is not used (as in oversampling), some of the quantization error will fall out of band, effectively improving the SQNR. In an oversampled system, noise shaping can be used to further increase SQNR by forcing more quantization error out of the band. Dither Main article: dither In ADCs, performance can usually be improved using dither. This is a very small amount of random noise (white noise), which is added to the input before conversion. Its effect is to cause the state of the LSB to randomly oscillate between 0 and 1 in the presence of very low levels of input, rather than sticking at a fixed value. Rather than the signal simply getting cut off altogether at this low level (which is only being quantized to a resolution of 1 bit), it extends the effective range of signals that the ADC can convert, at the expense of a slight increase in noise – effectively the quantization error is diffused across a series of noise values which is far less objectionable than a hard cutoff. The result is an accurate representation of the signal over time. A suitable filter at the output of the system can thus recover this small signal variation. An audio signal of very low level (with respect to the bit depth of the ADC) sampled without dither sounds extremely distorted and unpleasant. Without dither the low level may cause the least significant bit to "stick" at 0 or 1. With dithering, the true level of the audio may be calculated by averaging the actual quantized sample with a series of other samples [the dither] that are recorded over time. A virtually identical process, also called dither or dithering, is often used when quantizing photographic images to a fewer number of bits per pixel—the image becomes noisier but to the eye looks far more realistic than the quantized image, which otherwise becomes banded. This analogous process may help to visualize the effect of dither on an analogue audio signal that is converted to digital. Dithering is also used in integrating systems such as electricity meters. Since the values are added together, the dithering produces results that are more exact than the LSB of the analog-to-digital converter.

Analog-to-digital converter

24

Note that dither can only increase the resolution of a sampler, it cannot improve the linearity, and thus accuracy does not necessarily improve.

Accuracy An ADC has several sources of errors. Quantization error and (assuming the ADC is intended to be linear) non-linearity are intrinsic to any analog-to-digital conversion. These errors are measured in a unit called the least significant bit (LSB). In the above example of an eight-bit ADC, an error of one LSB is 1/256 of the full signal range, or about 0.4%. Non-linearity All ADCs suffer from non-linearity errors caused by their physical imperfections, causing their output to deviate from a linear function (or some other function, in the case of a deliberately non-linear ADC) of their input. These errors can sometimes be mitigated by calibration, or prevented by testing. Important parameters for linearity are integral non-linearity (INL) and differential non-linearity (DNL). These non-linearities reduce the dynamic range of the signals that can be digitized by the ADC, also reducing the effective resolution of the ADC.

Jitter When digitizing a sine wave

, the use of a non-ideal sampling clock will result in some

uncertainty in when samples are recorded. Provided that the actual sampling time uncertainty due to the clock jitter is

, the error caused by this phenomenon can be estimated as

. This will

result in additional recorded noise that will reduce the effective number of bits (ENOB) below that predicted by quantization error alone. The error is zero for DC, small at low frequencies, but significant when high frequencies have high amplitudes. This effect can be ignored if it is drowned out by the quantizing error. Jitter requirements can be calculated using the following formula:

, where q is the number of ADC bits. Output size (bits)

Signal Frequency 1 Hz

1 kHz 10 kHz 1 MHz 10 MHz 100 MHz 1 GHz

8 1,243 µs 1.24 µs

124 ns 1.24 ns

311 ns 31.1 ns

124 ps

12.4 ps 1.24 ps

10

311 µs

311 ps

31.1 ps

3.11 ps 0.31 ps

12

77.7 µs 77.7 ns 7.77 ns 77.7 ps

7.77 ps

0.78 ps 0.08 ps

14

19.4 µs 19.4 ns 1.94 ns 19.4 ps

1.94 ps

0.19 ps 0.02 ps

16

4.86 µs 4.86 ns

486 ps 4.86 ps

0.49 ps

0.05 ps

18

1.21 ns

121 ps 6.32 ps 1.21 ps

0.12 ps

–

–

20

304 ps 30.4 ps 1.58 ps 0.16 ps

–

–

–

–

Clock jitter is caused by phase noise.[1] The resolution of ADCs with a digitization bandwidth between 1 MHz and 1 GHz is limited by jitter. When sampling audio signals at 44.1 kHz, the anti-aliasing filter should have eliminated all frequencies above 22 kHz. The input frequency (in this case, < 22 kHz kHz), not the ADC clock frequency, is the determining factor with respect to jitter performance.[2]

Analog-to-digital converter

Sampling rate Main article: Sampling rate See also: Sampling (signal processing) The analog signal is continuous in time and it is necessary to convert this to a flow of digital values. It is therefore required to define the rate at which new digital values are sampled from the analog signal. The rate of new values is called the sampling rate or sampling frequency of the converter. A continuously varying bandlimited signal can be sampled (that is, the signal values at intervals of time T, the sampling time, are measured and stored) and then the original signal can be exactly reproduced from the discrete-time values by an interpolation formula. The accuracy is limited by quantization error. However, this faithful reproduction is only possible if the sampling rate is higher than twice the highest frequency of the signal. This is essentially what is embodied in the Shannon-Nyquist sampling theorem. Since a practical ADC cannot make an instantaneous conversion, the input value must necessarily be held constant during the time that the converter performs a conversion (called the conversion time). An input circuit called a sample and hold performs this task—in most cases by using a capacitor to store the analog voltage at the input, and using an electronic switch or gate to disconnect the capacitor from the input. Many ADC integrated circuits include the sample and hold subsystem internally. Aliasing Main article: Aliasing See also: Undersampling An ADC works by sampling the value of the input at discrete intervals in time. Provided that the input is sampled above the Nyquist rate, defined as twice the highest frequency of interest, then all frequencies in the signal can be reconstructed. If frequencies above half the Nyquist rate are sampled, they are incorrectly detected as lower frequencies, a process referred to as aliasing. Aliasing occurs because instantaneously sampling a function at two or fewer times per cycle results in missed cycles, and therefore the appearance of an incorrectly lower frequency. For example, a 2 kHz sine wave being sampled at 1.5 kHz would be reconstructed as a 500 Hz sine wave. To avoid aliasing, the input to an ADC must be low-pass filtered to remove frequencies above half the sampling rate. This filter is called an anti-aliasing filter, and is essential for a practical ADC system that is applied to analog signals with higher frequency content. In applications where protection against aliasing is essential, oversampling may be used to greatly reduce or even eliminate it. Although aliasing in most systems is unwanted, it should also be noted that it can be exploited to provide simultaneous down-mixing of a band-limited high frequency signal (see undersampling and frequency mixer). The alias is effectively the lower heterodyne of the signal frequency and sampling frequency. Oversampling Main article: Oversampling Signals are often sampled at the minimum rate required, for economy, with the result that the quantization noise introduced is white noise spread over the whole pass band of the converter. If a signal is sampled at a rate much higher than the Nyquist frequency and then digitally filtered to limit it to the signal bandwidth there are the following advantages: • digital filters can have better properties (sharper rolloff, phase) than analogue filters, so a sharper anti-aliasing filter can be realised and then the signal can be downsampled giving a better result • a 20-bit ADC can be made to act as a 24-bit ADC with 256× oversampling • the signal-to-noise ratio due to quantization noise will be higher than if the whole available band had been used. With this technique, it is possible to obtain an effective resolution larger than that provided by the converter alone

25

Analog-to-digital converter • The improvement in SNR is 3 dB (equivalent to 0.5 bits) per octave of oversampling which is not sufficient for many applications. Therefore, oversampling is usually coupled with noise shaping (see sigma-delta modulators). With noise shaping, the improvement is 6L+3 dB per octave where L is the order of loop filter used for noise shaping. e.g. – a 2nd order loop filter will provide an improvement of 15 dB/octave. Oversampling is typically used in audio frequency ADCs where the required sampling rate (typically 44.1 or 48 kHz) is very low compared to the clock speed of typical transistor circuits (>1 MHz). In this case, by using the extra bandwidth to distribute quantization error onto out of band frequencies, the accuracy of the ADC can be greatly increased at no cost. Furthermore, as any aliased signals are also typically out of band, aliasing can often be completely eliminated using very low cost filters.

Relative speed and precision The speed of an ADC varies by type. The Wilkinson ADC is limited by the clock rate which is processable by current digital circuits. Currently,Wikipedia:Manual of Style/Dates and numbers#Chronological items frequencies up to 300 MHz are possible.[3] For a successive-approximation ADC, the conversion time scales with the logarithm of the resolution, e.g. the number of bits. Thus for high resolution, it is possible that the successive-approximation ADC is faster than the Wilkinson. However, the time consuming steps in the Wilkinson are digital, while those in the successive-approximation are analog. Since analog is inherently slower than digital, as the resolution increases, the time required also increases. Thus there are competing processes at work. Flash ADCs are certainly the fastest type of the three. The conversion is basically performed in a single parallel step. For an 8-bit unit, conversion takes place in a few tens of nanoseconds. There is, as expected, somewhat of a tradeoff between speed and precision. Flash ADCs have drifts and uncertainties associated with the comparator levels. This results in poor linearity. For successive-approximation ADCs, poor linearity is also present, but less so than for flash ADCs. Here, non-linearity arises from accumulating errors from the subtraction processes. Wilkinson ADCs have the highest linearity of the three. These have the best differential non-linearity. The other types require channel smoothing to achieve the level of the Wilkinson.

The sliding scale principle The sliding scale or randomizing method can be employed to greatly improve the linearity of any type of ADC, but especially flash and successive approximation types. For any ADC the mapping from input voltage to digital output value is not exactly a floor or ceiling function as it should be. Under normal conditions, a pulse of a particular amplitude is always converted to a digital value. The problem lies in that the ranges of analog values for the digitized values are not all of the same width, and the differential linearity decreases proportionally with the divergence from the average width. The sliding scale principle uses an averaging effect to overcome this phenomenon. A random, but known analog voltage is added to the sampled input voltage. It is then converted to digital form, and the equivalent digital amount is subtracted, thus restoring it to its original value. The advantage is that the conversion has taken place at a random point. The statistical distribution of the final levels is decided by a weighted average over a region of the range of the ADC. This in turn desensitizes it to the width of any specific level.

ADC types These are the most common ways of implementing an electronic ADC: • A direct-conversion ADC or flash ADC has a bank of comparators sampling the input signal in parallel, each firing for their decoded voltage range. The comparator bank feeds a logic circuit that generates a code for each voltage range. Direct conversion is very fast, capable of gigahertz sampling rates, but usually has only 8 bits of resolution or fewer, since the number of comparators needed, 2N – 1, doubles with each additional bit, requiring a large, expensive circuit. ADCs of this type have a large die size, a high input capacitance, high power dissipation, and are prone to produce glitches at the output (by outputting an out-of-sequence code). Scaling to newer

26

Analog-to-digital converter submicrometre technologies does not help as the device mismatch is the dominant design limitation. They are often used for video, wideband communications or other fast signals in optical storage. • A successive-approximation ADC uses a comparator to successively narrow a range that contains the input voltage. At each successive step, the converter compares the input voltage to the output of an internal digital to analog converter which might represent the midpoint of a selected voltage range. At each step in this process, the approximation is stored in a successive approximation register (SAR). For example, consider an input voltage of 6.3 V and the initial range is 0 to 16 V. For the first step, the input 6.3 V is compared to 8 V (the midpoint of the 0–16 V range). The comparator reports that the input voltage is less than 8 V, so the SAR is updated to narrow the range to 0–8 V. For the second step, the input voltage is compared to 4 V (midpoint of 0–8). The comparator reports the input voltage is above 4 V, so the SAR is updated to reflect the input voltage is in the range 4–8 V. For the third step, the input voltage is compared with 6 V (halfway between 4 V and 8 V); the comparator reports the input voltage is greater than 6 volts, and search range becomes 6–8 V. The steps are continued until the desired resolution is reached. • A ramp-compare ADC produces a saw-tooth signal that ramps up or down then quickly returns to zero. When the ramp starts, a timer starts counting. When the ramp voltage matches the input, a comparator fires, and the timer's value is recorded. Timed ramp converters require the least number of transistors. The ramp time is sensitive to temperature because the circuit generating the ramp is often a simple oscillator. There are two solutions: use a clocked counter driving a DAC and then use the comparator to preserve the counter's value, or calibrate the timed ramp. A special advantage of the ramp-compare system is that comparing a second signal just requires another comparator, and another register to store the voltage value. A very simple (non-linear) ramp-converter can be implemented with a microcontroller and one resistor and capacitor.[4] Vice versa, a filled capacitor can be taken from an integrator, time-to-amplitude converter, phase detector, sample and hold circuit, or peak and hold circuit and discharged. This has the advantage that a slow comparator cannot be disturbed by fast input changes. • The Wilkinson ADC was designed by D. H. Wilkinson in 1950. The Wilkinson ADC is based on the comparison of an input voltage with that produced by a charging capacitor. The capacitor is allowed to charge until its voltage is equal to the amplitude of the input pulse (a comparator determines when this condition has been reached). Then, the capacitor is allowed to discharge linearly, which produces a ramp voltage. At the point when the capacitor begins to discharge, a gate pulse is initiated. The gate pulse remains on until the capacitor is completely discharged. Thus the duration of the gate pulse is directly proportional to the amplitude of the input pulse. This gate pulse operates a linear gate which receives pulses from a high-frequency oscillator clock. While the gate is open, a discrete number of clock pulses pass through the linear gate and are counted by the address register. The time the linear gate is open is proportional to the amplitude of the input pulse, thus the number of clock pulses recorded in the address register is proportional also. Alternatively, the charging of the capacitor could be monitored, rather than the discharge. • An integrating ADC (also dual-slope or multi-slope ADC) applies the unknown input voltage to the input of an integrator and allows the voltage to ramp for a fixed time period (the run-up period). Then a known reference voltage of opposite polarity is applied to the integrator and is allowed to ramp until the integrator output returns to zero (the run-down period). The input voltage is computed as a function of the reference voltage, the constant run-up time period, and the measured run-down time period. The run-down time measurement is usually made in units of the converter's clock, so longer integration times allow for higher resolutions. Likewise, the speed of the converter can be improved by sacrificing resolution. Converters of this type (or variations on the concept) are used in most digital voltmeters for their linearity and flexibility. • A delta-encoded ADC or counter-ramp has an up-down counter that feeds a digital to analog converter (DAC). The input signal and the DAC both go to a comparator. The comparator controls the counter. The circuit uses negative feedback from the comparator to adjust the counter until the DAC's output is close enough to the input signal. The number is read from the counter. Delta converters have very wide ranges and high resolution, but the

27

Analog-to-digital converter conversion time is dependent on the input signal level, though it will always have a guaranteed worst-case. Delta converters are often very good choices to read real-world signals. Most signals from physical systems do not change abruptly. Some converters combine the delta and successive approximation approaches; this works especially well when high frequencies are known to be small in magnitude. • A pipeline ADC (also called subranging quantizer) uses two or more steps of subranging. First, a coarse conversion is done. In a second step, the difference to the input signal is determined with a digital to analog converter (DAC). This difference is then converted finer, and the results are combined in a last step. This can be considered a refinement of the successive-approximation ADC wherein the feedback reference signal consists of the interim conversion of a whole range of bits (for example, four bits) rather than just the next-most-significant bit. By combining the merits of the successive approximation and flash ADCs this type is fast, has a high resolution, and only requires a small die size. • A sigma-delta ADC (also known as a delta-sigma ADC) oversamples the desired signal by a large factor and filters the desired signal band. Generally, a smaller number of bits than required are converted using a Flash ADC after the filter. The resulting signal, along with the error generated by the discrete levels of the Flash, is fed back and subtracted from the input to the filter. This negative feedback has the effect of noise shaping the error due to the Flash so that it does not appear in the desired signal frequencies. A digital filter (decimation filter) follows the ADC which reduces the sampling rate, filters off unwanted noise signal and increases the resolution of the output (sigma-delta modulation, also called delta-sigma modulation). • A time-interleaved ADC uses M parallel ADCs where each ADC samples data every M:th cycle of the effective sample clock. The result is that the sample rate is increased M times compared to what each individual ADC can manage. In practice, the individual differences between the M ADCs degrade the overall performance reducing the SFDR. However, technologies exist to correct for these time-interleaving mismatch errors. • An ADC with intermediate FM stage first uses a voltage-to-frequency converter to convert the desired signal into an oscillating signal with a frequency proportional to the voltage of the desired signal, and then uses a frequency counter to convert that frequency into a digital count proportional to the desired signal voltage. Longer integration times allow for higher resolutions. Likewise, the speed of the converter can be improved by sacrificing resolution. The two parts of the ADC may be widely separated, with the frequency signal passed through an opto-isolator or transmitted wirelessly. Some such ADCs use sine wave or square wave frequency modulation; others use pulse-frequency modulation. Such ADCs were once the most popular way to show a digital display of the status of a remote analog sensor.[5][6][7][8][9] There can be other ADCs that use a combination of electronics and other technologies: • A time-stretch analog-to-digital converter (TS-ADC) digitizes a very wide bandwidth analog signal, that cannot be digitized by a conventional electronic ADC, by time-stretching the signal prior to digitization. It commonly uses a photonic preprocessor frontend to time-stretch the signal, which effectively slows the signal down in time and compresses its bandwidth. As a result, an electronic backend ADC, that would have been too slow to capture the original signal, can now capture this slowed down signal. For continuous capture of the signal, the frontend also divides the signal into multiple segments in addition to time-stretching. Each segment is individually digitized by a separate electronic ADC. Finally, a digital signal processor rearranges the samples and removes any distortions added by the frontend to yield the binary data that is the digital representation of the original analog signal.

28

Analog-to-digital converter

Commercial analog-to-digital converters Commercial ADCs are usually implemented as integrated circuits. Most converters sample with 6 to 24 bits of resolution, and produce fewer than 1 megasample per second. Thermal noise generated by passive components such as resistors masks the measurement when higher resolution is desired. For audio applications and in room temperatures, such noise is usually a little less than 1 μV (microvolt) of white noise. If the MSB corresponds to a standard 2 V of output signal, this translates to a noise-limited performance that is less than 20~21 bits, and obviates the need for any dithering. As of February 2002, Mega- and giga-sample per second converters are available. Mega-sample converters are required in digital video cameras, video capture cards, and TV tuner cards to convert full-speed analog video to digital video files. Commercial converters usually have ±0.5 to ±1.5 LSB error in their output. In many cases, the most expensive part of an integrated circuit is the pins, because they make the package larger, and each pin has to be connected to the integrated circuit's silicon. To save pins, it is common for slow ADCs to send their data one bit at a time over a serial interface to the computer, with the next bit coming out when a clock signal changes state, say from 0 to 5 V. This saves quite a few pins on the ADC package, and in many cases, does not make the overall design any more complex (even microprocessors which use memory-mapped I/O only need a few bits of a port to implement a serial bus to an ADC). Commercial ADCs often have several inputs that feed the same converter, usually through an analog multiplexer. Different models of ADC may include sample and hold circuits, instrumentation amplifiers or differential inputs, where the quantity measured is the difference between two voltages.

Applications Music recording Analog-to-digital converters are integral to current music reproduction technology. People produce much music on computers using an analog recording and therefore need analog-to-digital converters to create the pulse-code modulation (PCM) data streams that go onto compact discs and digital music files. The current crop of analog-to-digital converters utilized in music can sample at rates up to 192 kilohertz. Considerable literature exists on these matters, but commercial considerations often play a significant role. MostWikipedia:Citation needed high-profile recording studios record in 24-bit/192-176.4 kHz pulse-code modulation (PCM) or in Direct Stream Digital (DSD) formats, and then downsample or decimate the signal for Red-Book CD production (44.1 kHz) or to 48 kHz for commonly used radio and television broadcast applications.

Digital signal processing People must use ADCs to process, store, or transport virtually any analog signal in digital form. TV tuner cards, for example, use fast video analog-to-digital converters. Slow on-chip 8, 10, 12, or 16 bit analog-to-digital converters are common in microcontrollers. Digital storage oscilloscopes need very fast analog-to-digital converters, also crucial for software defined radio and their new applications.

Scientific instruments Digital imaging systems commonly use analog-to-digital converters in digitizing pixels. Some radar systems commonly use analog-to-digital converters to convert signal strength to digital values for subsequent signal processing. Many other in situ and remote sensing systems commonly use analogous technology. The number of binary bits in the resulting digitized numeric values reflects the resolution, the number of unique discrete levels of quantization (signal processing). The correspondence between the analog signal and the digital

29

Analog-to-digital converter signal depends on the quantization error. The quantization process must occur at an adequate speed, a constraint that may limit the resolution of the digital signal. Many sensors produce an analog signal; temperature, pressure, pH, light intensity etc. All these signals can be amplified and fed to an ADC to produce a digital number proportional to the input signal.

Electrical Symbol

Testing Testing an Analog to Digital Converter requires an analog input source, hardware to send control signals and capture digital data output. Some ADCs also require an accurate source of reference signal. The key parameters to test a SAR ADC are following: 1. 2. 3. 4. 5. 6. 7. 8.

DC Offset Error DC Gain Error Signal to Noise Ratio (SNR) Total Harmonic Distortion (THD) Integral Non Linearity (INL) Differential Non Linearity (DNL) Spurious Free Dynamic Range Power Dissipation

Notes [1] Maxim App 800: "Design a Low-Jitter Clock for High-Speed Data Converters" (http:/ / www. maxim-ic. com/ appnotes. cfm/ an_pk/ 800/ ). maxim-ic.com (July 17, 2002). [2] Redmayne, Derek and Steer, Alison (8 December 2008) Understanding the effect of clock jitter on high-speed ADCs (http:/ / www. eetimes. com/ design/ automotive-design/ 4010074/ Understanding-the-effect-of-clock-jitter-on-high-speed-ADCs-Part-1-of-2-). eetimes.com [3] 310 Msps ADC by Linear Technology, http:/ / www. linear. com/ product/ LTC2158-14. [4] Atmel Application Note AVR400: Low Cost A/D Converter (http:/ / www. atmel. com/ dyn/ resources/ prod_documents/ doc0942. pdf). atmel.com [5] Analog Devices MT-028 Tutorial: "Voltage-to-Frequency Converters" (http:/ / www. analog. com/ static/ imported-files/ tutorials/ MT-028. pdf) by Walt Kester and James Bryant 2009, apparently adapted from Kester, Walter Allan (2005) Data conversion handbook (http:/ / books. google. com/ books?id=0aeBS6SgtR4C& pg=RA2-PA274), Newnes, p. 274, ISBN 0750678410. [6] Microchip AN795 "Voltage to Frequency / Frequency to Voltage Converter" (http:/ / ww1. microchip. com/ downloads/ en/ AppNotes/ 00795a. pdf) p. 4: "13-bit A/D converter" [7] Carr, Joseph J. (1996) Elements of electronic instrumentation and measurement (http:/ / books. google. com/ books?id=1yBTAAAAMAAJ), Prentice Hall, p. 402, ISBN 0133416860. [8] "Voltage-to-Frequency Analog-to-Digital Converters" (http:/ / www. globalspec. com/ reference/ 3127/ Voltage-to-Frequency-Analog-to-Digital-Converters). globalspec.com [9] Pease, Robert A. (1991) Troubleshooting Analog Circuits (http:/ / books. google. com/ books?id=3kY4-HYLqh0C& pg=PA130), Newnes, p. 130, ISBN 0750694998.

30

Analog-to-digital converter

References • Knoll, Glenn F. (1989). Radiation Detection and Measurement (2nd ed.). New York: John Wiley & Sons. ISBN 0471815047. • Nicholson, P. W. (1974). Nuclear Electronics. New York: John Wiley & Sons. pp. 315–316. ISBN 0471636975.

Further reading • Allen, Phillip E.; Holberg, Douglas R. CMOS Analog Circuit Design. ISBN 0-19-511644-5. • Fraden, Jacob (2010). Handbook of Modern Sensors: Physics, Designs, and Applications. Springer. ISBN 978-1441964656. • Kester, Walt, ed. (2005). The Data Conversion Handbook (http://www.analog.com/library/analogDialogue/ archives/39-06/data_conversion_handbook.html). Elsevier: Newnes. ISBN 0-7506-7841-0. • Johns, David; Martin, Ken. Analog Integrated Circuit Design. ISBN 0-471-14448-7. • Liu, Mingliang. Demystifying Switched-Capacitor Circuits. ISBN 0-7506-7907-7. • Norsworthy, Steven R.; Schreier, Richard; Temes, Gabor C. (1997). Delta-Sigma Data Converters. IEEE Press. ISBN 0-7803-1045-4. • Razavi, Behzad (1995). Principles of Data Conversion System Design. New York, NY: IEEE Press. ISBN 0-7803-1093-4. • Staller, Len (February 24, 2005). "Understanding analog to digital converter specifications" (http://www. embedded.com/design/configurable-systems/4025078/ Understanding-analog-to-digital-converter-specifications). Embedded Systems Design. • Walden, R. H. (1999). "Analog-to-digital converter survey and analysis". IEEE Journal on Selected Areas in Communications 17 (4): 539–550. doi: 10.1109/49.761034 (http://dx.doi.org/10.1109/49.761034).

External links • Counting Type ADC (http://ikalogic.com/tut_adc.php) A simple tutorial showing how to build your first ADC. • An Introduction to Delta Sigma Converters (http://www.beis.de/Elektronik/DeltaSigma/DeltaSigma.html) A very nice overview of Delta-Sigma converter theory. • Digital Dynamic Analysis of A/D Conversion Systems through Evaluation Software based on FFT/DFT Analysis (http://www.ieee.li/pdf/adc_evaluation_rf_expo_east_1987.pdf) RF Expo East, 1987 • Which ADC Architecture Is Right for Your Application? (http://www.analog.com/library/analogDialogue/ archives/39-06/architecture.html) article by Walt Kester • ADC and DAC Glossary (http://www.maxim-ic.com/appnotes.cfm/an_pk/641/CMP/ELK-11) Defines commonly used technical terms. • Introduction to ADC in AVR (http://www.robotplatform.com/knowledge/ADC/adc_tutorial.html) – Analog to digital conversion with Atmel microcontrollers • Signal processing and system aspects of time-interleaved ADCs. (http://userver.ftw.at/~vogel/TIADC.html) • Explanation of analog-digital converters with interactive principles of operations. (http://www.onmyphd.com/ ?p=analog.digital.converter)

31

Window function

32

Window function For the term used in SQL statements, see Window function (SQL) In signal processing, a window function (also known as an apodization function or tapering function) is a mathematical function that is zero-valued outside of some chosen interval. For instance, a function that is constant inside the interval and zero elsewhere is called a rectangular window, which describes the shape of its graphical representation. When another function or waveform/data-sequence is multiplied by a window function, the product is also zero-valued outside the interval: all that is left is the part where they overlap, the "view through the window". Applications of window functions include spectral analysis, filter design, and beamforming. In typical applications, the window functions used are non-negative smooth "bell-shaped" curves, though rectangle, triangle, and other functions can be used. A more general definition of window functions does not require them to be identically zero outside an interval, as long as the product of the window multiplied by its argument is square integrable, and, more specifically, that the function goes sufficiently rapidly toward zero.

Applications Applications of window functions include spectral analysis and the design of finite impulse response filters.

Spectral analysis The Fourier transform of the function cos ωt is zero, except at frequency ±ω. However, many other functions and waveforms do not have convenient closed form transforms. Alternatively, one might be interested in their spectral content only during a certain time period. In either case, the Fourier transform (or something similar) can be applied on one or more finite intervals of the waveform. In general, the transform is applied to the product of the waveform and a window function. Any window (including rectangular) affects the spectral estimate computed by this method. Windowing Windowing of a simple waveform like cos ωt causes its Fourier transform to develop non-zero values (commonly called spectral leakage) at frequencies other than ω. The leakage tends to be worst (highest) near ω and least at frequencies farthest from ω. If the waveform under analysis comprises two sinusoids of different frequencies, leakage can interfere with the ability to distinguish them spectrally. If their frequencies are dissimilar and one component is weaker, then leakage from the larger component can obscure the weaker

Figure 1: Zoomed view of spectral leakage

Window function one’s presence. But if the frequencies are similar, leakage can render them unresolvable even when the sinusoids are of equal strength. The rectangular window has excellent resolution characteristics for sinusoids of comparable strength, but it is a poor choice for sinusoids of disparate amplitudes. This characteristic is sometimes described as low-dynamic-range. At the other extreme of dynamic range are the windows with the poorest resolution. These high-dynamic-range low-resolution windows are also poorest in terms of sensitivity; this is, if the input waveform contains random noise close to the frequency of a sinusoid, the response to noise, compared to the sinusoid, will be higher than with a higher-resolution window. In other words, the ability to find weak sinusoids amidst the noise is diminished by a high-dynamic-range window. High-dynamic-range windows are probably most often justified in wideband applications, where the spectrum being analyzed is expected to contain many different components of various amplitudes. In between the extremes are moderate windows, such as Hamming and Hann. They are commonly used in narrowband applications, such as the spectrum of a telephone channel. In summary, spectral analysis involves a tradeoff between resolving comparable strength components with similar frequencies and resolving disparate strength components with dissimilar frequencies. That tradeoff occurs when the window function is chosen. Discrete-time signals When the input waveform is time-sampled, instead of continuous, the analysis is usually done by applying a window function and then a discrete Fourier transform (DFT). But the DFT provides only a coarse sampling of the actual DTFT spectrum. Figure 1 shows a portion of the DTFT for a rectangularly windowed sinusoid. The actual frequency of the sinusoid is indicated as "0" on the horizontal axis. Everything else is leakage, exaggerated by the use of a logarithmic presentation. The unit of frequency is "DFT bins"; that is, the integer values on the frequency axis correspond to the frequencies sampled by the DFT. So the figure depicts a case where the actual frequency of the sinusoid happens to coincide with a DFT sample,[1] and the maximum value of the spectrum is accurately measured by that sample. When it misses the maximum value by some amount (up to 1/2 bin), the measurement error is referred to as scalloping loss (inspired by the shape of the peak). But the most interesting thing about this case is that all the other samples coincide with nulls in the true spectrum. (The nulls are actually zero-crossings, which cannot be shown on a logarithmic scale such as this.) So in this case, the DFT creates the illusion of no leakage. Despite the unlikely conditions of this example, it is a common misconception that visible leakage is some sort of artifact of the DFT. But since any window function causes leakage, its apparent absence (in this contrived example) is actually the DFT artifact.

33

Window function

Noise bandwidth The concepts of resolution and dynamic range tend to be somewhat subjective, depending on what the user is actually trying to do. But they also tend to be highly correlated with the total leakage, which is quantifiable. It is usually expressed as an equivalent bandwidth, B. It can be thought of as redistributing the DTFT into a rectangular shape with height equal to the spectral maximum and width B.[2] The more leakage, the greater the bandwidth. It is sometimes called noise equivalent bandwidth or equivalent noise bandwidth, because it is proportional to the average power that will be registered by each DFT bin when the input signal contains a random noise component (or is just random noise). A graph of the power This figure compares the processing losses of three window functions for sinusoidal spectrum, averaged over time, inputs, with both minimum and maximum scalloping loss. typically reveals a flat noise floor, caused by this effect. The height of the noise floor is proportional to B. So two different window functions can produce different noise floors.

Processing gain and losses In signal processing, operations are chosen to improve some aspect of quality of a signal by exploiting the differences between the signal and the corrupting influences. When the signal is a sinusoid corrupted by additive random noise, spectral analysis distributes the signal and noise components differently, often making it easier to detect the signal's presence or measure certain characteristics, such as amplitude and frequency. Effectively, the signal to noise ratio (SNR) is improved by distributing the noise uniformly, while concentrating most of the sinusoid's energy around one frequency. Processing gain is a term often used to describe an SNR improvement. The processing gain of spectral analysis depends on the window function, both its noise bandwidth (B) and its potential scalloping loss. These effects partially offset, because windows with the least scalloping naturally have the most leakage. The figure at right depicts the effects of three different window functions on the same data set, comprising two equal strength sinusoids in additive noise. The frequencies of the sinusoids are chosen such that one encounters no scalloping and the other encounters maximum scalloping. Both sinusoids suffer less SNR loss under the Hann window than under the Blackman–Harris window. In general (as mentioned earlier), this is a deterrent to using high-dynamic-range windows in low-dynamic-range applications.

34

Window function

Filter design Main article: Filter design Windows are sometimes used in the design of digital filters, in particular to convert an "ideal" impulse response of infinite duration, such as a sinc function, to a finite impulse response (FIR) filter design. That is called the window method.[3][4]

Symmetry and asymmetry Window functions generated for digital Sampled window functions are generated differently for filter design and spectral analysis filter design are symmetrical applications. And the asymmetrical ones often used in spectral analysis are also generated in a couple of different ways. Using the triangular function, for example, 3 different sequences, usually an odd length with outcomes for an 8-point window sequence are illustrated. a single maximum at the center. Windows for DFT/FFT usage, such as in spectral analysis, are often created by deleting the right-most coefficient of an odd-length, symmetrical window. Such truncated sequences are known as periodic.[5] The deleted coefficient is effectively restored (by a virtual copy of the symmetrical left-most coefficient) when the truncated sequence is periodically extended (which is the time-domain equivalent of sampling the DTFT). A different way of saying the same thing is that the DFT "samples" the DTFT of the Three different ways to create an 8-point Hann window sequence. window at the exact points that are not affected by spectral leakage from the discontinuity. The advantage of this trick is that a 512 length window (for example) enjoys the slightly better performance metrics of a 513 length design. Such a window is generated by the Matlab function hann(512,'periodic'), for instance. To generate it with the formula in this article (below), the window length (N) is 513, and the 513th coefficient of the generated sequence is discarded. Another type of asymmetric window, called DFT-even, is limited to even length sequences. The generated sequence is offset (cyclically) from its zero-phase [6] counterpart by exactly half the sequence length. In the frequency domain, that corresponds to a multiplication by the trivial sequence (-1)k, which can have implementation advantages for windows defined by their frequency domain form. Compared to a symmetrical window, the DFT-even sequence has an offset of ½ sample. As illustrated in the figure at right, that means the asymmetry is limited to just one missing coefficient. Therefore, as in the periodic case, it is effectively restored (by a virtual copy of the symmetrical left-most coefficient) when the truncated sequence is periodically extended.

35

Window function

36

Applications for which windows should not be used In some applications, it is preferable not to use a window function. For example: • In impact modal testing, when analyzing transient signals such as an excitation signal from hammer blow (see Impulse excitation technique), where most of the energy is located at the beginning of the recording. Using a non-rectangular window would attenuate most of the energy and spread the frequency response unnecessarily.[7] • A generalization of above, when measuring a self-windowing signal, such as an impulse, a shock response, a sine burst, a chirp burst, noise burst. Such signals are used in modal analysis. Applying a window function in this case would just deteriorate the signal-to-noise ratio. • When measuring a pseudo-random noise (PRN) excitation signal with period T, and using the same recording period T. A PRN signal is periodic and therefore all spectral components of the signal will coincide with FFT bin centers with no leakage.[8] • When measuring a repetitive signal locked-in to the sampling frequency, for example measuring the vibration spectrum analysis during Shaft alignment, fault diagnosis of bearings, engines, gearboxes etc. Since the signal is repetitive, all spectral energy is confined to multiples of the base repetition frequency. • In an OFDM receiver, the input signal is directly multiplied by FFT without a window function. The frequency sub-carriers (aka symbols) are designed to align exactly to the FFT frequency bins. A cyclic prefix is usually added to the transmitted signal, allowing frequency-selective fading due to multipath to be modeled as circular convolution, thus avoiding intersymbol interference, which in OFDM is equivalent to spectral leakage.

A list of window functions Terminology: • N represents the width, in samples, of a discrete-time, symmetrical window function When N is an odd number, the non-flat windows have a singular maximum point. When N is even, they have a double maximum. • It is sometimes useful to express

as the lagged version of a sequence of samples of a zero-phase [6]

function:

• For instance, for even values of N we can describe the related DFT-even window as discussed in the previous section. The DFT of such a sequence, in terms of the DFT of the

as sequence, is

• Each figure label includes the corresponding noise equivalent bandwidth metric (B), in units of DFT bins.

B-spline windows B-spline windows can be obtained as k-fold convolutions of the rectangular window. They include the rectangular window itself (k = 1), the triangular window (k = 2) and the Parzen window (k = 4). Alternative definitions sample the appropriate normalized B-spline basis functions instead of convolving discrete-time windows. A kth order B-spline basis function is a piece-wise polynomial function of degree k−1 that is obtained by k-fold self-convolution of the rectangular function.

Window function

37

Rectangular window The rectangular window (sometimes known as the boxcar or Dirichlet window) is the simplest window, equivalent to replacing all but N values of a data sequence by zeros, making it appear as though the waveform suddenly turns on and off:

Other windows are designed to moderate these sudden changes Rectangular window; B = 1.0000. because discontinuities have undesirable effects on the discrete-time Fourier transform (DTFT) and/or the algorithms that produce samples of the DTFT.[9][10] The rectangular window is the 1st order B-spline window as well as the 0th power cosine window. Triangular window Triangular windows are given by:

where L can be N,[11] N+1, or N-1.[12] The last one is also known as Bartlett window. All three definitions converge at large N. The triangular window is the 2nd order Triangular window or equivalently the Bartlett window; B = 1.3333. B-spline window and can be seen as the convolution of two half-sized rectangular windows, giving it twice the width of the regular windows. Parzen window Not to be confused with Kernel density estimation. The Parzen window, also known as the de la Vallée Poussin window, is the 4th order B-spline window.

Other polynomial windows

Parzen window; B = 1.92.

Window function

38

Welch window The Welch window consists of a single parabolic section: . The defining quadratic polynomial reaches a value of zero at the samples just outside the span of the window.

Generalized Hamming windows

Welch window; B = 1.20.

Generalized Hamming windows are of the form: . They have only three non-zero DFT coefficients and share the benefits of a sparse frequency domain representation with higher-order generalized cosine windows. Hann (Hanning) window Main article: Hann function The Hann window named after Julius von Hann and also known as the Hanning (for being similar in name and form to the Hamming window), von Hann and the raised cosine window is defined by:[13][14]

• zero-phase version: Hann window; B = 1.5000.

The ends of the cosine just touch zero, so the side-lobes roll off at about 18 dB per octave.[15]

Window function

39

Hamming window The window with these particular coefficients was proposed by Richard W. Hamming. The window is optimized to minimize the maximum (nearest) side lobe, giving it a height of about one-fifth that of the Hann window.[16]

with Hamming window, α = 0.53836 and β = 0.46164; B = 1.37. The original Hamming window would have α = 0.54 and β = 0.46; B = 1.3628.

instead of both constants being equal to 1/2 in the Hann window. The constants are approximations of values α = 25/46 and β = 21/46, which cancel the first sidelobe of the Hann window by placing a zero at frequency 5π/(N − 1). Approximation of the constants to two decimal places substantially lowers the level of sidelobes, to a nearly equiripple condition. In the equiripple sense, the optimal values for the coefficients are α = 0.53836 and β = 0.46164. • zero-phase version:

Higher-order generalized cosine windows Windows of the form:

have only 2K + 1 non-zero DFT coefficients, which makes them good choices for applications that require windowing by convolution in the frequency-domain. In those applications, the DFT of the unwindowed data vector is needed for a different purpose than spectral analysis. (see Overlap-save method). Generalized cosine windows with just two terms (K = 1) belong in the subfamily generalized Hamming windows.

Window function

40

Blackman windows Blackman windows are defined as:

By common convention, the unqualified term Blackman window refers to α = 0.16, as this most closely approximates the "exact [17] Blackman", with a0 = 7938/18608 ≈ 0.42659, a1 = 9240/18608 ≈ 0.49656, and a2 = 1430/18608 ≈ 0.076849.[18] These exact values place zeros at the third and fourth sidelobes.

Blackman window; α = 0.16; B = 1.73.

Nuttall window, continuous first derivative Considering n as a real number, the function and its first derivative are continuous everywhere.

Nuttall window, continuous first derivative; B = 2.0212.

Blackman–Nuttall window

Blackman–Nuttall window; B = 1.9761.

Window function

41

Blackman–Harris window A generalization of the Hamming family, produced by adding more shifted sinc functions, meant to minimize side-lobe levels[19][20]

Blackman–Harris window; B = 2.0044.

Flat top window A flat top window is a partially negative-valued window that has a flat top in the frequency domain. Such windows have been made available in spectrum analyzers for the measurement of amplitudes of sinusoidal frequency components. They have a low amplitude measurement error suitable for this purpose, achieved by the spreading of the energy of a sine wave over multiple SRS flat top window; B = 3.7702. bins in the spectrum. This ensures that the unattenuated amplitude of the sinusoid can be found on at least one of the neighboring bins. The drawback of the broad bandwidth is poor frequency resolution. To compensate, a longer window length may be chosen. Flat top windows can be designed using low-pass filter design methods, or they may be of the usual sum-of-cosine-terms variety. An example of the latter is the flat top window available in the Stanford Research Systems (SRS) SR785 spectrum analyzer:

Rife–Vincent window Rife and Vincent define three classes of windows constructed as sums of cosines; the classes are generalizations of the Hanning window. Their order-P windows are of the form (normalized to have unity average as opposed to unity max as the windows above are): . For order 1, this formula can match the Hanning window for a1 = −1; this is the Rife–Vincent class-I window, defined by minimizing the high-order sidelobe amplitude. The class-I order-2 Rife–Vincent window has a1 = −4/3 and a2 = 1/3. Coefficients for orders up to 4 are tabulated. For orders greater than 1, the Rife–Vincent window

Window function

42

coefficients can be optimized for class II, meaning minimized main-lobe width for a given maximum side-lobe, or for class III, a compromise for which order 2 resembles Blackmann's window. Given the wide variety of Rife–Vincent windows, plots are not given here.

Power-of-cosine windows Window functions in the power-of-cosine family are of the form:

The rectangular window (α = 0), the cosine window (α = 1), and the Hann window (α = 2) are members of this family. Cosine window

The cosine window is also known as the sine window. Cosine window describes the shape of A cosine window convolved by itself is known as the Bohman window.

Adjustable windows Cosine window; B = 1.23.

Gaussian window The Fourier transform of a Gaussian is also a Gaussian (it is an eigenfunction of the Fourier Transform). Since the Gaussian function extends to infinity, it must either be truncated at the ends of the window, or itself windowed with another zero-ended window.[21] Since the log of a Gaussian produces a parabola, this can be used for exact quadratic interpolation in frequency estimation.[22][23][24]

Gaussian window, σ = 0.4; B = 1.45.

The standard deviation of the Gaussian function is σ(N−1)/2 sampling periods.

Window function

43

Confined Gaussian window The confined Gaussian window yields the smallest possible root mean square frequency width σω for a given temporal width σt. These windows optimize the RMS time-frequency bandwidth products. They are computed as the minimum eigenvectors of a parameter-dependent matrix. The confined Gaussian window family contains the cosine window and the Gaussian window in the limiting cases of large and small σt, respectively.

Confined Gaussian window, σt = 0.1N; B = 1.9982.

Approximate confined Gaussian window A confined Gaussian window of temporal width σt is well approximated by:

with the Gaussian:

Approximate confined Gaussian window, σt = 0.1N; B = 1.9979.

The temporal width of the approximate window is asymptotically equal to σt for σt 1, then the conditions result in what is sometimes referred to as undersampling, bandpass sampling, or using a sampling rate less than the Nyquist rate (2fH). For the case of a given sampling frequency, simpler formulae for the constraints on the signal's spectral band are given below. Example: Consider FM radio to illustrate the idea of undersampling. In the US, FM radio operates on the frequency band from fL = 88 MHz to fH = 108 MHz. The bandwidth is given by

The sampling conditions are satisfied for

Therefore, n can be 1, 2, 3, 4, or 5. The value n = 5 gives the lowest sampling frequencies interval and this is a

Spectrum of the FM radio band (88–108 MHz) and its baseband alias under 44 MHz (n = 5) sampling. An anti-alias filter quite tight to the FM radio band is required, and there's not room for stations at nearby expansion channels such as 87.9 without aliasing.

scenario of undersampling. In this case, the signal spectrum fits between 2 and 2.5 times the sampling rate (higher than 86.4–88 MHz but lower than 108–110 MHz). A lower value of n will also lead to a useful sampling rate. For example, using n = 4, the FM band spectrum fits easily between 1.5 and 2.0 times the sampling rate, for a sampling rate near 56 MHz (multiples of the Nyquist frequency being 28, 56, 84, 112, etc.). See the illustrations at the right.

Spectrum of the FM radio band (88–108 MHz) and its baseband alias under 56 MHz (n = 4) sampling, showing plenty of room for bandpass anti-aliasing filter transition bands. The baseband image is frequency-reversed in this case (even n).

When undersampling a real-world signal, the sampling circuit must be fast enough to capture the highest signal frequency of interest. Theoretically, each sample should be taken during an infinitesimally short interval, but this is not practically feasible. Instead, the sampling of the signal should be made in a short enough interval that it can represent the instantaneous value of the signal with the highest frequency. This means that in the FM radio example above, the sampling circuit must be able to capture a signal with a frequency of 108 MHz, not 43.2 MHz. Thus, the sampling frequency may be only a little bit greater than 43.2 MHz, but the input bandwidth of the system must be at least 108 MHz. Similarly, the accuracy of the sampling timing, or aperture uncertainty of the sampler, frequently the analog-to-digital converter, must be appropriate for the frequencies being sampled 108MHz, not the lower sample rate. If the sampling theorem is interpreted as requiring twice the highest frequency, then the required sampling rate would be assumed to be greater than the Nyquist rate 216 MHz. While this does satisfy the last condition on the sampling rate, it is grossly oversampled. Note that if a band is sampled with n > 1, then a band-pass filter is required for the anti-aliasing filter, instead of a lowpass filter.

Undersampling

99

As we have seen, the normal baseband condition for reversible sampling is that X(f) = 0 outside the interval: and the reconstructive interpolation function, or lowpass filter impulse response, is To accommodate undersampling, the bandpass condition is that X(f) = 0 outside the union of open positive and negative frequency bands for some positive integer

.

which includes the normal baseband condition as case n = 1 (except that where the intervals come together at 0 frequency, they can be closed). The corresponding interpolation function is the bandpass filter given by this difference of lowpass impulse responses: . On the other hand, reconstruction is not usually the goal with sampled IF or RF signals. Rather, the sample sequence can be treated as ordinary samples of the signal frequency-shifted to near baseband, and digital demodulation can proceed on that basis, recognizing the spectrum mirroring when n is even. Further generalizations of undersampling for the case of signals with multiple bands are possible, and signals over multidimensional domains (space or space-time) and have been worked out in detail by Igor Kluvánek.

References

Delta-sigma modulation "Sigma delta" redirects here. For the sorority, see Sigma Delta. Delta-sigma (ΔΣ; or sigma-delta, ΣΔ) modulation is a digital signal processing, or DSP method for encoding analog signals into digital signals as found in an ADC. It is also used to transfer higher-resolution digital signals into lower-resolution digital signals as part of the process to convert digital signals into analog. In a conventional ADC, an analog signal is integrated, or sampled, with a sampling frequency and subsequently quantized in a multi-level quantizer into a digital signal. This process introduces quantization error noise. The first step in a delta-sigma modulation is delta modulation. In delta modulation the change in the signal (its delta) is encoded, rather than the absolute value. The result is a stream of pulses, as opposed to a stream of numbers as is the case with PCM. In delta-sigma modulation, the accuracy of the modulation is improved by passing the digital output through a 1-bit DAC and adding (sigma) the resulting analog signal to the input signal, thereby reducing the error introduced by the delta-modulation. This technique has found increasing use in modern electronic components such as converters, frequency synthesizers, switched-mode power supplies and motor controllers, primarily because of its cost efficiency and reduced circuit complexity.[1] Both analog-to-digital converters (ADCs) and digital-to-analog converters (DACs) can employ delta-sigma modulation. A delta-sigma ADC first encodes an analog signal using high-frequency delta-sigma modulation, and then applies a digital filter to form a higher-resolution but lower sample-frequency digital output. On the other hand, a delta-sigma DAC encodes a high-resolution digital input signal into a lower-resolution but higher sample-frequency signal that is mapped to voltages, and then smoothed with an analog filter. In both cases, the temporary use of a lower-resolution signal simplifies circuit design and improves efficiency.

Delta-sigma modulation

100

The coarsely-quantized output of a delta-sigma modulator is occasionally used directly in signal processing or as a representation for signal storage. For example, the Super Audio CD (SACD) stores the output of a delta-sigma modulator directly on a disk.

Motivation Why convert an analog signal into a stream of pulses? In brief, because it is very easy to regenerate pulses at the receiver into the ideal form transmitted. The only part of the transmitted waveform required at the receiver is the time at which the pulse occurred. Given the timing information the transmitted waveform can be reconstructed electronically with great precision. In contrast, without conversion to a pulse stream but simply transmitting the analog signal directly, all noise in the system is added to the analog signal, reducing its quality. Each pulse is made up of a step up followed after a short interval by a step down. It is possible, even in the presence of electronic noise, to recover the timing of these steps and from that regenerate the transmitted pulse stream almost noiselessly. Then the accuracy of the transmission process reduces to the accuracy with which the transmitted pulse stream represents the input waveform.

Why delta-sigma modulation? Delta-sigma modulation converts the analog voltage into a pulse frequency and is alternatively known as Pulse Density modulation or Pulse Frequency modulation. In general, frequency may vary smoothly in infinitesimal steps, as may voltage, and both may serve as an analog of an infinitesimally varying physical variable such as acoustic pressure, light intensity, etc. The substitution of frequency for voltage is thus entirely natural and carries in its train the transmission advantages of a pulse stream. The different names for the modulation method are the result of pulse frequency modulation by different electronic implementations, which all produce similar transmitted waveforms.

Why the delta-sigma analog to digital conversion? The ADC converts the mean of an analog voltage into the mean of an analog pulse frequency and counts the pulses in a known interval so that the pulse count divided by the interval gives an accurate digital representation of the mean analog voltage during the interval. This interval can be chosen to give any desired resolution or accuracy. The method is cheaply produced by modern methods; and it is widely used.

Analog to digital conversion Description The ADC generates a pulse stream in which the frequency of pulses in the stream is proportional to the analog voltage input, , so that the frequency, where k is a constant for the particular implementation. A counter sums the number of pulses that occur in a predetermined period,

so that the sum,

, is

. is chosen so that a digital display of the count, Because

, is a display of

with a predetermined scaling factor.

may take any designed value it may be made large enough to give any desired resolution or accuracy.

Each pulse of the pulse stream has a known, constant amplitude

and duration

, and thus has a known integral

but variable separating interval. In a formal analysis an impulse such as integral

is treated as the Dirac δ (delta) function and is specified by the

step produced on integration. Here we indicate that step as

.

Delta-sigma modulation

101

The interval between pulses, p, is determined by a feedback loop arranged so that

.

The action of the feedback loop is to monitor the integral of v and when that integral has incremented by is indicated by the integral waveform crossing a threshold, then subtracting combined waveform sawtooths between the threshold and ( threshold -

, which

from the integral of v so that the

). At each step a pulse is added to the

pulse stream. Between impulses the slope of the integral is proportional to

. Whence

. It is the pulse stream which is transmitted for delta-sigma modulation but the pulses are counted to form sigma in the case of analogue to digital conversion.

Analysis Shown below the block diagram illustrated in Fig. 1 are waveforms at points designated by numbers 1 to 5 for an input of 0.2 volts on the left and 0.4 volts on the right. In most practical applications the summing interval is large compared with the impulse duration and for signals which are a significant fraction of full scale the variable separating interval is also small compared with the summing interval. The Nyquist–Shannon sampling theorem requires two samples to render a varying input signal. The samples appropriate to this criterion are two successive Σ counts taken in two successive summing intervals. The summing interval, which must accommodate a large count in order to achieve adequate precision, is inevitably long so that the converter can only render relatively low frequencies. Hence it is convenient and fair to represent the input voltage (1) as constant over a few impulses. Consider first the waveforms on the left. 1 is the input and for this short interval is constant at 0.2 V. The stream of

Fig. 1: Block diagram and waveforms for a sigma delta ADC.

Delta-sigma modulation

delta impulses is shown at 2 and the difference between 1 and 2 is shown at 3. This difference is integrated to produce the waveform 4. The threshold detector generates a pulse 5 which starts as the waveform 4 crosses the threshold and is sustained until the waveform 4 falls below the threshold. Within the loop 5 triggers the impulse generator and external to the loop increments the counter. The summing interval is a prefixed time and at its expiry the count is strobed into the buffer and the counter reset. It is necessary that the ratio between Fig. 1a: Effect of clocking impulses the impulse interval and the summing interval is equal to the maximum (full scale) count. It is then possible for the impulse duration and the summing interval to be defined by the same clock with a suitable arrangement of logic and counters. This has the advantage that neither interval has to be defined with absolute precision as only the ratio is important. Then to achieve overall accuracy it is only necessary that the amplitude of the impulse be accurately defined. On the right the input is now 0.4 V and the sum during the impulse is −0.6 V as opposed to −0.8 V on the left. Thus the negative slope during the impulse is lower on the right than on the left. Also the sum is 0.4 V on the right during the interval as opposed to 0.2 V on the left. Thus the positive slope outside the impulse is higher on the right than on the left. The resultant effect is that the integral (4) crosses the threshold more quickly on the right than on the left. A full analysis would show that in fact the interval between threshold crossings on the right is half that on the left. Thus the frequency of impulses is doubled. Hence the count increments at twice the speed on the right to that on the left which is consistent with the input voltage being doubled. Construction of the waveforms illustrated at (4) is aided by concepts associated with the Dirac delta function in that all impulses of the same strength produce the same step when integrated, by definition. Then (4) is constructed using an intermediate step (6) in which each integrated impulse is represented by a step of the assigned strength which decays to zero at the rate determined by the input voltage. The effect of the finite duration of the impulse is constructed in (4) by drawing a line from the base of the impulse step at zero volts to intersect the decay line from (6) at the full duration of the impulse. As stated, Fig. 1 is a simplified block diagram of the delta-sigma ADC in which the various functional elements have been separated out for individual treatment and which tries to be independent of any particular implementation. Many particular implementations seek to define the impulse duration and the summing interval from the same clock as discussed above but in such a way that the start of the impulse is delayed until the next occurrence of the appropriate clock pulse boundary. The effect of this delay is illustrated in Fig. 1a for a sequence of impulses which occur at a nominal 2.5 clock intervals, firstly for impulses generated immediately the threshold is crossed as previously discussed and secondly for impulses delayed by the clock. The effect of the delay is firstly that the ramp continues until the onset of the impulse, secondly that the impulse produces a fixed amplitude step so that the integral retains the excess it acquired during the impulse delay and so the ramp restarts from a higher point and is now on the same locus as the unclocked integral. The effect is that, for this example, the undelayed impulses will occur at clock points 0, 2.5, 5, 7.5, 10, etc. and the clocked impulses will occur at 0, 3, 5, 8, 10, etc. The maximum error that can occur due to clocking is marginally less than one count. Although the Sigma-Delta converter is

102

Delta-sigma modulation

103

generally implemented using a common clock to define the impulse duration and the summing interval it is not absolutely necessary and an implementation in which the durations are independently defined avoids one source of noise, the noise generated by waiting for the next common clock boundary. Where noise is a primary consideration that overrides the need for absolute amplitude accuracy; e.g., in bandwidth limited signal transmission, separately defined intervals may be implemented.

Practical Implementation A circuit diagram for a practical implementation is illustrated, Fig 1b and the associated waveforms Fig. 1c. This circuit diagram is mainly for illustration purposes, details of particular manufacturers implementations will usually be available from the particular manufacturer. A scrap view of an alternative front end is shown in Fig. 1b which has the advantage that the voltage at the switch terminals are relatively constant and close to 0.0 V. Also the current generated through R by −Vref is constant at −Vref/R so that Fig. 1b: circuit diagram much less noise is radiated to adjacent parts of the circuit. Then this would be the preferred front end in practice but, in order to show the impulse as a voltage pulse so as to be consistent with previous discussion, the front end given here, which is an electrical equivalent, is used. From the top of Fig 1c the waveforms, labelled as they are on the circuit diagram, are:The clock. (a) Vin. This is shown as varying from 0.4 V initially to 1.0 V and then to zero volts to show the effect on the feedback loop. (b) The impulse waveform. It will be discovered how this acquires its form as we traverse the feedback loop. (c)

The

current

into

the

capacitor,

Ic,

is

the

linear

sum

of

the

impulse

voltage

Delta-sigma modulation

104

upon R and Vin upon R. To show this sum as a voltage the product R × Ic is plotted. The input impedance of the amplifier is regarded as so high that the current drawn by the input is neglected. (d) The negated integral of Ic. This negation is standard for the op. amp. implementation of an integrator and comes about because the current into the capacitor at the amplifier input is the current out of the capacitor at the amplifier output and the voltage is the integral of the current divided by the capacitance of C.

Fig. 1c: ADC waveforms

(e) The comparator output. The comparator is a very high gain amplifier with its plus input terminal connected for reference to 0.0 V. Whenever the negative input terminal is taken negative with respect the positive terminal of the amplifier the output saturates positive and conversely negative saturation for positive input. Thus the output saturates positive whenever the integral (d) goes below the 0 V reference level and remains there until (d) goes positive with respect to the reference level. (f) The impulse timer is a D type positive edge triggered flip flop. Input information applied at D is transferred to Q on the occurrence of the positive edge of the clock pulse. thus when the comparator output (e) is positive Q goes positive or remains positive at the next positive clock edge. Similarly, when (e) is negative Q goes negative at the next positive clock edge. Q controls the electronic switch to generate the current impulse into the integrator. Examination of the waveform (e) during the initial period illustrated, when Vin is 0.4 V, shows (e) crossing the threshold well before the trigger edge (positive edge of the clock pulse) so that there is an appreciable delay before the impulse starts. After the start of the impulse there is further delay while (e) climbs back past the threshold. During this time the comparator output remains high but goes low before the next trigger edge. At that next trigger edge the impulse timer goes low to follow the comparator. Thus the clock determines the duration of the impulse. For the next impulse the threshold is crossed immediately before the trigger edge and so the comparator is only briefly positive. Vin (a) goes to full scale, +Vref, shortly before the end of the next impulse. For the remainder of that impulse the capacitor current (c) goes to zero and hence the integrator slope briefly goes to zero. Following this impulse the full scale positive current is flowing (c) and the integrator sinks at its maximum rate and so crosses the threshold well before the next trigger edge. At that edge the impulse starts and the Vin current is now matched by the reference current so that the net capacitor current (c) is zero. Then the integration now has zero slope and remains at the negative value it had at the start of the impulse. This has the effect that the impulse current remains switched on because Q is stuck positive because the comparator is stuck positive at every trigger edge. This is consistent with

Delta-sigma modulation contiguous, butting impulses which is required at full scale input. Eventually Vin (a) goes to zero which means that the current sum (c) goes fully negative and the integral ramps up. It shortly thereafter crosses the threshold and this in turn is followed by Q, thus switching the impulse current off. The capacitor current (c) is now zero and so the integral slope is zero, remaining constant at the value it had acquired at the end of the impulse. (g) The countstream is generated by gating the negated clock with Q to produce this waveform. Thereafter the summing interval, sigma count and buffered count are produced using appropriate counters and registers. The Vin waveform is approximated by passing the countstream (g) into a low pass filter, however it suffers from the defect discussed in the context of Fig. 1a. One possibility for reducing this error is to halve the feedback pulse length to half a clock period and double its amplitude by halving the impulse defining resistor thus producing an impulse of the same strength but one which never butts onto its adjacent impulses. Then there will be a threshold crossing for every impulse. In this arrangement a monostable flip flop triggered by the comparator at the threshold crossing will closely follow the threshold crossings and thus eliminate one source of error, both in the ADC and the sigma delta modulator.

Remarks In this section we have mainly dealt with the analogue to digital converter as a stand alone function which achieves astonishing accuracy with what is now a very simple and cheap architecture. Initially the Delta-Sigma configuration was devised by INOSE et al. to solve problems in the accurate transmission of analog signals. In that application it was the pulse stream that was transmitted and the original analog signal recovered with a low pass filter after the received pulses had been reformed. This low pass filter performed the summation function associated with Σ. The highly mathematical treatment of transmission errors was introduced by them and is appropriate when applied to the pulse stream but these errors are lost in the accumulation process associated with Σ to be replaced with the errors associated with the mean of means when discussing the ADC. For those uncomfortable with this assertion consider this. It is well known that by Fourier analysis techniques the incoming waveform can be represented over the summing interval by the sum of a constant plus a fundamental and harmonics each of which has an exact integer number of cycles over the sampling period. It is also well known that the integral of a sine wave or cosine wave over one or more full cycles is zero. Then the integral of the incoming waveform over the summing interval reduces to the integral of the constant and when that integral is divided by the summing interval it becomes the mean over that interval. The interval between pulses is proportional to the inverse of the mean of the input voltage during that interval and thus over that interval, ts, is a sample of the mean of the input voltage proportional to V/ts. Thus the average of the input voltage over the summing period is VΣ/N and is the mean of means and so subject to little variance. Unfortunately the analysis for the transmitted pulse stream has, in many cases, been carried over, uncritically, to the ADC. It was indicated in section 2.2 Analysis that the effect of constraining a pulse to only occur on clock boundaries is to introduce noise, that generated by waiting for the next clock boundary. This will have its most deleterious effect on the high frequency components of a complex signal. Whilst the case has been made for clocking in the ADC environment, where it removes one source of error, namely the ratio between the impulse duration and the summing interval, it is deeply unclear what useful purpose clocking serves in a single channel transmission environment since it is a source of both noise and complexity but it is conceivable that it would be useful in a TDM (time division multiplex) environment. A very accurate transmission system with constant sampling rate may be formed using the full arrangement shown here by transmitting the samples from the buffer protected with redundancy error correction. In this case there will be a trade off between bandwidth and N, the size of the buffer. The signal recovery system will require redundancy

105

Delta-sigma modulation error checking, digital to analog conversion,and sample and hold circuitry. A possible further enhancement is to include some form of slope regeneration.This amounts to PCM (pulse code modulation) with digitization performed by a sigma-delta ADC. The above description shows why the impulse is called delta. The integral of an impulse is a step. A one bit DAC may be expected to produce a step and so must be a conflation of an impulse and an integration. The analysis which treats the impulse as the output of a 1-bit DAC hides the structure behind the name (sigma delta) and cause confusion and difficulty interpreting the name as an indication of function. This analysis is very widespread but is deprecated. A modern alternative method for generating voltage to frequency conversion is discussed in synchronous voltage to frequency converter (SVFC) which may be followed by a counter to produce a digital representation in a similar manner to that described above.[2]

Digital to analog conversion Discussion Delta-sigma modulators are often used in digital to analog converters (DACs). In general, a DAC converts a digital number representing some analog value into that analog value. For example, the analog voltage level into a speaker may be represented as a 20 bit digital number, and the DAC converts that number into the desired voltage. To actually drive a load (like a speaker) a DAC is usually connected to or integrated with an electronic amplifier. This can be done using a delta-sigma modulator in a Class D Amplifier. In this case, a multi-bit digital number is input to the delta-sigma modulator, which converts it into a faster sequence of 0's and 1's. These 0's and 1's are then converted into analog voltages. The conversion, usually with MOSFET drivers, is very efficient in terms of power because the drivers are usually either fully on or fully off, and in these states have low power loss. The resulting two-level signal is now like the desired signal, but with higher frequency components to change the signal so that it only has two levels. These added frequency components arise from the quantization error of the delta-sigma modulator, but can be filtered away by a simple low-pass filter. The result is a reproduction of the original, desired analog signal from the digital values. The circuit itself is relatively inexpensive. The digital circuit is small, and the MOSFETs used for the power amplification are simple. This is in contrast to a multi-bit DAC which can have very stringent design conditions to precisely represent digital values with a large number of bits. The use of a delta-sigma modulator in the digital to analog conversion has enabled a cost-effective, low power, and high performance solution.

106

Delta-sigma modulation

107

Relationship to Δ-modulation ΔΣ modulation (SDM) is inspired by Δ modulation (DM), as shown in Fig. 2. If quantization were homogeneous (e.g., if it were linear), the following would be a sufficient derivation of the equivalence of DM and SDM: 1. Start with a block diagram of a Δ-modulator/demodulator. 2. The linearity property of integration ( ) makes it possible to move the integrator, which reconstructs the analog signal in the demodulator section, in front of the Δ-modulator. 3. Again, the linearity property of the integration allows the two integrators to be combined and a ΔΣ-modulator/demodulator block diagram is obtained.

Fig. 2: Derivation of ΔΣ- from Δ-modulation

However, the quantizer is not homogeneous, and so this explanation is flawed. It's true that ΔΣ is inspired by Δ-modulation, but the two are distinct in operation. From the first block diagram in Fig. 2, the integrator in the feedback path can be removed if the feedback is taken directly from the input of the low-pass filter. Hence, for delta modulation of input signal , the low-pass filter sees the signal

However, sigma-delta modulation of the same input signal places at the low-pass filter

In other words, SDM and DM swap the position of the integrator and quantizer. The net effect is a simpler implementation that has the added benefit of shaping the quantization noise away from signals of interest (i.e., signals of interest are low-pass filtered while quantization noise is high-pass filtered). This effect becomes more dramatic with increased oversampling, which allows for quantization noise to be somewhat programmable. On the other hand, Δ-modulation shapes both noise and signal equally. Additionally, the quantizer (e.g., comparator) used in DM has a small output representing a small step up and down the quantized approximation of the input while the quantizer used in SDM must take values outside of the range of the input signal, as shown in Fig. 3.

Fig. 3: An example of SDM of 100 samples of one period a sine wave. 1-bit samples (e.g., comparator output) overlaid with sine wave where logic high (e.g., ) represented by blue and logic low (e.g., ) represented by white.

In general, ΔΣ has some advantages versus Δ modulation:

Delta-sigma modulation

108

• The whole structure is simpler: • Only one integrator is needed • The demodulator can be a simple linear filter (e.g., RC or LC filter) to reconstruct the signal • The quantizer (e.g., comparator) can have full-scale outputs • The quantized value is the integral of the difference signal, which makes it less sensitive to the rate of change of the signal.

Principle The principle of the ΔΣ architecture is explained at length in section 2. Initially, when a sequence starts, the circuit will have an arbitrary state which is dependant on the integral of all previous history. In mathematical terms this corresponds to the arbitrary integration constant of the indefinite integral. This follows from the fact that at the heart of the method there is an integrator which can have any arbitrary state dependant on previous input, see Fig. 1c (d). From the occurrence of the first pulse onward the frequency of the pulse stream is proportional to the input voltage to be transformed. A demonstration applet is available online to simulate the whole architecture.[3]

Variations There are many kinds of ADC that use this delta-sigma structure. The above analysis focuses on the simplest 1st-order, 2-level, uniform-decimation sigma-delta ADC. Many ADCs use a second-order 5-level sinc3 sigma-delta structure.

2nd order and higher order modulator The number of integrators, and consequently, the numbers of feedback loops, indicates the order of a ΔΣ-modulator; a 2nd order ΔΣ modulator is shown in Fig. 4. First order modulators are unconditionally stable, but stability analysis must be performed for higher order modulators.

Fig. 4: Block diagram of a 2nd order ΔΣ modulator

3-level and higher quantizer The modulator can also be classified by the number of bits it has in output, which strictly depends on the output of the quantizer. The quantizer can be realized with a N-level comparator, thus the modulator has log2N-bit output. A simple comparator has 2 levels and so is 1 bit quantizer; a 3-level quantizer is called a "1.5" bit quantizer; a 4-level quantizer is a 2 bit quantizer; a 5-level quantizer is called a "2.5 bit" quantizer.[4]

Decimation structures The conceptually simplest decimation structure is a counter that is reset to zero at the beginning of each integration period, then read out at the end of the integration period. The multi-stage noise shaping (MASH) structure has a noise shaping property, and is commonly used in digital audio and fractional-N frequency synthesizers. It comprises two or more cascaded overflowing accumulators, each of which is equivalent to a first-order sigma delta modulator. The carry outputs are combined through summations and delays to produce a binary output, the width of which depends on the number of stages (order) of the MASH. Besides its noise shaping function, it has two more attractive properties:

Delta-sigma modulation

109

• simple to implement in hardware; only common digital blocks such as accumulators, adders, and D flip-flops are required • unconditionally stable (there are no feedback loops outside the accumulators) A very popular decimation structure is the sinc filter. For 2nd order modulators, the sinc3 filter is close to optimum.[5][6]

Quantization theory formulas Main article: Quantization (signal processing) When a signal is quantized, the resulting signal approximately has the second-order statistics of a signal with independent additive white noise. Assuming that the signal value is in the range of one step of the quantized value with an equal distribution, the root mean square value of this quantization noise is

In reality, the quantization noise is of course not independent of the signal; this dependence is the source of idle tones and pattern noise in Sigma-Delta converters. Over sampling ratio (OSR), where

is the sampling frequency and

is Nyquist rate

The RMS noise voltage within the band of interest can be expressed in terms of OSR

Oversampling Main article: Oversampling Let's consider a signal at frequency and a sampling frequency of much higher than Nyquist rate (see fig. 5). ΔΣ modulation is based on the technique of oversampling to reduce the noise in the band of interest (green), which also avoids the use of high-precision analog circuits for the anti-aliasing filter. The quantization noise is the same both in a Nyquist converter (in yellow) and in an oversampling converter (in blue), but it is distributed over a larger spectrum. In ΔΣ-converters, noise is further reduced at low frequencies, which is the band Fig. 5: Noise shaping curves and noise spectrum in ΔΣ modulator where the signal of interest is, and it is increased at the higher frequencies, where it can be filtered. This technique is known as noise shaping. For a first order delta sigma modulator, the noise is shaped by a filter with transfer function Assuming that the sampling frequency approximated as:

.

, the quantization noise in the desired signal bandwidth can be

Delta-sigma modulation

110

. Similarly for a second order delta sigma modulator, the noise is shaped by a filter with transfer function . The in-band quantization noise can be approximated as: . In general, for a

-order ΔΣ-modulator, the variance of the in-band quantization noise: .

When the sampling frequency is doubled, the signal to quantization noise is improved by

for a

-order ΔΣ-modulator. The higher the oversampling ratio, the higher the signal-to-noise ratio and the higher the resolution in bits. Another key aspect given by oversampling is the speed/resolution tradeoff. In fact, the decimation filter put after the modulator not only filters the whole sampled signal in the band of interest (cutting the noise at higher frequencies), but also reduces the frequency of the signal increasing its resolution. This is obtained by a sort of averaging of the higher data rate bitstream.

Example of decimation Let's have, for instance, an 8:1 decimation filter and a 1-bit bitstream; if we have an input stream like 10010110, counting the number of ones, we get 4. Then the decimation result is 4/8 = 0.5. We can then represent it with a 3-bits number 100 (binary), which means half of the largest possible number. In other words, • the sample frequency is reduced by a factor of eight • the serial (1-bit) input bus becomes a parallel (3-bits) output bus.

Naming The technique was first presented in the early 1960s by professor Haruhiko Yasuda while he was a student at Waseda University, Tokyo, Japan.Wikipedia:Citation needed The name Delta-Sigma comes directly from the presence of a Delta modulator and an integrator, as firstly introduced by Inose et al. in their patent application.[7] That is, the name comes from integrating or "summing" differences, which are operations usually associated with Greek letters Sigma and Delta respectively. Both names Sigma-Delta and Delta-Sigma are frequently used.

References [1] http:/ / www. numerix-dsp. com/ appsnotes/ APR8-sigma-delta. pdf [2] Voltage-to-Frequency Converters (http:/ / www. analog. com/ static/ imported-files/ tutorials/ MT-028. pdf) by Walt Kester and James Bryant 2009. Analog Devices. [3] Analog Devices : Virtual Design Center : Interactive Design Tools : Sigma-Delta ADC Tutorial (http:/ / designtools. analog. com/ dt/ sdtutorial/ sdtutorial. html) [4] Sigma-delta class-D amplifier and control method for a sigma-delta class-D amplifier (http:/ / www. faqs. org/ patents/ app/ 20090072897) by Jwin-Yen Guo and Teng-Hung Chang [5] A Novel Architecture for DAQ in Multi-channel, Large Volume, Long Drift Liquid Argon TPC (http:/ / www. slac. stanford. edu/ econf/ C0604032/ papers/ 0232. PDF) by S. Centro, G. Meng, F. Pietropaola, S. Ventura 2006 [6] A Low Power Sinc3 Filter for ΣΔ Modulators (http:/ / ieeexplore. ieee. org/ xpl/ freeabs_all. jsp?arnumber=4253561) by A. Lombardi, E. Bonizzoni, P. Malcovati, F. Maloberti 2007 [7] H. Inose, Y. Yasuda, J. Murakami, "A Telemetering System by Code Manipulation -- ΔΣ Modulation", IRE Trans on Space Electronics and Telemetry, Sep. 1962, pp. 204-209.

• Walt Kester (October 2008). "ADC Architectures III: Sigma-Delta ADC Basics" (http://www.analog.com/ static/imported-files/tutorials/MT-022.pdf) (PDF). Analog Devices. Retrieved 2010-11-02.

Delta-sigma modulation • R. Jacob Baker (2009). CMOS Mixed-Signal Circuit Design (http://CMOSedu.com/) (2nd ed.). Wiley-IEEE. ISBN 978-0-470-29026-2. • R. Schreier, G. Temes (2005). Understanding Delta-Sigma Data Converters. ISBN 0-471-46585-2. • S. Norsworthy, R. Schreier, G. Temes (1997). Delta-Sigma Data Converters. ISBN 0-7803-1045-4. • J. Candy, G. Temes (1992). Oversampling Delta-sigma Data Converters. ISBN 0-87942-285-8.

External links • 1-bit A/D and D/A Converters (http://www.cs.tut.fi/sgn/arg/rosti/1-bit/) • Sigma-delta techniques extend DAC resolution (http://www.embedded.com/design/configurable-systems/ 4006431/Sigma-delta-techniques-extend-DAC-resolution) article by Tim Wescott 2004-06-23 • Tutorial on Designing Delta-Sigma Modulators: Part I (http://www.commsdesign.com/design_corner/ showArticle.jhtml?articleID=18402743) and Part II (http://www.commsdesign.com/design_corner/ showArticle.jhtml?articleID=18402763) by Mingliang (Michael) Liu • Gabor Temes' Publications (http://eecs.oregonstate.edu/research/members/temes/pubs.html) • Simple Sigma Delta Modulator example (http://electronjunkie.wordpress.com/tag/sigma-delta-modulation/) Contains Block-diagrams, code, and simple explanations • Example Simulink model & scripts for continuous-time sigma-delta ADC (http://www.circuitdesign.info/blog/ 2008/09/example-simulink-model-scripts/) Contains example matlab code and Simulink model • Bruce Wooley's Delta-Sigma Converter Projects (http://www-cis.stanford.edu/icl/wooley-grp/projects.html) • An Introduction to Delta Sigma Converters (http://www.beis.de/Elektronik/DeltaSigma/DeltaSigma.html) (which covers both ADC's and DAC's sigma-delta) • Demystifying Sigma-Delta ADCs (http://www.maxim-ic.com/appnotes.cfm/an_pk/1870/CMP/WP-10). This in-depth article covers the theory behind a Delta-Sigma analog-to-digital converter. • Motorola digital signal processors: Principles of sigma-delta modulation for analog-to-digital converters (http:// digitalsignallabs.com/SigmaDelta.pdf) • One-Bit Delta Sigma D/A Conversion Part I: Theory (http://www.digitalsignallabs.com/presentation.pdf) article by Randy Yates presented at the 2004 comp.dsp conference • MASH (Multi-stAge noise SHaping) structure (http://www.aholme.co.uk/Frac2/Mash.htm) with both theory and a block-level implementation of a MASH • Continuous time sigma-delta ADC noise shaping filter circuit architectures (http://www.circuitdesign.info/ blog/2008/11/continuous-time-sigma-delta-adc-noise-shaping-filter-circuit-architectures-2/) discusses architectural trade-offs for continuous-time sigma-delta noise-shaping filters • Some intuitive motivation for why a Delta Sigma modulator works (http://www.cardinalpeak.com/blog/ ?p=392/)

111

Jitter

112

Jitter For other meanings of this word, see Jitter (disambiguation). Jitter is the undesired deviation from true periodicity of an assumed periodic signal in electronics and telecommunications, often in relation to a reference clock source. Jitter may be observed in characteristics such as the frequency of successive pulses, the signal amplitude, or phase of periodic signals. Jitter is a significant, and usually undesired, factor in the design of almost all communications links (e.g., USB, PCI-e, SATA, OC-48). In clock recovery applications it is called timing jitter.[1] Jitter can be quantified in the same terms as all time-varying signals, e.g., RMS, or peak-to-peak displacement. Also like other time-varying signals, jitter can be expressed in terms of spectral density (frequency content). Jitter period is the interval between two times of maximum effect (or minimum effect) of a signal characteristic that varies regularly with time. Jitter frequency, the more commonly quoted figure, is its inverse. ITU-T G.810 classifies jitter frequencies below 10 Hz as wander and frequencies at or above 10 Hz as jitter. Jitter may be caused by electromagnetic interference (EMI) and crosstalk with carriers of other signals. Jitter can cause a display monitor to flicker, affect the performance of processors in personal computers, introduce clicks or other undesired effects in audio signals, and loss of transmitted data between network devices. The amount of tolerable jitter depends on the affected application.

Sampling jitter In analog to digital and digital to analog conversion of signals, the sampling is normally assumed to be periodic with a fixed period—the time between every two samples is the same. If there is jitter present on the clock signal to the analog-to-digital converter or a digital-to-analog converter, the time between samples varies and instantaneous signal error arises. The error is proportional to the slew rate of the desired signal and the absolute value of the clock error. Various effects such as noise (random jitter), or spectral components (periodic jitter)Wikipedia:Citing sources can come about depending on the pattern of the jitter in relation to the signal. In some conditions, less than a nanosecond of jitter can reduce the effective bit resolution of a converter with a Nyquist frequency of 22 kHz to 14 bits.Wikipedia:Citation needed This is a consideration in high-frequency signal conversion, or where the clock signal is especially prone to interference.

Packet jitter in computer networks Main article: Packet delay variation In the context of computer networks, jitter is the variation in latency as measured in the variability over time of the packet latency across a network. A network with constant latency has no variation (or jitter). Packet jitter is expressed as an average of the deviation from the network mean latency. However, for this use, the term is imprecise. The standards-based term is "packet delay variation" (PDV).[2] PDV is an important quality of service factor in assessment of network performance.

Compact disc seek jitter In the context of digital audio extraction from Compact Discs, seek jitter causes extracted audio samples to be doubled-up or skipped entirely if the Compact Disc drive re-seeks. The problem occurs because the Red Book does not require block-accurate addressing during seeking. As a result, the extraction process may restart a few samples early or late, resulting in doubled or omitted samples. These glitches often sound like tiny repeating clicks during playback. A successful approach to correction in software involves performing overlapping reads and fitting the data

Jitter

113 to find overlaps at the edges. Most extraction programs perform seek jitter correction. CD manufacturers avoid seek jitter by extracting the entire disc in one continuous read operation, using special CD drive models at slower speeds so the drive does not re-seek. A jitter meter is a testing instrument for measuring clock jitter values, and is used in manufacturing DVD and CD-ROM discs. Due to additional sector level addressing added in the Yellow Book, CD-ROM data discs are not subject to seek jitter.

Jitter metrics For clock jitter, there are three commonly used metrics: absolute jitter, period jitter, and cycle to cycle jitter. Absolute jitter is the absolute difference in the position of a clock's edge from where it would ideally be. Period jitter (aka cycle jitter) is the difference between any one clock period and the ideal/average clock period. Accordingly, it can be thought of as the discrete-time derivative of absolute jitter. Period jitter tends to be important in synchronous circuitry like digital state machines where the error-free operation of the circuitry is limited by the shortest possible clock period, and the performance of the circuitry is limited by the average clock period. Hence, synchronous circuitry benefits from minimizing period jitter, so that the shortest clock period approaches the average clock period. Cycle-to-cycle jitter is the difference in length/duration of any two adjacent clock periods. Accordingly, it can be thought of as the discrete-time derivative of period jitter. It can be important for some types of clock generation circuitry used in microprocessors and RAM interfaces. Since they have different generation mechanisms, different circuit effects, and different measurement methodology, it is useful to quantify them separately. In telecommunications, the unit used for the above types of jitter is usually the Unit Interval (abbreviated UI) which quantifies the jitter in terms of a fraction of the ideal period of a bit. This unit is useful because it scales with clock frequency and thus allows relatively slow interconnects such as T1 to be compared to higher-speed internet backbone links such as OC-192. Absolute units such as picoseconds are more common in microprocessor applications. Units of degrees and radians are also used. If jitter has a Gaussian distribution, it is usually quantified using the standard deviation of this distribution (aka. RMS). Often, jitter distribution is significantly non-Gaussian. This can occur if the jitter is caused by external sources such as power supply noise. In these cases, peak-to-peak measurements are more useful. Many efforts have been made to meaningfully quantify distributions In the normal distribution one standard deviation from the mean (dark blue) accounts for that are neither Gaussian nor have about 68% of the set, while two standard deviations from the mean (medium and dark blue) account for about 95% and three standard deviations (light, medium, and dark blue) meaningful peaks (which is the case in account for about 99.7%. all real jitter). All have shortcomings but most tend to be good enough for the purposes of engineering work. Note that typically, the reference point for jitter is defined such that the mean jitter is 0.

Jitter

114 In networking, in particular IP networks such as the Internet, jitter can refer to the variation (statistical dispersion) in the delay of the packets.

Types Random jitter Random Jitter, also called Gaussian jitter, is unpredictable electronic timing noise. Random jitter typically follows a Gaussian distribution or Normal distribution. It is believed to follow this pattern because most noise or jitter in an electrical circuit is caused by thermal noise, which has a Gaussian distribution. Another reason for random jitter to have a distribution like this is due to the central limit theorem. The central limit theorem states that composite effect of many uncorrelated noise sources, regardless of the distributions, approaches a Gaussian distribution. One of the main differences between random and deterministic jitter is that deterministic jitter is bounded and random jitter is unbounded.

Deterministic jitter Deterministic jitter is a type of clock timing jitter or data signal jitter that is predictable and reproducible. The peak-to-peak value of this jitter is bounded, and the bounds can easily be observed and predicted. Deterministic jitter can either be correlated to the data stream (data-dependent jitter) or uncorrelated to the data stream (bounded uncorrelated jitter). Examples of data-dependent jitter are duty-cycle dependent jitter (also known as duty-cycle distortion) and intersymbol interference. Deterministic jitter (or DJ) has a known non-Gaussian probability distribution. n

BER

6.4 10−10 6.7 10−11 7

10−12

7.3 10−13 7.6 10−14

Total jitter Total jitter (T) is the combination of random jitter (R) and deterministic jitter (D): T = Dpeak-to-peak + 2× n×Rrms, in which the value of n is based on the bit error rate (BER) required of the link. A common bit error rate used in communication standards such as Ethernet is 10−12.

Testing Testing for jitter and its measurement is of growing importance to electronics engineers because of increased clock frequencies in digital electronic circuitry to achieve higher device performance. Higher clock frequencies have commensurately smaller eye openings, and thus impose tighter tolerances on jitter. For example, modern computer motherboards have serial bus architectures with eye openings of 160 picoseconds or less. This is extremely small compared to parallel bus architectures with equivalent performance, which may have eye openings on the order of 1000 picoseconds.

Jitter

115 Testing of device performance for jitter tolerance often involves the injection of jitter into electronic components with specialized test equipment. Jitter is measured and evaluated in various ways depending on the type of circuitry under test. For example, jitter in serial bus architectures is measured by means of eye diagrams, according to industry accepted standards. A less direct approach—in which analog waveforms are digitized and the resulting data stream analyzed—is employed when measuring pixel jitter in frame grabbers. In all cases, the goal of jitter measurement is to verify that the jitter will not disrupt normal operation of the circuitry. There are standards for jitter measurement in serial bus architectures. The standards cover jitter tolerance, jitter transfer function and jitter generation, with the required values for these attributes varying among different applications. Where applicable, compliant systems are required to conform to these standards.

Mitigation Anti-jitter circuits Anti-jitter circuits (AJCs) are a class of electronic circuits designed to reduce the level of jitter in a regular pulse signal. AJCs operate by re-timing the output pulses so they align more closely to an idealised pulse signal. They are widely used in clock and data recovery circuits in digital communications, as well as for data sampling systems such as the analog-to-digital converter and digital-to-analog converter. Examples of anti-jitter circuits include phase-locked loop and delay-locked loop. Inside digital to analog converters jitter causes unwanted high-frequency distortions. In this case it can be suppressed with high fidelity clock signal usage.

Jitter buffers Jitter buffers or de-jitter buffers are used to counter jitter introduced by queuing in packet switched networks so that a continuous playout of audio (or video) transmitted over the network can be ensured. The maximum jitter that can be countered by a de-jitter buffer is equal to the buffering delay introduced before starting the play-out of the mediastream. In the context of packet-switched networks, the term packet delay variation is often preferred over jitter. Some systems use sophisticated delay-optimal de-jitter buffers that are capable of adapting the buffering delay to changing network jitter characteristics. These are known as adaptive de-jitter buffers and the adaptation logic is based on the jitter estimates computed from the arrival characteristics of the media packets. Adaptive de-jittering involves introducing discontinuities in the media play-out, which may appear offensive to the listener or viewer. Adaptive de-jittering is usually carried out for audio play-outs that feature a VAD/DTX encoded audio, that allows the lengths of the silence periods to be adjusted, thus minimizing the perceptual impact of the adaptation.

Dejitterizer A dejitterizer is a device that reduces jitter in a digital signal. A dejitterizer usually consists of an elastic buffer in which the signal is temporarily stored and then retransmitted at a rate based on the average rate of the incoming signal. A dejitterizer is usually ineffective in dealing with low-frequency jitter, such as waiting-time jitter.

Filtering A filter can be designed to minimize the effect of sampling jitter. For more information, see the paper by S. Ahmed and T. Chen entitled, "Minimizing the effects of sampling jitters in wireless sensors networks".

Jitter

116

Video and image jitter Video or image jitter occurs when the horizontal lines of video image frames are randomly displaced due to the corruption of synchronization signals or electromagnetic interference during video transmission. Model based dejittering study has been carried out under the framework of digital image/video restoration.

Notes [1] Wolaver, 1991, p.211 [2] RFC 3393, IP Packet Delay Variation Metric for IP Performance Metrics (IPPM), IETF (2002)

References •

This article incorporates public domain material from the General Services Administration document "Federal Standard 1037C" (http://www.its.bldrdoc.gov/fs-1037/fs-1037c.htm) (in support of MIL-STD-188).

• Trischitta, Patrick R. and Varma, Eve L. (1989). Jitter in Digital Transmission Systems. Artech. ISBN 0-89006-248-X. • Wolaver, Dan H. (1991). Phase-Locked Loop Circuit Design. Prentice Hall. ISBN 0-13-662743-9. pages 211–237.

Further reading • Levin, Igor. Terms and concepts involved with digital clocking related to Jitter issues in professional quality digital audio (http://www.antelopeaudio.com/blog/word-clock-sync-jitter/) • Li, Mike P. Jitter and Signal Integrity Verification for Synchronous and Asynchronous I/Os at Multiple to 10 GHz/Gbps (http://www.altera.com/literature/cp/cp-01049-jitter-si-verification.pdf). Presented at International Test Conference 2008. • Li, Mike P. A New Jitter Classification Method Based on Statistical, Physical, and Spectroscopic Mechanisms (http://www.altera.com/literature/cp/cp-01052-jitter-classification.pdf). Presented at DesignCon 2009. • Liu, Hui, Hong Shi, Xiaohong Jiang, and Zhe Li. Pre-Driver PDN SSN, OPD, Data Encoding, and Their Impact on SSJ (http://www.altera.com/literature/cp/cp-01055-impact-ssj.pdf). Presented at Electronics Components and Technology Conference 2009. • Miki, Ohtani, and Kowalski Jitter Requirements (https://mentor.ieee.org/802.11/dcn/04/ 11-04-1458-00-000n-jitter-requirements.ppt) (Causes, solutions and recommended values for digital audio) • Zamek, Iliya. SOC-System Jitter Resonance and Its Impact on Common Approach to the PDN Impedance (http:// www.altera.com/literature/cp/cp-01048-jitter-resonance.pdf). Presented at International Test Conference 2008.

External links • Jitter in VoIP - Causes, solutions and recommended values (http://www.en.voipforo.com/QoS/QoS_Jitter. php) • Jitter Buffer (http://searchenterprisevoice.techtarget.com/sDefinition/0,,sid66_gci906844,00.html) • Definition of Jitter in a QoS Testing Methodology (ftp://ftp.iol.unh.edu/pub/mplsServices/other/ QoS_Testing_Methodology.pdf) • An Introduction to Jitter in Communications Systems (http://www.maxim-ic.com/appnotes.cfm/an_pk/1916/ CMP/WP-34) • Jitter Specifications Made Easy (http://www.maxim-ic.com/appnotes.cfm/an_pk/377/CMP/WP-35) A Heuristic Discussion of Fibre Channel and Gigabit Ethernet Methods

Jitter

117 • Jitter in Packet Voice Networks (http://www.cisco.com/en/US/tech/tk652/tk698/ technologies_tech_note09186a00800945df.shtml) • Phabrix SxE - Hand-held Tool for eye and jitter measurement and analysis (http://www.phabrix.com)

Aliasing This article is about aliasing in signal processing, including computer graphics. For aliasing in computer programming, see aliasing (computing). In signal processing and related disciplines, aliasing is an effect that causes different signals to become indistinguishable (or aliases of one another) when sampled. It also refers to the distortion or artifact that results when the signal reconstructed from samples is different from the original continuous signal. Aliasing can occur in signals sampled in time, for instance digital audio, and is referred to as temporal aliasing. Aliasing can also occur in spatially sampled signals, for instance digital images. Aliasing in spatially sampled signals is called spatial aliasing.

Properly sampled image of brick wall.

Spatial aliasing in the form of a Moiré pattern.

Aliasing

Description When a digital image is viewed, a reconstruction is performed by a display or printer device, and by the eyes and the brain. If the image data is not properly processed during sampling or reconstruction, the reconstructed image will differ from the original image, and an alias is seen. An example of spatial aliasing is the Moiré pattern one can Aliasing example of the A letter in Times New Roman. observe in a poorly pixelized image of a brick wall. Techniques Left: aliased image, right: anti-aliased image. that avoid such poor pixelizations are called spatial anti-aliasing. Aliasing can be caused either by the sampling stage or the reconstruction stage; these may be distinguished by calling sampling aliasing prealiasing and reconstruction aliasing postaliasing. Temporal aliasing is a major concern in the sampling of video and audio signals. Music, for instance, may contain high-frequency components that are inaudible to humans. If a piece of music is sampled at 32000 samples per second (Hz), any frequency components above 16000 Hz (the Nyquist frequency) will cause aliasing when the music is reproduced by a digital to analog converter (DAC). To prevent this an anti-aliasing filter is used to remove components above the Nyquist frequency prior to sampling. In video or cinematography, temporal aliasing results from the limited frame rate, and causes the wagon-wheel effect, whereby a spoked wheel appears to rotate too slowly or even backwards. Aliasing has changed its apparent frequency of rotation. A reversal of direction can be described as a negative frequency. Temporal aliasing frequencies in video and cinematography are determined by the frame rate of the camera, but the relative intensity of the aliased frequencies is determined by the shutter timing (exposure time) or the use of a temporal aliasing reduction filter during filming.[1] Like the video camera, most sampling schemes are periodic; that is, they have a characteristic sampling frequency in time or in space. Digital cameras provide a certain number of samples (pixels) per degree or per radian, or samples per mm in the focal plane of the camera. Audio signals are sampled (digitized) with an analog-to-digital converter, which produces a constant number of samples per second. Some of the most dramatic and subtle examples of aliasing occur when the signal being sampled also has periodic content.

Bandlimited functions Main article: Nyquist–Shannon sampling theorem Actual signals have finite duration and their frequency content, as defined by the Fourier transform, has no upper bound. Some amount of aliasing always occurs when such functions are sampled. Functions whose frequency content is bounded (bandlimited) have infinite duration. If sampled at a high enough rate, determined by the bandwidth, the original function can in theory be perfectly reconstructed from the infinite set of samples.

Bandpass signals Main article: Undersampling Sometimes aliasing is used intentionally on signals with no low-frequency content, called bandpass signals. Undersampling, which creates low-frequency aliases, can produce the same result, with less effort, as frequency-shifting the signal to lower frequencies before sampling at the lower rate. Some digital channelizers exploit aliasing in this way for computational efficiency. See Sampling (signal processing), Nyquist rate (relative to sampling), and Filter bank.

118

Aliasing

119

Sampling sinusoidal functions Sinusoids are an important type of periodic function, because realistic signals are often modeled as the summation of many sinusoids of different frequencies and different amplitudes (with a Fourier series or transform). Understanding what aliasing does to the individual sinusoids is useful in understanding what happens to their sum. Here, a plot depicts a set of samples whose sample-interval is 1, and two (of many) different sinusoids that could have produced the samples. The sample-rate in this case is . For instance, if the interval is 1 second, the rate is 1 sample per second. Nine cycles of the red sinusoid and 1 cycle of the blue sinusoid span an interval of 10. The respective sinusoid frequencies are and

Two different sinusoids that fit the same set of samples.

. In general, when a sinusoid of frequency

is sampled with frequency

resulting samples are indistinguishable from those of another sinusoid of frequency The values corresponding to N ≠ 0 are called images or aliases of frequency

the

for any integer N.

In our example, the N = ±1 aliases

of are and A negative frequency is equivalent to its absolute value, because sin(−wt + θ) = sin(wt − θ + π), and cos(−wt + θ) = cos(wt − θ). Therefore we can express all the image frequencies as for any integer N (with being the actual signal frequency). Then the N = 1 alias of

is

(and vice versa).

Aliasing matters when one attempts to reconstruct the original waveform from its samples. The most common reconstruction technique produces the smallest of the frequencies. So it is usually important that be the unique minimum. A necessary and sufficient condition for that is commonly called the Nyquist frequency of a system that samples at rate is satisfied if the original signal is the blue sinusoid ( ). But if will produce the blue sinusoid instead of the red one.

where

is

In our example, the Nyquist condition the usual reconstruction method

Folding In the example above, and symmetrical around the frequency

are

And in general, as increases from 0 to decreases from to Similarly, as to

increases from

continues decreasing from

to 0. The black dots are aliases of each other. The solid red line is an example of A graph of amplitude vs frequency for a adjusting amplitude vs frequency. The dashed red lines are the corresponding paths single sinusoid at frequency and some of the aliases. of its aliases at and would look like the 4 black dots in the adjacent figure. The red lines depict the paths (loci) of the 4 dots if we were to adjust the frequency and amplitude of the sinusoid along the solid red segment (between and ). No matter what function we choose to change the

amplitude vs frequency, the graph will exhibit symmetry between 0 and This symmetry is commonly referred to as folding, and another name for (the Nyquist frequency) is folding frequency. Folding is most often observed in practice when viewing the frequency spectrum of real-valued samples using a discrete Fourier transform.

Aliasing

120

Complex sinusoids Complex sinusoids are waveforms whose samples are complex numbers, and the concept of negative frequency is necessary to distinguish them. In that case, the frequencies of the aliases are given by just: Therefore, as increases from goes from

to

up to 0. Consequently,

complex sinusoids do not exhibit folding. Complex samples of real-valued sinusoids have zero-valued imaginary parts and do exhibit folding.

Two complex sinusoids, colored gold and cyan, that fit the same sets of real and imaginary sample points when sampled at the rate (fs) indicated by the grid lines. The case shown here is:

Sample frequency When the condition

is met for the

highest frequency component of the original signal, then it is met for all the frequency components, a condition known as the Nyquist criterion. That is typically approximated by filtering the original signal to attenuate high frequency components before it is sampled. They still generate low-frequency aliases, but at very low amplitude levels, so as not to cause a problem. A filter chosen in anticipation of a certain sample frequency is called an anti-aliasing filter. The filtered signal can subsequently be reconstructed without significant additional distortion, for example by the Whittaker–Shannon interpolation formula.

Illustration of 4 waveforms reconstructed from samples taken at six different rates. Two of the waveforms are sufficiently sampled to avoid aliasing at all six rates. The other two illustrate increasing distortion (aliasing) at the lower rates.

The Nyquist criterion presumes that the frequency content of the signal being sampled has an upper bound. Implicit in that assumption is that the signal's duration has no upper bound. Similarly, the Whittaker–Shannon interpolation formula represents an interpolation filter with an unrealizable frequency response. These assumptions make up a mathematical model that is an idealized approximation, at best, to any realistic situation. The conclusion, that perfect reconstruction is possible, is mathematically correct for the model, but only an approximation for actual samples of an actual signal.

Aliasing

121

Historical usage Historically the term aliasing evolved from radio engineering because of the action of superheterodyne receivers. When the receiver shifts multiple signals down to lower frequencies, from RF to IF by heterodyning, an unwanted signal, from an RF frequency equally far from the local oscillator (LO) frequency as the desired signal, but on the wrong side of the LO, can end up at the same IF frequency as the wanted one. If it is strong enough it can interfere with reception of the desired signal. This unwanted signal is known as an image or alias of the desired signal.

Angular aliasing Aliasing occurs whenever the use of discrete elements to capture or produce a continuous signal causes frequency ambiguity. Spatial aliasing, particular of angular frequency, can occur when reproducing a light field[2] or sound field with discrete elements, as in 3D displays or wave field synthesis of sound. This aliasing is visible in images such as posters with lenticular printing: if they have low angular resolution, then as one moves past them, say from left-to-right, the 2D image does not initially change (so it appears to move left), then as one moves to the next angular image, the image suddenly changes (so it jumps right) – and the frequency and amplitude of this side-to-side movement corresponds to the angular resolution of the image (and, for frequency, the speed of the viewer's lateral movement), which is the angular aliasing of the 4D light field. The lack of parallax on viewer movement in 2D images and in 3-D film produced by stereoscopic glasses (in 3D films the effect is called "yawing", as the image appears to rotate on its axis) can similarly be seen as loss of angular resolution, all angular frequencies being aliased to 0 (constant).

More examples Online audio example The qualitative effects of aliasing can be heard in the following audio demonstration. Six sawtooth waves are played in succession, with the first two sawtooths having a fundamental frequency of 440 Hz (A4), the second two having fundamental frequency of 880 Hz (A5), and the final two at 1760 Hz (A6). The sawtooths alternate between bandlimited (non-aliased) sawtooths and aliased sawtooths and the sampling rate is 22.05 kHz. The bandlimited sawtooths are synthesized from the sawtooth waveform's Fourier series such that no harmonics above the Nyquist frequency are present. The aliasing distortion in the lower frequencies is increasingly obvious with higher fundamental frequencies, and while the bandlimited sawtooth is still clear at 1760 Hz, the aliased sawtooth is degraded and harsh with a buzzing audible at frequencies lower than the fundamental. Sawtooth aliasing demo 440 Hz bandlimited, 440 Hz aliased, 880 Hz bandlimited, 880 Hz aliased, 1760 Hz bandlimited, 1760 Hz aliased

Problems playing this file? See media help.

Aliasing

122

Direction finding A form of spatial aliasing can also occur in antenna arrays or microphone arrays used to estimate the direction of arrival of a wave signal, as in geophysical exploration by seismic waves. Waves must be sampled at more than two points per wavelength, or the wave arrival direction becomes ambiguous.[3]

Further reading "Sampling and reconstruction," Chapter 7 [4] in.

References [1] [2] [3] [4]

Tessive, LLC (2010). "Time Filter Technical Explanation" (http:/ / tessive. com/ time-filter-technical-explanation) The (New) Stanford Light Field Archive (http:/ / lightfield. stanford. edu/ lfs. html) Flanagan J.L., ‘Beamwidth and useable bandwidth of delay- steered microphone arrays’, AT&T Tech. J., 1985, 64, pp. 983–995 http:/ / graphics. stanford. edu/ ~mmp/ chapters/ pbrt_chapter7. pdf

External links • Aliasing by a sampling oscilloscope (http://www.youtube.com/watch?v=g3svU5VJ8Gk&feature=plcp) by Tektronix Application Engineer • Anti-Aliasing Filter Primer (http://lavidaleica.com/content/anti-aliasing-filter-primer)Wikipedia:Link rot by La Vida Leica discusses its purpose and effect on the image recorded. • Frequency Aliasing Demonstration (http://burtonmackenzie.com/2006/07/i-cant-drive-55.html) by Burton MacKenZie using stop frame animation and a clock. • Interactive examples demonstrating the aliasing effect (http://www.onmyphd.com/?p=aliasing)

Anti-aliasing filter An anti-aliasing filter is a filter used before a signal sampler, to restrict the bandwidth of a signal to approximately satisfy the sampling theorem. Since the theorem states that unambiguous interpretation of the signal from its samples is possible when the power of frequencies above the Nyquist frequency is zero, a real anti-aliasing filter can generally not completely satisfy the theorem. A realizable anti-aliasing filter will typically permit some aliasing to occur; the amount of aliasing that does occur depends on a design trade-off between reduction of aliasing and maintaining signal up to the Nyquist frequency and the frequency content of the input signal.

Optical applications In the case of optical image sampling, as by image sensors in digital cameras, the anti-aliasing filter is also known as an optical lowpass filter or blur filter or AA filter. The mathematics of sampling in two spatial dimensions is similar to the mathematics of time-domain sampling, but the filter implementation technologies are different. The typical implementation in digital cameras is two layers of birefringent material such as lithium niobate, which spreads each optical point into a cluster of four points. The choice of spot separation for such a filter involves a tradeoff among sharpness, aliasing, and fill factor (the ratio of the active refracting area of a microlens array to the total contiguous area occupied by the array). In a monochrome or three-CCD or Foveon X3 camera, the microlens array alone, if near 100% effective, can provide a significant anti-aliasing effect, while in color filter array (CFA, e.g. Bayer filter) cameras, an additional filter is generally needed to reduce aliasing to an acceptable level.

Anti-aliasing filter

Sensor based anti-aliasing filter simulation The Pentax K-3 from Ricoh introduced a unique digital sensor based anti-aliasing filter. The filter works by micro vibrating the sensor element. A toggle can on-off anti-aliasing filter as the world's first camera which has its capability.

Audio applications Anti-aliasing filters are commonly used at the input of digital signal processing systems, for example in sound digitization systems; similar filters are used as reconstruction filters at the output of such systems, for example in music players. In the latter case, the filter functions to prevent aliasing in the conversion of samples back to a continuous signal, where again perfect stop-band rejection would be required to guarantee zero aliasing.

Oversampling A technique known as oversampling is commonly used in audio conversion, especially audio output. The idea is to use a higher intermediate digital sample rate, so that a nearly-ideal digital filter can sharply cut off aliasing near the original low Nyquist frequency, while a much simpler analog filter can stop frequencies above the new higher Nyquist frequency. The purpose of oversampling is to relax the requirements on the anti-aliasing filter, or to further reduce the aliasing. Since the initial anti-aliasing filter is analog, oversampling allows for the filter to be cheaper because the requirements are not as stringent, and also allows the anti-aliasing filter to have a smoother frequency response, and thus a less complex phase response. On input, an initial analog anti-aliasing filter is relaxed, the signal is sampled at a high rate, and then downsampled using a nearly ideal digital anti-aliasing filter.

Bandpass signals See also: Undersampling Often, an anti-aliasing filter is a low-pass filter; however, this is not a requirement. Generalizations of the Nyquist–Shannon sampling theorem allow sampling of other band-limited passband signals instead of baseband signals. For signals that are bandwidth limited, but not centered at zero, a band-pass filter can be used as an anti-aliasing filter. For example, this could be done with a single-sideband modulated or frequency modulated signal. If one desired to sample an FM radio broadcast centered at 87.9 MHz and bandlimited to a 200 kHz band, then an appropriate anti-alias filter would be centered on 87.9 MHz with 200 kHz bandwidth (or pass-band of 87.8 MHz to 88.0 MHz), and the sampling rate would be no less than 400 kHz, but should also satisfy other constraints to prevent aliasing.

Signal overload See also: Clipping (audio) It is very important to avoid input signal overload when using an anti-aliasing filter. If the signal is strong enough, it can cause clipping at the analog-to-digital converter, even after filtering. When distortion due to clipping occurs after the anti-aliasing filter, it can create components outside the passband of the anti-aliasing filter; these components can then alias, causing the reproduction of other non-harmonically-related frequencies.

123

Anti-aliasing filter

References

Flash ADC A Flash ADC (also known as a Direct conversion ADC) is a type of analog-to-digital converter that uses a linear voltage ladder with a comparator at each "rung" of the ladder to compare the input voltage to successive reference voltages. Often these reference ladders are constructed of many resistors; however modern implementations show that capacitive voltage division is also possible. The output of these comparators is generally fed into a digital encoder which converts the inputs into a binary value (the collected outputs from the comparators can be thought of as a unary value).

Benefits and drawbacks Flash converters are extremely fast compared to many other types of ADCs which usually narrow in on the "correct" answer over a series of stages. Compared to these, a Flash converter is also quite simple and, apart from the analog comparators, only requires logic for the final conversion to binary. For best accuracy often a track-and-hold circuit is inserted in front of the ADC input. This is needed for many ADC types (like successive approximation ADC), but for Flash ADCs there is no real need for this, because the comparators are the sampling devices. A Flash converter requires a huge number of comparators compared to other ADCs, especially as the precision increases. A Flash converter requires comparators for an n-bit conversion. The size, power consumption and cost of all those comparators makes Flash converters generally impractical for precisions much greater than 8 bits (255 comparators). In place of these comparators, most other ADCs substitute more complex logic and/or analog circuitry which can be scaled more easily for increased precision.

124

Flash ADC

Implementation Flash ADCs have been implemented in many technologies, varying from silicon based bipolar (BJT) and complementary metal oxide FETs (CMOS) technologies to rarely used III-V technologies. Often this type of ADC is used as a first medium sized analog circuit verification. The earliest implementations consisted of a reference ladder of well matched resistors connected to a reference voltage. Each tap at the resistor ladder is used for one comparator, possibly preceded by an amplification stage, and thus generates a logical '0' or '1' depending if the measured voltage is above or below the reference voltage of the resistor tap. The reason to add an amplifier is twofold: it amplifies the A 2-bit Flash ADC Example Implementation with Bubble Error Correction and Digital voltage difference and therefore Encoding suppresses the comparator offset, and the kick-back noise of the comparator towards the reference ladder is also strongly suppressed. Typically designs from 4-bit up to 6-bit, and sometimes 7-bit are produced. Designs with power-saving capacitive reference ladders have been demonstrated. In addition to clocking the comparator(s), these systems also sample the reference value on the input stage. As the sampling is done at a very high rate, the leakage of the capacitors is negligible. Recently, offset calibration has been introduced into flash ADC designs. Instead of high precision analog circuits (which increase component size to suppress variation) comparators with relatively large offset errors are measured and adjusted. A test signal is applied and the offset of each comparator is calibrated to below the LSB size of the ADC. Another improvement to many flash ADCs is the inclusion of digital error correction. When the ADC is used in harsh environments or constructed from very small integrated circuit processes, there is a heightened risk a single comparator will randomly change state resulting in a wrong code. Bubble error correction is a digital correction mechanism that will prevent a comparator that has, for example, tripped high from reporting logic high if it is surrounded by comparators that are reporting logic low.

Folding ADC The number of comparators can be reduced somewhat by adding a folding circuit in front, making a so called folding ADC. Instead of using the comparators in a Flash ADC only once, during a ramp input signal, the folding ADC re-uses the comparators multiple times. If a m-times folding circuit is used in an n-bit ADC, the actual number of comparator can be reduced from to (there is always one needed to detect the range crossover). Typical folding circuits are, e.g., the Gilbert multiplier, or analog wired-or circuits.

125

Flash ADC

Application The very high sample rate of this type of ADC enable gigahertz applications like radar detection, wide band radio receivers and optical communication links. More often the flash ADC is embedded in a large IC containing many digital decoding functions. Also a small flash ADC circuit may be present inside a delta-sigma modulation loop. Flash ADCs are also used in NAND Flash Memory, where up to 3 bits are stored per cell as 8 level voltages on floating gates.

References • Analog to Digital Conversion [1] • Understanding Flash ADCs [2] • "Integrated Analog-to-Digital and Digital-to-Analog Converters ", R. van de Plassche, ADCs, Kluwer Academic Publishers, 1994. • "A Precise Four-Quadrant Multiplier with Subnanosecond Response", Barrie Gilbert, IEEE Journal of Solid-State Circuits, Vol. 3, No. 4 (1968), pp. 365-373

References [1] http:/ / hyperphysics. phy-astr. gsu. edu/ hbase/ electronic/ adc. html#c4 [2] http:/ / www. maxim-ic. com/ appnotes. cfm/ appnote_number/ 810/ CMP/ WP-17

Successive approximation ADC "Successive Approximation" redirects here. For behaviorist B.F. Skinner's method of guiding learned behavior, see Shaping (psychology). A successive approximation ADC is a type of analog-to-digital converter that converts a continuous analog waveform into a discrete digital representation via a binary search through all possible quantization levels before finally converging upon a digital output for each conversion.

126

Successive approximation ADC

127

Block diagram Key • • • • •

DAC = Digital-to-Analog converter EOC = end of conversion SAR = successive approximation register S/H = sample and hold circuit Vin = input voltage

• Vref = reference voltage

Algorithm The successive approximation Analog to digital converter circuit typically consists of four chief subcircuits: 1. A sample and hold circuit to acquire the input voltage (Vin).

Successive Approximation ADC Block Diagram

2. An analog voltage comparator that compares Vin to the output of the internal DAC and outputs the result of the comparison to the successive approximation register (SAR). 3. A successive approximation register subcircuit designed to supply an approximate digital code of Vin to the internal DAC. 4. An internal reference DAC that, for comparison with V, supplies the comparator with an analog voltage equal to the digital code output of the SARin. The successive approximation register is initialized so that the most significant bit (MSB) is equal to a digital 1. This code is fed into the DAC, which then supplies the analog equivalent of this digital code (Vref/2) into the comparator circuit for comparison with the sampled input voltage. If this analog voltage exceeds Vin the comparator causes the SAR to reset this bit; otherwise, the bit is left a 1. Then the next bit is set to 1 and the same test is done, continuing this binary search until every bit in the SAR has been tested. The resulting code is the digital approximation of the sampled input voltage and is finally output by the SAR at the end of the conversion (EOC). Mathematically, let Vin = xVref, so x in [-1, 1] is the normalized input voltage. The objective is to approximately digitize x to an accuracy of 1/2n. The algorithm proceeds as follows: 1. Initial approximation x0 = 0. 2. ith approximation xi = xi-1 - s(xi-1 - x)/2i. where, s(x) is the signum-function(sgn(x)) (+1 for x ≥ 0, -1 for x < 0). It follows using mathematical induction that |xn - x| ≤ 1/2n. As shown in the above algorithm, a SAR ADC requires: 1. 2. 3. 4. 5.

An input voltage source Vin. A reference voltage source Vref to normalize the input. A DAC to convert the ith approximation xi to a voltage. A Comparator to perform the function s(xi - x) by comparing the DAC's voltage with the input voltage. A Register to store the output of the comparator and apply xi-1 - s(xi-1 - x)/2i.

Successive approximation ADC

128

Charge-redistribution successive approximation ADC One of the most common implementations of the successive approximation ADC, the charge-redistribution successive approximation ADC, uses a charge scaling DAC. The charge scaling DAC simply consists of an array of individually switched binary-weighted capacitors. The amount of charge upon each capacitor in the array is used to perform the aforementioned binary search in conjunction with a comparator internal to the DAC and the successive approximation register.

Charge Scaling DAC

1. First, the capacitor array is completely discharged to the offset voltage of the comparator, VOS. This step provides automatic offset cancellation(i.e. The offset voltage represents nothing but dead charge which can't be juggled by the capacitors). 2. Next, all of the capacitors within the array are switched to the input signal, vIN. The capacitors now have a charge equal to their respective capacitance times the input voltage minus the offset voltage upon each of them. 3. In the third step, the capacitors are then switched so that this charge is applied across the comparator's input, creating a comparator input voltage equal to -vIN. 4. Finally, the actual conversion process proceeds. First, the MSB capacitor is switched to VREF, which corresponds to the full-scale range of the ADC. Due to the binary-weighting of the array the MSB capacitor forms a 1:1 charge divider with the rest of the array. Thus, the input voltage to the comparator is now -vIN plus VREF/2. Subsequently, if vIN is greater than VREF/2 then the comparator outputs a digital 1 as the MSB, otherwise it outputs a digital 0 as the MSB. Each capacitor is tested in the same manner until the comparator input voltage converges to the offset voltage, or at least as close as possible given the resolution of the DAC.

Use with non-ideal analog circuits When implemented as an analog circuit - where the value of each successive bit is not perfectly 2^N (e.g. 1.1, 2.12, 4.05, 8.01, etc.) - a successive approximation approach might not output the ideal value because the binary search algorithm incorrectly removes what it believes to be half of the values the unknown input cannot be. Depending on the difference between actual and ideal performance, the maximum error can easily exceed several LSBs, especially as the error 3 bits simulation of a capacitive ADC between the actual and ideal 2^N becomes large for one or more bits. Since we don't know the actual unknown input, it is therefore very important that accuracy of the analog circuit used to implement a SAR ADC be very close to the ideal 2^N values; otherwise, we cannot guarantee a best match search. RECENT IMPROVEMENTS 1. New SAR ADC include now calibration to improve their accuracy from less than 10bits to up to 18bits 2. Another new technique use non-binary weighted DAC and/or redundancy to solve the problem of non-ideal analog circuits and improve speed ADVANTAGES

Successive approximation ADC 1. The conversion time is equal to the "n" clock cycle period for an n-bit ADC. Thus conversion time is very short. For example for a 10-bit ADC with a clock frequency of 1 MHz, the conversion time will be only 10*10^-6 i.e. 10 microseconds. 2. Conversion time is constant and independent of the amplitude of analog signal V to the base A

References • R. J. Baker, CMOS Circuit Design, Layout, and Simulation, Third Edition, Wiley-IEEE, 2010. ISBN 978-0-470-88132-3

External links • Understanding SAR ADCs [1]

References [1] http:/ / www. maxim-ic. com/ appnotes. cfm/ appnote_number/ 1080/ CMP/ WP-50

Integrating ADC An integrating ADC is a type of analog-to-digital converter that converts an unknown input voltage into a digital representation through the use of an integrator. In its most basic implementation, the unknown input voltage is applied to the input of the integrator and allowed to ramp for a fixed time period (the run-up period). Then a known reference voltage of opposite polarity is applied to the integrator and is allowed to ramp until the integrator output returns to zero (the run-down period). The input voltage is computed as a function of the reference voltage, the constant run-up time period, and the measured run-down time period. The run-down time measurement is usually made in units of the converter's clock, so longer integration times allow for higher resolutions. Likewise, the speed of the converter can be improved by sacrificing resolution. Converters of this type can achieve high resolution, but often do so at the expense of speed. For this reason, these converters are not found in audio or signal processing applications. Their use is typically limited to digital voltmeters and other instruments requiring highly accurate measurements.

Basic design The basic integrating ADC circuit consists of an integrator, a switch to select between the voltage to be measured and the reference voltage, a timer that determines how long to integrate the unknown and measures how long the reference integration took, a comparator to detect zero crossing, and a controller. Depending on the implementation, a switch may also be present in parallel with the integrator capacitor to allow Basic integrator of a Dual-slope Integrating ADC. the integrator to be reset (by discharging the integrator capacitor). The The comparator, the timer, and the controller are switches will be controlled electrically by means of the converter's not shown. controller (a microprocessor or dedicated control logic). Inputs to the controller include a clock (used to measure time) and the output of a comparator used to detect when the integrator's output reaches zero. The conversion takes place in two phases: the run-up phase, where the input to the integrator is the voltage to be measured, and the run-down phase, where the input to the integrator is a known reference voltage. During the run-up phase, the switch selects the measured voltage as the input to the integrator. The integrator is allowed to ramp for a

129

Integrating ADC

130

fixed period of time to allow a charge to build on the integrator capacitor. During the run-down phase, the switch selects the reference voltage as the input to the integrator. The time that it takes for the integrator's output to return to zero is measured during this phase. In order for the reference voltage to ramp the integrator voltage down, the reference voltage needs to have a polarity opposite to that of the input voltage. In most cases, for positive input voltages, this means that the reference voltage will be negative. To handle both positive and negative input voltages, a positive and negative reference voltage is required. The selection of which reference to use during the run-down phase would be based on the polarity of the integrator output at the end of the run-up phase. That is, if the integrator's output were negative at the end of the run-up phase, a negative reference voltage would be required. If the integrator's output were positive, a positive reference voltage would be required. The basic equation for the output of the integrator (assuming a constant input) is:

Assuming that the initial integrator voltage at the start of each conversion is zero and that the integrator voltage at the end of the run down period will be zero, we have the following two equations that cover the integrator's output during the two phases of the conversion:

The two equations can be combined and solved for

Integrator output voltage in a basic dual-slope integrating ADC

, the unknown input voltage:

From the equation, one of the benefits of the dual-slope integrating ADC becomes apparent: the measurement is independent of the values of the circuit elements (R and C). This does not mean, however, that the values of R and C are unimportant in the design of a dual-slope integrating ADC (as will be explained below). Note that in the graph to the right, the voltage is shown as going up during the run-up phase and down during the run-down phase. In reality, because the integrator uses the op-amp in a negative feedback configuration, applying a positive will cause the output of the integrator to go down. The up and down more accurately refer to the process of adding charge to the integrator capacitor during the run-up phase and removing charge during the run-down phase. The resolution of the dual-slope integrating ADC is determined primarily by the length of the run-down period and by the time measurement resolution (i.e., the frequency of the controller's clock). The required resolution (in number of bits) dictates the minimum length of the run-down period for a full-scale input ( ):

During the measurement of a full-scale input, the slope of the integrator's output will be the same during the run-up and run-down phases. This also implies that the time of the run-up period and run-down period will be equal ( ) and that the total measurement time will be . Therefore, the total measurement time for a full-scale input will be based on the desired resolution and the frequency of the controller's clock:

Integrating ADC If a resolution of 16 bits is required with a controller clock of 10 MHz, the measurement time will be 13.1 milliseconds (or a sampling rate of just 76 samples per second). However, the sampling time can be improved by sacrificing resolution. If the resolution requirement is reduced to 10 bits, the measurement time is also reduced to only 0.2 milliseconds (almost 4900 samples per second).

Limitations There are limits to the maximum resolution of the dual-slope integrating ADC. It is not possible to increase the resolution of the basic dual-slope ADC to arbitrarily high values by using longer measurement times or faster clocks. Resolution is limited by: • The range of the integrating amplifier. The voltage rails on an op-amp limit the output voltage of the integrator. An input left connected to the integrator for too long will eventually cause the op amp to limit its output to some maximum value, making any calculation based on the run-down time meaningless. The integrator's resistor and capacitor are therefore chosen carefully based on the voltage rails of the op-amp, the reference voltage and expected full-scale input, and the longest run-up time needed to achieve the desired resolution. • The accuracy of the comparator used as the null detector. Wideband circuit noise limits the ability of the comparator to identify exactly when the output of the integrator has reached zero. Goerke suggests a typical limit is a comparator resolution of 1 millivolt.[1] • The quality of the integrator's capacitor. Although the integrating capacitor need not be perfectly linear, it does need to be time-invariant. Dielectric absorption causes errors.[2]

Enhancements The basic design of the dual-slope integrating ADC has a limitations in both conversion speed and resolution. A number of modifications to the basic design have been made to overcome both of these to some degree.

Run-up improvements Enhanced dual-slope The run-up phase of the basic dual-slope design integrates the input voltage for a fixed period of time. That is, it allows an unknown amount of charge to build up on the integrator's capacitor. The run-down phase is then used to measure this unknown charge to determine the unknown voltage. For a full-scale input, half of the measurement time is spent in the run-up phase. For smaller inputs, an even larger percentage of the total measurement time is spent in the run-up phase. Reducing the amount of time spent in the run-up phase can significantly reduce the total measurement time. Enhanced run-up dual-slope integrating ADC A simple way to reduce the run-up time is to increase the rate that charge accumulates on the integrator capacitor by reducing the size of the resistor used on the input, a method referred to as enhanced dual-slope. This still allows the same total amount of charge accumulation, but it does so over a smaller period of time. Using the same algorithm for the run-down phase results in the following equation for the calculation of the unknown input voltage ( ):

Note that this equation, unlike the equation for the basic dual-slope converter, has a dependence on the values of the integrator resistors. Or, more importantly, it has a dependence on the ratio of the two resistance values. This modification does nothing to improve the resolution of the converter (since it doesn't address either of the resolution

131

Integrating ADC

132

limitations noted above). Multi-slope run-up One method to improve the resolution of the converter is to artificially increase the range of the integrating amplifier during the run-up phase. As mentioned above, the purpose of the run-up phase is to add an unknown amount of charge to the integrator to be later measured during the run-down phase. Having the ability to add larger quantities of charge allows for more higher-resolution measurements. For Circuit diagram for a multi-slope run-up converter example, assume that we are capable of measuring the charge on the integrator during the run-down phase to a granularity of 1 coulomb. If our integrator amplifier limits us to being able to add only up to 16 coulombs of charge to the integrator during the run-up phase, our total measurement will be limited to 4 bits (16 possible values). If we can increase the range of the integrator to allow us to add up to 32 coulombs, our measurement resolution is increased to 5 bits. One method to increase the integrator capacity is by periodically adding or subtracting known quantities of charge during the run-up phase in order to keep the integrator's output within the range of the integrator amplifier. Then, the total amount of artificially-accumulated charge is the charge introduced by the unknown input voltage plus the sum of the known charges that were added or subtracted. The circuit diagram shown to the right is an example of how multi-slope run-up could be implemented. The concept is that the unknown input voltage, , is always applied to the integrator. Positive and negative reference voltages controlled by the two independent switches add and subtract charge as needed to keep the output of the integrator within its limits. The reference resistors,

and

are necessarily smaller than

to ensure that the references

can overcome the charge introduced by the input. A comparator is connected to the output to compare the integrator's voltage with a threshold voltage. The output of the comparator is used by the converter's controller to decide which reference voltage should be applied. This can be a relatively simple algorithm: if the integrator's output above the threshold, enable the positive reference (to cause the output to go down); if the integrator's output is below the threshold, enable the negative reference (to cause the output to go up). The controller keeps track of how often each switch is turned on in order to estimate how much additional charge was placed onto (or removed from) the integrator capacitor as a result of the reference voltages. To the right is a graph of sample output from the integrator during a multi-slope run-up. Each dashed vertical line represents a decision point by the controller where it samples the polarity of the output and chooses to apply either the positive or negative reference voltage to the input. Ideally, the output voltage of the integrator at the end of the run-up period can be represented by the following equation: Output from multi-slope run-up

where

is the sampling period,

is the number of periods in which the positive reference is switched in,

the number of periods in which the negative reference is switched in, and

is

is the total number of periods in the

run-up phase. The resolution obtained during the run-up period can be determined by making the assumption that the integrator output at the end of the run-up phase is zero. This allows us to relate the unknown input, , to just the references and the

values:

Integrating ADC

133

The resolution can be expressed in terms of the difference between single steps of the converter's output. In this case, if we solve the above equation for using and (the sum of and

must always equal

), the difference will equal the smallest resolvable quantity. This results in an

equation for the resolution of the multi-slope run-up phase (in bits) of:

Using typical values of the reference resistors

and

of 10k ohms and an input resistor of 50k ohms, we can

achieve a 16 bit resolution during the run-up phase with 655360 periods (65.5 milliseconds with a 10 MHz clock). While it is possible to continue the multi-slope run-up indefinitely, it is not possible to increase the resolution of the converter to arbitrarily high levels just by using a longer run-up time. Error is introduced into the multi-slope run-up through the action of the switches controlling the references, cross-coupling between the switches, unintended switch charge injection, mismatches in the references, and timing errors.[3] Some of this error can be reduced by careful operation of the switches.[4] In particular, during the run-up period, each switch should be activated a constant number of times. The algorithm explained above does not do this and just toggles switches as needed to keep the integrator output within the limits. Activating each switch a constant number of times makes the error related to switching approximately constant. Any output offset that is a result of the switching error can be measured and then subtracted from the result.

Run-down improvements Multi-slope run-down The simple, single-slope run-down is slow. Typically, the run down time is measured in clock ticks, so to get four digit resolution, the rundown time may take as long as 10,000 clock cycles. A multi-slope run-down can speed the measurement up without sacrificing accuracy. By using 4 slope rates that are each a power of ten more gradual than the previous, four digit resolution can be achieved in roughly 40 or fewer clock ticks—a huge speed improvement.[1]

Multi-slope run-down integrating ADC

The circuit shown to the right is an example of a multi-slope run-down circuit with four run-down slopes with each being ten times more gradual than the previous. The switches control which slope is selected. The switch containing selects the steepest slope (i.e., will cause the integrator output to move toward zero the fastest). At the start of the run-down interval, the unknown input is removed from the circuit by opening the switch connected to and closing the switch. Once the integrator's output reaches zero (and the run-down time measured), the final slope of

switch is opened and the next slope is selected by closing the

switch. This repeats until the

has reached zero. The combination of the run-down times for each of the slopes determines the

value of the unknown input. In essence, each slope adds one digit of resolution to the result. In the example circuit, the slope resistors differ by a factor of 10. This value, known as the base (

), can be any

value. As explained below, the choice of the base affects the speed of the converter and determines the number of slopes needed to achieve the desired resolution.

Integrating ADC

134

The basis of this design is the assumption that there will always be overshoot when trying to find the zero crossing at the end of a run-down interval. This will necessarily be true given any hysteresis in the output of the comparator measuring the zero crossing and due to the periodic sampling of the comparator based on the converter's clock. If we assume that the converter switches from one slope to the next in a single clock cycle (which may or may not be possible), the maximum amount of overshoot for a given slope would be the largest integrator output change in one clock period:

To overcome this overshoot, the next slope would require no more than

Output of the multi-slope run-down integrating ADC

clock cycles, which helps to place a

bound on the total time of the run-down. The time for the first-run down (using the steepest slope) is dependent on the unknown input (i.e., the amount of charge placed on the integrator capacitor during the run-up phase). At most, this will be:

where

is the maximum number of clock periods for the first slope,

is the maximum integrator voltage

at the start of the run-down phase, and is the resistor used for the first slope. The remainder of the slopes have a limited duration based on the selected base, so the remaining time of the conversion (in converter clock periods) is:

where

is the number of slopes.

Converting the measured time intervals during the multi-slope run-down into a measured voltage is similar to the charge-balancing method used in the multi-slope run-up enhancement. Each slope adds or subtracts known amounts of charge to/from the integrator capacitor. The run-up will have added some unknown amount of charge to the integrator. Then, during the run-down, the first slope subtracts a large amount of charge, the second slope adds a smaller amount of charge, etc. with each subsequent slope moving a smaller amount in the opposite direction of the previous slope with the goal of reaching closer and closer to zero. Each slope adds or subtracts a quantity of charge proportional to the slope's resistor and the duration of the slope:

is necessarily an integer and will be less than or equal to circuit above as an example, the second slope,

for the second and subsequent slopes. Using the

, can contribute the following charge,

, to the

integrator: in steps of That is,

possible values with the largest equal to the first slope's smallest step, or one (base 10) digit of resolution

per slope. Generalizing this, we can represent the number of slopes, resolution,

, in terms of the base and the required

:

Substituting this back into the equation representing the run-down time required for the second and subsequent slopes gives us this:

Integrating ADC

Which, when evaluated, shows that the minimum run-down time can be achieved using a base of e. This base may be difficult to use both in terms of complexity in the calculation of the result and of finding an appropriate resistor network, so a base of 2 or 4 would be more common. Residue ADC When using run-up enhancements like the multi-slope run-up, where a portion of the converter's resolution is resolved during the run-up phase, it is possible to eliminate the run-down phase altogether by using a second type of analog-to-digital converter.[5] At the end of the run-up phase of a multi-slope run-up conversion, there will still be an unknown amount of charge remaining on the integrator's capacitor. Instead of using a traditional run-down phase to determine this unknown charge, the unknown voltage can be converted directly by a second converter and combined with the result from the run-up phase to determine the unknown input voltage. Assuming that multi-slope run-up as described above is being used, the unknown input voltage can be related to the multi-slope run-up counters, and , and the measured integrator output voltage, using the following equation (derived from the multi-slope run-up output equation):

This equation represents the theoretical calculation of the input voltage assuming ideal components. Since the equation depends on nearly all of the circuit's parameters, any variances in reference currents, the integrator capacitor, or other values will introduce errors in the result. A calibration factor is typically included in the term to account for measured errors (or, as described in the referenced patent, to convert the residue ADC's output into the units of the run-up counters). Instead of being used to eliminate the run-down phase completely, the residue ADC can also be used to make the run-down phase more accurate than would otherwise be possible.[6] With a traditional run-down phase, the run-down time measurement period ends with the integrator output crossing through zero volts. There is a certain amount of error involved in detecting the zero crossing using a comparator (one of the short-comings of the basic dual-slope design as explained above). By using the residue ADC to rapidly sample the integrator output (synchronized with the converter controller's clock, for example), a voltage reading can be taken both immediately before and immediately after the zero crossing (as measured with a comparator). As the slope of the integrator voltage is constant during the run-down phase, the two voltage measurements can be used as inputs to an interpolation function that more accurately determines the time of the zero-crossing (i.e., with a much higher resolution than the controller's clock alone would allow).

Other improvements Continuously-integrating converter By combining some of these enhancements to the basic dual-slope design (namely multi-slope run-up and the residue ADC), it is possible to construct an integrating analog-to-digital converter that is capable of operating continuously without the need for a run-down interval.[7] Conceptually, the multi-slope run-up algorithm is allowed to operate continuously. To start a conversion, two things happen simultaneously: the residue ADC is used to measure the approximate charge currently on the integrator capacitor and the counters monitoring the multi-slope run-up are reset. At the end of a conversion period, another residue ADC reading is taken and the values of the multi-slope run-up counters are noted. The unknown input is calculated using a similar equation as used for the residue ADC, except that two output voltages are included ( representing the measured integrator voltage at the start of the conversion, and

135

Integrating ADC representing the measured integrator voltage at the end of the conversion.

Such a continuously-integrating converter is very similar to a delta-sigma analog-to-digital converter.

Calibration In most variants of the dual-slope integrating converter, the converter's performance is dependent on one or more of the circuit parameters. In the case of the basic design, the output of the converter is in terms of the reference voltage. In more advanced designs, there are also dependencies on one or more resistors used in the circuit or on the integrator capacitor being used. In all cases, even using expensive precision components there may be other effects that are not accounted for in the general dual-slope equations (dielectric effect on the capacitor or frequency or temperature dependencies on any of the components). Any of these variations result in error in the output of the converter. In the best case, this is simply gain and/or offset error. In the worst case, nonlinearity or nonmonotonicity could result. Some calibration can be performed internal to the converter (i.e., not requiring any special external input). This type of calibration would be performed every time the converter is turned on, periodically while the converter is running, or only when a special calibration mode is entered. Another type of calibration requires external inputs of known quantities (e.g., voltage standards or precision resistance references) and would typically be performed infrequently (every year for equipment used in normal conditions, more often when being used in metrology applications). Of these types of error, offset error is the simplest to correct (assuming that there is a constant offset over the entire range of the converter). This is often done internal to the converter itself by periodically taking measurements of the ground potential. Ideally, measuring the ground should always result in a zero output. Any non-zero output indicates the offset error in the converter. That is, if the measurement of ground resulted in an output of 0.001 volts, one can assume that all measurements will be offset by the same amount and can subtract 0.001 from all subsequent results. Gain error can similarly be measured and corrected internally (again assuming that there is a constant gain error over the entire output range). The voltage reference (or some voltage derived directly from the reference) can be used as the input to the converter. If the assumption is made that the voltage reference is accurate (to within the tolerances of the converter) or that the voltage reference has been externally calibrated against a voltage standard, any error in the measurement would be a gain error in the converter. If, for example, the measurement of a converter's 5 volt reference resulted in an output of 5.3 volts (after accounting for any offset error), a gain multiplier of 0.94 (5 / 5.3) can be applied to any subsequent measurement results.

Footnotes [1] Goeke, HP Journal, page 9 [2] Hewlett-Packard Catalog, 1981, page 49, stating, "For small inputs, noise becomes a problem and for large inputs, the dielectric absorption of the capacitor becomes a problem." [3] Eng 1994 [4] Eng 1994, Goeke 1989 [5] Riedel 1992 [6] Regier 2001 [7] Goeke 1992

136

Integrating ADC

References • US 5321403 (http://worldwide.espacenet.com/textdoc?DB=EPODOC&IDX=US5321403), Eng, Jr., Benjamin & Don Matson, "Multiple Slope Analog-to-Digital Converter", issued 14 June 1994 • Goeke, Wayne (April 1989), "8.5-Digit Integrating Analog-to-Digital Converter with 16-Bit, 100,000-Sample-per-Second Performance" (http://www.hpl.hp.com/hpjournal/pdfs/IssuePDFs/1989-04. pdf), HP Journal 40 (2): 8–15 • US 5117227 (http://worldwide.espacenet.com/textdoc?DB=EPODOC&IDX=US5117227), Goeke, Wayne, "Continuously-integrating high-resolution analog-to-digital converter", issued 26 May 1992 • Kester, Walt, The Data Conversion Handbook (http://www.analog.com/library/analogDialogue/archives/ 39-06/data_conversion_handbook.html), ISBN 0-7506-7841-0 • US 6243034 (http://worldwide.espacenet.com/textdoc?DB=EPODOC&IDX=US6243034), Regier, Christopher, "Integrating analog to digital converter with improved resolution", issued 5 June 2001 • US 5101206 (http://worldwide.espacenet.com/textdoc?DB=EPODOC&IDX=US5101206), Riedel, Ronald, "Integrating analog to digital converter", issued 31 March 1992

Time-stretch analog-to-digital converter The time-stretch analog-to-digital converter (TS-ADC),[1][2][3] also known as the Time Stretch Enhanced Recorder (TiSER), is an analog-to-digital converter (ADC) system that has the capability of digitizing very high bandwidth signals that cannot be captured by conventional electronic ADCs. Alternatively, it is also known as the photonic time stretch (PTS) digitizer,[4] since it uses an optical frontend. It relies on the process of time-stretch, which effectively slows down the analog signal in time (or compresses its bandwidth) before it can be digitized by a slow electronic ADC.

Background There is a huge demand for very high speed analog-to-digital converters (ADCs), as they are needed for test and measurement equipment in laboratories and in high speed data communications systems. Most of the ADCs are based purely on electronic circuits, which have limited speeds and add a lot of impairments, limiting the bandwidth of the signals that can be digitized and the achievable signal-to-noise ratio. In the TS-ADC, this limitation is overcome by time-stretching the analog signal, which effectively slows down the signal in time prior to digitization. By doing so, the bandwidth (and carrier frequency) of the signal is compressed. Electronic ADCs that would have been too slow to digitize the original signal, can now be used to capture this slowed down signal.

137

Time-stretch analog-to-digital converter

138

Operation principle The basic operating principle of the TS-ADC is shown in Fig. 1. The time-stretch processor, which is generally an optical frontend, stretches the signal in time. It also divides the signal into multiple segments using a filter, for example a wavelength division multiplexing (WDM) filter, to ensure that the stretched replica of the original analog signal segments do not overlap each other in time after stretching. The time-stretched and slowed down signal segments are then converted into digital samples by slow electronic ADCs. Finally, these samples are collected by a digital signal processor (DSP) and rearranged in a manner such that output data is the digital representation of the original analog signal. Any distortion added to the signal by the time-stretch preprocessor is also removed by the DSP.

Fig. 1 A time-stretch analog-to-digital converter (with a stretch factor of 4) is shown. The original analog signal is time-stretched and segmented with the help of a time-stretch preprocessor (generally on optical frontend). Slowed down segments are captured by conventional electronic ADCs. The digitized samples are rearranged to obtain the digital representation of the original signal.

An optical front-end is commonly used to accomplish this process of time-stretch, as shown in Fig. 2. An ultrashort optical pulse (typically 100 to 200 femtoseconds long), also called Fig. 2 Optical frontend for a time-stretch analog-to-digital converter is shown. The a supercontinuum pulse, which has a original analog signal is modulated over a chirped optical pulse (obtained by dispersing an ultra-short supercontinuum pulse). Second dispersive medium stretches the optical broad optical bandwidth, is pulse further. At the photodetector (PD) output, stretched replica of original signal is time-stretched by dispersing it in a obtained. highly dispersive medium (such as a dispersion compensating fiber). This process results in (an almost) linear time-to-wavelength mapping in the stretched pulse, because different wavelengths travel at different speeds in the dispersive medium. The obtained pulse is called a chirped pulse as its frequency is changing with time, and it is typically a few nanoseconds long. The analog signal is modulated onto this chirped pulse using an electro-optic intensity modulator. Subsequently, the modulated pulse is stretched further in the second dispersive medium which has much higher dispersion value. Finally, this obtained optical pulse is converted to electrical domain by a photodetector, giving the stretched replica of the original analog signal. For continuous operation, a train of supercontinuum pulses is used. The chirped pulses arriving at the electro-optic modulator should be wide enough (in time) such that the trailing edge of one pulse overlaps the leading edge of the next pulse. For segmentation, optical filters separate the signal into multiple wavelength channels at the output of the second dispersive medium. For each channel, a separate photodetector and backend electronic ADC is used. Finally the output of these ADCs are passed on to the DSP which generates the desired digital output.

Time-stretch analog-to-digital converter

Impulse response of the photonic time-stretch (PTS) system The PTS processor is based on specialized analog optical (or microwave photonic) fiber links such as those used in cable TV distribution. While the dispersion of fiber is a nuisance in conventional analog optical links, time-stretch technique exploits it to slow down the electrical waveform in the optical domain. In the cable TV link, the light source is a continuous-wave (CW) laser. In PTS, the source is a chirped pulse laser. In a conventional analog optical link, dispersion causes the upper and lower modulation sidebands, foptical ± felectrical, to slip in relative phase. At certain frequencies, their beats with the optical carrier interfere destructively, creating nulls in the frequency response of the system. For practical systems the first null is at tens of GHz, which is sufficient for handling most electrical signals of interest. Although it may seem that the dispersion penalty places a fundamental limit on the impulse response (or the bandwidth) of Fig. 4 Capture of a 95-GHz RF tone using the photonic time-stretch digitizer. The signal the time-stretch system, it can be is captured at an effective sample rate of 10-Terasamples-per-second. eliminated. The dispersion penalty vanishes with single-sideband modulation. Alternatively, one can use the modulator’s secondary (inverse) output port to eliminate the dispersion penalty, in much the same way as two antennas can eliminate spatial nulls in wireless communication (hence the two antennas on top of a WiFi access point). This configuration is termed phase-diversity. For illustration, two calculated complementary transfer functions from a typical phase-diverse time-stretch configuration are plotted in Fig. 4.[5] Combining the complementary outputs using a maximal ratio combining (MRC) algorithm results in a transfer function with a flat response in the frequency domain. Thus, the impulse response (bandwidth) of a time-stretch system is limited only by the bandwidth of the electro-optic modulator, which is about 120 GHz—a value that is adequate for capturing most electrical waveforms of interest. Extremely large stretch factors can be obtained using long lengths of fiber, but at the cost of larger loss—a problem that has been overcome by employing Raman amplification within the dispersive fiber itself, leading to the world’s fastest real-time digitizer,[6] as shown in Fig. 3. Also, using PTS, capture of very high frequency signals with a world record resolution in 10-GHz bandwidth range has been achieved.[7]

Comparison with time lens imaging Another technique, temporal imaging using a time lens, can also be used to slow down (mostly optical) signals in time. The time-lens concept relies on the mathematical equivalence between spatial diffraction and temporal dispersion, the so-called space-time duality.[8] A lens held at fixed distance from an object produces a magnified visible image. The lens imparts a quadratic phase shift to the spatial frequency components of the optical waves; in conjunction with the free space propagation (object to lens, lens to eye), this generates a magnified image. Owing to the mathematical equivalence between paraxial diffraction and temporal dispersion, an optical waveform can be temporally imaged by a three-step process of dispersing it in time, subjecting it to a phase shift that is quadratic in time (the time lens itself), and dispersing it again. Theoretically, a focused aberration-free image is obtained under a specific condition when the two dispersive elements and the phase shift satisfy the temporal equivalent of the classic

139

Time-stretch analog-to-digital converter lens equation. Alternatively, the time lens can be used without the second dispersive element to transfer the waveform’s temporal profile to the spectral domain, analogous to the property that an ordinary lens produces the spatial Fourier transform of an object at its focal points.[9] In contrast to the time-lens approach, PTS is not based on the space-time duality – there is no lens equation that needs to be satisfied to obtain an error-free slowed-down version of the input waveform. Time-stretch technique also offers continuous-time acquisition performance, a feature needed for mainstream applications of oscilloscopes. Another important difference between the two techniques is that the time lens requires the input signal to be subjected to high amount of dispersion before further processing. For electrical waveforms, the electronic devices that have the required characteristics: (1) high dispersion to loss ratio, (2) uniform dispersion, and (3) broad bandwidths, do not exist. This renders time lens not suitable for slowing down wideband electrical waveforms. In contrast, PTS does not have such a requirement. It was developed specifically for slowing down electrical waveforms and enable high speed digitizers.

Application to imaging and spectroscopy In addition to wideband A/D conversion, photonic time-stretch (PTS) is also an enabling technology for high-throughput real-time instrumentation such as imaging [10] and spectroscopy.[11][12] The world's fastest optical imaging method called serial time-encoded amplified microscopy (STEAM) makes use of the PTS technology to acquire image using a single-pixel photodetector and commercial ADC. Wavelength-time spectroscopy, which also relies on photonic time-stretch technique, permits real-time single-shot measurements of rapidly evolving or ﬂuctuating spectra.

References [1] A. S. Bhushan, F. Coppinger, and B. Jalali, “Time-stretched analogue-to-digital conversion," Electronics Letters vol. 34, no. 9, pp. 839–841, April 1998. (http:/ / ieeexplore. ieee. org/ xpls/ abs_all. jsp?arnumber=682797) [2] A. Fard, S. Gupta, and B. Jalali, "Photonic time-stretch digitizer and its extension to real-time spectroscopy and imaging," Laser & Photonics Reviews vol. 7, no. 2, pp. 207-263, March 2013. (http:/ / onlinelibrary. wiley. com/ doi/ 10. 1002/ lpor. 201200015/ abstract) [3] Y. Han and B. Jalali, “Photonic Time-Stretched Analog-to-Digital Converter: Fundamental Concepts and Practical Considerations," Journal of Lightwave Technology, Vol. 21, Issue 12, pp. 3085–3103, Dec. 2003. (http:/ / www. opticsinfobase. org/ abstract. cfm?& uri=JLT-21-12-3085) [4] J. Capmany and D. Novak, “Microwave photonics combines two worlds," Nature Photonics 1, 319-330 (2007). (http:/ / www. nature. com/ nphoton/ journal/ v1/ n6/ abs/ nphoton. 2007. 89. html) [5] Yan Han, Ozdal Boyraz, Bahram Jalali, "Ultrawide-Band Photonic Time-Stretch A/D Converter Employing Phase Diversity," "IEEE TRANSACTIONS ON MICROWAVE THEORY AND TECHNIQUES" VOL. 53, NO. 4, APRIL 2005 (http:/ / ieeexplore. ieee. org/ xpls/ abs_all. jsp?arnumber=1420773) [6] J. Chou, O. Boyraz, D. Solli, and B. Jalali, “Femtosecond real-time single-shot digitizer," Applied Physics Letters 91, 161105 (2007). (http:/ / scitation. aip. org/ getabs/ servlet/ GetabsServlet?prog=normal& id=APPLAB000091000016161105000001& idtype=cvips& gifs=yes) [7] S. Gupta and B. Jalali, “Time-warp correction and calibration in photonic time-stretch analog-to-digital converter," Optics Letters 33, 2674–2676 (2008). (http:/ / www. opticsinfobase. org/ abstract. cfm?uri=ol-33-22-2674) [8] B. H. Kolner and M. Nazarathy, “Temporal imaging with a time lens," Optics Letters 14, 630-632 (1989) (http:/ / www. opticsinfobase. org/ ol/ abstract. cfm?URI=ol-14-12-630) [9] J. W. Goodman, “Introduction to Fourier Optics," McGraw-Hill (1968). [10] K. Goda, K.K. Tsia, and B. Jalali, "Serial time-encoded amplified imaging for real-time observation of fast dynamic phenomena," Nature 458, 1145–1149, 2009. (http:/ / www. nature. com/ nature/ journal/ v458/ n7242/ full/ nature07980. html) [11] D. R. Solli, J. Chou, and B. Jalali, "Amplified wavelength–time transformation for real-time spectroscopy," Nature Photonics 2, 48-51, 2008. (http:/ / www. nature. com/ nphoton/ journal/ v2/ n1/ full/ nphoton. 2007. 253. html) [12] J. Chou, D. Solli, and B. Jalali, "Real-time spectroscopy with subgigahertz resolution using amplified dispersive Fourier transformation," Applied Physics Letters 92, 111102, 2008. (http:/ / apl. aip. org/ resource/ 1/ applab/ v92/ i11/ p111102_s1)

140

Time-stretch analog-to-digital converter

Other resources • G. C. Valley, “Photonic analog-to-digital converters," Opt. Express, vol. 15, no. 5, pp. 1955–1982, March 2007. (http://www.opticsinfobase.org/oe/abstract.cfm?uri=oe-15-5-1955) • Photonic Bandwidth Compression for Instantaneous Wideband A/D Conversion (PHOBIAC) project. (http:// www.darpa.mil/MTO/Programs/phobiac/index.html) • Short time Fourier transform for time-frequency analysis of ultrawideband signals (http://www.researchgate. net/publication/3091384_Time-stretched_short-time_Fourier_transform/)

141

142

Fourier Transforms, Discrete and Fast Discrete Fourier transform Fourier transforms Continuous Fourier transform Fourier series Discrete-time Fourier transform Discrete Fourier transform Fourier analysis Related transforms

In mathematics, the discrete Fourier transform (DFT) converts a finite list of equally spaced samples of a function into the list of coefficients of a finite combination of complex sinusoids, ordered by their frequencies, that has those same sample values. It can be said to convert the sampled function from its original domain (often time or position along a line) to the frequency domain.

Relationship between the (continuous) Fourier transform and the discrete Fourier transform. Left column: A continuous function (top) and its Fourier transform (bottom). Center-left column: Periodic summation of the original function (top). Fourier transform (bottom) is zero except at discrete points. The inverse transform is a sum of sinusoids called Fourier series. Center-right column: Original function is discretized (multiplied by a Dirac comb) (top). Its Fourier transform (bottom) is a periodic summation (DTFT) of the original transform. Right column: The DFT (bottom) computes discrete samples of the continuous DTFT. The inverse DFT (top) is a periodic summation of the original samples. The FFT algorithm computes one cycle of the DFT and its inverse is one cycle of the DFT inverse.

The input samples are complex numbers (in practice, usually real numbers), and the output coefficients are complex as well. The frequencies of the output sinusoids are integer multiples of a fundamental frequency, whose corresponding period is the length of the sampling interval. The combination of sinusoids obtained through the DFT is therefore periodic with that same period. The DFT differs from the discrete-time Fourier transform (DTFT) in that its input and output sequences are both finite; it is therefore said to be the Fourier analysis of finite-domain (or periodic) discrete-time functions.

Discrete Fourier transform

143

The DFT is the most important discrete transform, used to perform Fourier analysis in many practical applications. In digital signal processing, the function is any quantity or signal that varies over time, such as the pressure of a sound wave, a radio signal, or daily temperature readings, sampled over a finite time interval (often defined by a window function). In image processing, the samples can be the values of pixels along a row or column of a raster image. The DFT is also used to efficiently solve partial differential equations, and to perform other operations such as convolutions or multiplying large integers. Illustration of using Dirac comb functions and the convolution theorem to model the Since it deals with a finite amount of effects of sampling and/or periodic summation. At lower left is a DTFT, the spectral data, it can be implemented in result of sampling s(t) at intervals of T. The spectral sequences at (a) upper right and (b) computers by numerical algorithms or lower right are respectively computed from (a) one cycle of the periodic summation of even dedicated hardware. These s(t) and (b) one cycle of the periodic summation of the s(nT) sequence. The respective formulas are (a) the Fourier series integral and (b) the DFT summation. Its similarities to implementations usually employ the original transform, S(f), and its relative computational ease are often the motivation efficient fast Fourier transform (FFT) for computing a DFT sequence. algorithms;[1] so much so that the terms "FFT" and "DFT" are often used interchangeably. Prior to its current usage, the "FFT" initialism may have also been used for the ambiguous term finite Fourier transform.

Definition The sequence of N complex numbers

is transformed into an N-periodic sequence of complex

numbers:

(integers)

Each

[2]

(Eq.1)

is a complex number that encodes both amplitude and phase of a sinusoidal component of function

.

The sinusoid's frequency is k/N cycles per sample. Its amplitude and phase are:

where atan2 is the two-argument form of the arctan function. Due to periodicity (see Periodicity), the customary domain of k actually computed is [0, N-1]. That is always the case when the DFT is implemented via the Fast Fourier transform algorithm. But other common domains are [-N/2, N/2-1] (N even) and [-(N-1)/2, (N-1)/2] (N odd), as when the left and right halves of an FFT output sequence are swapped. The transform is sometimes denoted by the symbol

, as in

Eq.1 can be interpreted or derived in various ways, for example:

or

or

.[3]

Discrete Fourier transform

144

• It completely describes the discrete-time Fourier transform (DTFT) of an N-periodic sequence, which comprises only discrete frequency components. (Discrete-time Fourier transform#Periodic data) • It can also provide uniformly spaced samples of the continuous DTFT of a finite length sequence. (Sampling the DTFT) • It is the cross correlation of the input sequence, xn, and a complex sinusoid at frequency k/N. Thus it acts like a matched filter for that frequency. • It is the discrete analogy of the formula for the coefficients of a Fourier series: (Eq.2)

which is also N-periodic. In the domain

this is the inverse transform of Eq.1.

The normalization factor multiplying the DFT and IDFT (here 1 and 1/N) and the signs of the exponents are merely conventions, and differ in some treatments. The only requirements of these conventions are that the DFT and IDFT have opposite-sign exponents and that the product of their normalization factors be 1/N. A normalization of for both the DFT and IDFT, for instance, makes the transforms unitary. In the following discussion the terms "sequence" and "vector" will be considered interchangeable.

Properties Completeness The discrete Fourier transform is an invertible, linear transformation

with

denoting the set of complex numbers. In other words, for any N > 0, an N-dimensional complex vector has a

DFT and an IDFT which are in turn N-dimensional complex vectors.

Orthogonality The vectors

form an orthogonal basis over the set of N-dimensional

complex vectors:

where

is the Kronecker delta. (In the last step, the summation is trivial if

, where it is 1+1+⋅⋅⋅=N, and

otherwise is a geometric series that can be explicitly summed to obtain zero.) This orthogonality condition can be used to derive the formula for the IDFT from the definition of the DFT, and is equivalent to the unitarity property below.

Discrete Fourier transform

145

The Plancherel theorem and Parseval's theorem If Xk and Yk are the DFTs of xn and yn respectively then the Plancherel theorem states:

where the star denotes complex conjugation. Parseval's theorem is a special case of the Plancherel theorem and states:

These theorems are also equivalent to the unitary condition below.

Periodicity The periodicity can be shown directly from the definition:

Similarly, it can be shown that the IDFT formula leads to a periodic extension.

Shift theorem Multiplying

by a linear phase

is replaced by shift of the input

for some integer m corresponds to a circular shift of the output

:

, where the subscript is interpreted modulo N (i.e., periodically). Similarly, a circular corresponds to multiplying the output

by a linear phase. Mathematically, if

represents the vector x then if then and

Circular convolution theorem and cross-correlation theorem The convolution theorem for the discrete-time Fourier transform indicates that a convolution of two infinite sequences can be obtained as the inverse transform of the product of the individual transforms. An important simplification occurs when the sequences are of finite length, N. In terms of the DFT and inverse DFT, it can be written as follows:

which is the convolution of the

sequence with a

Similarly, the cross-correlation of

and

sequence extended by periodic summation:

is given by:

When either sequence contains a string of zeros, of length L, L+1 of the circular convolution outputs are equivalent to values of Methods have also been developed to use this property as part of an efficient process that constructs

with an

or

sequence potentially much longer than the practical transform size (N). Two

Discrete Fourier transform

146

such methods are called overlap-save and overlap-add.[4] The efficiency results from the fact that a direct evaluation of either summation (above) requires operations for an output sequence of length N. An indirect method, using transforms, can take advantage of the

efficiency of the fast Fourier transform (FFT) to achieve much better

performance. Furthermore, convolutions can be used to efficiently compute DFTs via Rader's FFT algorithm and Bluestein's FFT algorithm.

Convolution theorem duality It can also be shown that:

which is the circular convolution of

and

.

Trigonometric interpolation polynomial The trigonometric interpolation polynomial for N even , for N odd, where the coefficients Xk are given by the DFT of xn above, satisfies the interpolation property

for

. For even N, notice that the Nyquist component

is handled specially.

This interpolation is not unique: aliasing implies that one could add N to any of the complex-sinusoid frequencies (e.g. changing to ) without changing the interpolation property, but giving different values in between the points. The choice above, however, is typical because it has two useful properties. First, it consists of sinusoids whose frequencies have the smallest possible magnitudes: the interpolation is bandlimited. Second, if the are real numbers, then is real as well. In contrast, the most obvious trigonometric interpolation polynomial is the one in which the frequencies range from 0 to (instead of roughly to as above), similar to the inverse DFT formula. This interpolation does not minimize the slope, and is not generally real-valued for real

; its use is a common mistake.

The unitary DFT Another way of looking at the DFT is to note that in the above discussion, the DFT can be expressed as a Vandermonde matrix:

where

is a primitive Nth root of unity. The inverse transform is then given by the inverse of the above matrix:

Discrete Fourier transform

147

With unitary normalization constants

, the DFT becomes a unitary transformation, defined by a unitary

matrix:

where det() is the determinant function. The determinant is the product of the eigenvalues, which are always

or

as described below. In a real vector space, a unitary transformation can be thought of as simply a rigid rotation of the coordinate system, and all of the properties of a rigid rotation can be found in the unitary DFT. The orthogonality of the DFT is now expressed as an orthonormality condition (which arises in many areas of mathematics as described in root of unity):

If

is defined as the unitary DFT of the vector

then

and the Plancherel theorem is expressed as:

If we view the DFT as just a coordinate transformation which simply specifies the components of a vector in a new coordinate system, then the above is just the statement that the dot product of two vectors is preserved under a unitary DFT transformation. For the special case , this implies that the length of a vector is preserved as well—this is just Parseval's theorem:

A consequence of the circular convolution theorem is that the DFT matrix

diagonalizes any circulant matrix.

Expressing the inverse DFT in terms of the DFT A useful property of the DFT is that the inverse DFT can be easily expressed in terms of the (forward) DFT, via several well-known "tricks". (For example, in computations, it is often convenient to only implement a fast Fourier transform corresponding to one transform direction and then to get the other transform direction from the first.) First, we can compute the inverse DFT by reversing the inputs (Duhamel et al., 1988):

(As usual, the subscripts are interpreted modulo N; thus, for

, we have

.)

Second, one can also conjugate the inputs and outputs:

Third, a variant of this conjugation trick, which is sometimes preferable because it requires no modification of the data values, involves swapping real and imaginary parts (which can be done on a computer simply by modifying pointers). Define swap( ) as with its real and imaginary parts swapped—that is, if then swap(

) is

. Equivalently, swap(

) equals

. Then

Discrete Fourier transform

148

That is, the inverse transform is the same as the forward transform with the real and imaginary parts swapped for both input and output, up to a normalization (Duhamel et al., 1988). The conjugation trick can also be used to define a new transform, closely related to the DFT, that is involutary—that is, which is its own inverse. In particular, is clearly its own inverse: .A closely related involutary transformation (by a factor of (1+i) /√2) is factors in

cancel the 2. For real inputs

, the real part of

, since the is none other than the

discrete Hartley transform, which is also involutary.

Eigenvalues and eigenvectors The eigenvalues of the DFT matrix are simple and well-known, whereas the eigenvectors are complicated, not unique, and are the subject of ongoing research. Consider the unitary form

defined above for the DFT of length N, where

This matrix satisfies the matrix polynomial equation:

This can be seen from the inverse properties above: operating operating

twice gives the original data in reverse order, so

four times gives back the original data and is thus the identity matrix. This means that the eigenvalues

satisfy the equation:

Therefore, the eigenvalues of

are the fourth roots of unity:

Since there are only four distinct eigenvalues for this

is +1, −1, +i, or −i. matrix, they have some multiplicity. The multiplicity

gives the number of linearly independent eigenvectors corresponding to each eigenvalue. (Note that there are N independent eigenvectors; a unitary matrix is never defective.) The problem of their multiplicity was solved by McClellan and Parks (1972), although it was later shown to have been equivalent to a problem solved by Gauss (Dickinson and Steiglitz, 1982). The multiplicity depends on the value of N modulo 4, and is given by the following table:

Multiplicities of the eigenvalues λ of the unitary DFT matrix U as a function of the transform size N (in terms of an integer m). size N λ = +1 λ = −1 λ = -i λ = +i 4m

m+1

m

m

m−1

4m + 1 m + 1

m

m

m

4m + 2 m + 1

m+1

m

m

4m + 3 m + 1

m+1

m+1 m

Otherwise stated, the characteristic polynomial of

is:

No simple analytical formula for general eigenvectors is known. Moreover, the eigenvectors are not unique because any linear combination of eigenvectors for the same eigenvalue is also an eigenvector for that eigenvalue. Various researchers have proposed different choices of eigenvectors, selected to satisfy useful properties like orthogonality and to have "simple" forms (e.g., McClellan and Parks, 1972; Dickinson and Steiglitz, 1982; Grünbaum, 1982; Atakishiyev and Wolf, 1997; Candan et al., 2000; Hanna et al., 2004; Gurevich and Hadani, 2008).

Discrete Fourier transform

149

A straightforward approach is to discretize an eigenfunction of the continuous Fourier transform, of which the most famous is the Gaussian function. Since periodic summation of the function means discretizing its frequency spectrum and discretization means periodic summation of the spectrum, the discretized and periodically summed Gaussian function yields an eigenvector of the discrete transform: •

. A closed form expression for the series is not known, but it converges rapidly.

Two other simple closed-form analytical eigenvectors for special DFT period N were found (Kong, 2008): For DFT period N = 2L + 1 = 4K +1, where K is an integer, the following is an eigenvector of DFT: • For DFT period N = 2L = 4K, where K is an integer, the following is an eigenvector of DFT: • The choice of eigenvectors of the DFT matrix has become important in recent years in order to define a discrete analogue of the fractional Fourier transform—the DFT matrix can be taken to fractional powers by exponentiating the eigenvalues (e.g., Rubio and Santhanam, 2005). For the continuous Fourier transform, the natural orthogonal eigenfunctions are the Hermite functions, so various discrete analogues of these have been employed as the eigenvectors of the DFT, such as the Kravchuk polynomials (Atakishiyev and Wolf, 1997). The "best" choice of eigenvectors to define a fractional discrete Fourier transform remains an open question, however.

Uncertainty principle If the random variable

then

is constrained by:

may be considered to represent a discrete probability mass function of n, with an associated

probability mass function constructed from the transformed variable: For the case of continuous functions P(x) and Q(k), the Heisenberg uncertainty principle states that:

where

and

are the variances of

and

respectively, with the equality attained in the case

of a suitably normalized Gaussian distribution. Although the variances may be analogously defined for the DFT, an analogous uncertainty principle is not useful, because the uncertainty will not be shift-invariant. Nevertheless, a meaningful uncertainty principle has been introduced by Massar and Spindel. However, the Hirschman uncertainty will have a useful analog for the case of the DFT. The Hirschman uncertainty principle is expressed in terms of the Shannon entropy of the two probability functions. In the discrete case, the Shannon entropies are defined as:

and

Discrete Fourier transform

150

and the Hirschman uncertainty principle becomes:

The equality is obtained for

equal to translations and modulations of a suitably normalized Kronecker comb of

period A where A is any exact integer divisor of N. The probability mass function

will then be proportional to a

suitably translated Kronecker comb of period B=N/A.

The real-input DFT If

are real numbers, as they often are in practical applications, then the DFT obeys the symmetry: where

denotes complex conjugation.

It follows that X0 and XN/2 are real-valued, and the remainder of the DFT is completely specified by just N/2-1 complex numbers.

Generalized DFT (shifted and non-linear phase) It is possible to shift the transform sampling in time and/or frequency domain by some real shifts a and b, respectively. This is sometimes known as a generalized DFT (or GDFT), also called the shifted DFT or offset DFT, and has analogous properties to the ordinary DFT:

Most often, shifts of

(half a sample) are used. While the ordinary DFT corresponds to a periodic signal in both

time and frequency domains,

produces a signal that is anti-periodic in frequency domain (

) and vice-versa for

. Thus, the specific case of

is known as an odd-time

2

odd-frequency discrete Fourier transform (or O DFT). Such shifted transforms are most often used for symmetric data, to represent different boundary symmetries, and for real-symmetric data they correspond to different forms of the discrete cosine and sine transforms. Another interesting choice is , which is called the centered DFT (or CDFT). The centered DFT has the useful property that, when N is a multiple of four, all four of its eigenvalues (see above) have equal multiplicities (Rubio and Santhanam, 2005)[5] The term GDFT is also used for the non-linear phase extensions of DFT. Hence, GDFT method provides a generalization for constant amplitude orthogonal block transforms including linear and non-linear phase types. GDFT is a framework to improve time and frequency domain properties of the traditional DFT, e.g. auto/cross-correlations, by the addition of the properly designed phase shaping function (non-linear, in general) to the original linear phase functions (Akansu and Agirman-Tosun, 2010).[6] The discrete Fourier transform can be viewed as a special case of the z-transform, evaluated on the unit circle in the complex plane; more general z-transforms correspond to complex shifts a and b above.

Multidimensional DFT The ordinary DFT transforms a one-dimensional sequence or array variable n. The multidimensional DFT of a multidimensional array variables

for

where more

in

that is a function of exactly one discrete that is a function of d discrete

is defined by:

as above and the d output indices run from compactly

expressed

in

vector

notation,

where

we

define

. This is and

Discrete Fourier transform

151 as d-dimensional vectors of indices from 0 to

where the division

is defined as

, which we define as

to be performed element-wise, and the

sum denotes the set of nested summations above. The inverse of the multi-dimensional DFT is, analogous to the one-dimensional case, given by:

As the one-dimensional DFT expresses the input as a superposition of sinusoids, the multidimensional DFT expresses the input as a superposition of plane waves, or multidimensional sinusoids. The direction of oscillation in space is . The amplitudes are . This decomposition is of great importance for everything from digital image processing (two-dimensional) to solving partial differential equations. The solution is broken up into plane waves. The multidimensional DFT can be computed by the composition of a sequence of one-dimensional DFTs along each dimension. In the two-dimensional case the independent DFTs of the rows (i.e., along ) are computed first to form a new array

. Then the

independent DFTs of y along the columns (along

) are

computed to form the final result

. Alternatively the columns can be computed first and then the rows. The

order is immaterial because the nested summations above commute. An algorithm to compute a one-dimensional DFT is thus sufficient to efficiently compute a multidimensional DFT. This approach is known as the row-column algorithm. There are also intrinsically multidimensional FFT algorithms.

The real-input multidimensional DFT For input data

consisting of real numbers, the DFT outputs have a conjugate symmetry similar to the

one-dimensional case above:

where the star again denotes complex conjugation and the

-th subscript is again interpreted modulo

(for

).

Applications The DFT has seen wide usage across a large number of fields; we only sketch a few examples below (see also the references at the end). All applications of the DFT depend crucially on the availability of a fast algorithm to compute discrete Fourier transforms and their inverses, a fast Fourier transform.

Spectral analysis When the DFT is used for spectral analysis, the time-samples of some signal

sequence usually represents a finite set of uniformly spaced

, where t represents time. The conversion from continuous time to samples

(discrete-time) changes the underlying Fourier transform of x(t) into a discrete-time Fourier transform (DTFT), which generally entails a type of distortion called aliasing. Choice of an appropriate sample-rate (see Nyquist rate) is the key to minimizing that distortion. Similarly, the conversion from a very long (or infinite) sequence to a manageable size entails a type of distortion called leakage, which is manifested as a loss of detail (aka resolution) in the DTFT. Choice of an appropriate sub-sequence length is the primary key to minimizing that effect. When the available data (and time to process it) is more than the amount needed to attain the desired frequency resolution, a standard technique is to perform multiple DFTs, for example to create a spectrogram. If the desired result is a power spectrum and noise or randomness is present in the data, averaging the magnitude components of the multiple DFTs

Discrete Fourier transform is a useful procedure to reduce the variance of the spectrum (also called a periodogram in this context); two examples of such techniques are the Welch method and the Bartlett method; the general subject of estimating the power spectrum of a noisy signal is called spectral estimation. A final source of distortion (or perhaps illusion) is the DFT itself, because it is just a discrete sampling of the DTFT, which is a function of a continuous frequency domain. That can be mitigated by increasing the resolution of the DFT. That procedure is illustrated at Sampling the DTFT. • The procedure is sometimes referred to as zero-padding, which is a particular implementation used in conjunction with the fast Fourier transform (FFT) algorithm. The inefficiency of performing multiplications and additions with zero-valued "samples" is more than offset by the inherent efficiency of the FFT. • As already noted, leakage imposes a limit on the inherent resolution of the DTFT. So there is a practical limit to the benefit that can be obtained from a fine-grained DFT.

Filter bank See FFT filter banks and Sampling the DTFT.

Data compression The field of digital signal processing relies heavily on operations in the frequency domain (i.e. on the Fourier transform). For example, several lossy image and sound compression methods employ the discrete Fourier transform: the signal is cut into short segments, each is transformed, and then the Fourier coefficients of high frequencies, which are assumed to be unnoticeable, are discarded. The decompressor computes the inverse transform based on this reduced number of Fourier coefficients. (Compression applications often use a specialized form of the DFT, the discrete cosine transform or sometimes the modified discrete cosine transform.) Some relatively recent compression algorithms, however, use wavelet transforms, which give a more uniform compromise between time and frequency domain than obtained by chopping data into segments and transforming each segment. In the case of JPEG2000, this avoids the spurious image features that appear when images are highly compressed with the original JPEG.

Partial differential equations Discrete Fourier transforms are often used to solve partial differential equations, where again the DFT is used as an approximation for the Fourier series (which is recovered in the limit of infinite N). The advantage of this approach is that it expands the signal in complex exponentials einx, which are eigenfunctions of differentiation: d/dx einx = in einx. Thus, in the Fourier representation, differentiation is simple—we just multiply by i n. (Note, however, that the choice of n is not unique due to aliasing; for the method to be convergent, a choice similar to that in the trigonometric interpolation section above should be used.) A linear differential equation with constant coefficients is transformed into an easily solvable algebraic equation. One then uses the inverse DFT to transform the result back into the ordinary spatial representation. Such an approach is called a spectral method.

152

Discrete Fourier transform

153

Polynomial multiplication Suppose we wish to compute the polynomial product c(x) = a(x) · b(x). The ordinary product expression for the coefficients of c involves a linear (acyclic) convolution, where indices do not "wrap around." This can be rewritten as a cyclic convolution by taking the coefficient vectors for a(x) and b(x) with constant term first, then appending zeros so that the resultant coefficient vectors a and b have dimension d > deg(a(x)) + deg(b(x)). Then,

Where c is the vector of coefficients for c(x), and the convolution operator

is defined so

But convolution becomes multiplication under the DFT:

Here the vector product is taken elementwise. Thus the coefficients of the product polynomial c(x) are just the terms 0, ..., deg(a(x)) + deg(b(x)) of the coefficient vector

With a fast Fourier transform, the resulting algorithm takes O (N log N) arithmetic operations. Due to its simplicity and speed, the Cooley–Tukey FFT algorithm, which is limited to composite sizes, is often chosen for the transform operation. In this case, d should be chosen as the smallest integer greater than the sum of the input polynomial degrees that is factorizable into small prime factors (e.g. 2, 3, and 5, depending upon the FFT implementation). Multiplication of large integers The fastest known algorithms for the multiplication of very large integers use the polynomial multiplication method outlined above. Integers can be treated as the value of a polynomial evaluated specifically at the number base, with the coefficients of the polynomial corresponding to the digits in that base. After polynomial multiplication, a relatively low-complexity carry-propagation step completes the multiplication. Convolution When data is convolved with a function with wide support, such as for downsampling by a large sampling ratio, because of the Convolution theorem and the FFT algorithm, it may be faster to transform it, multiply pointwise by the transform of the filter and then reverse transform it. Alternatively, a good filter is obtained by simply truncating the transformed data and re-transforming the shortened data set.

Some discrete Fourier transform pairs Some DFT pairs Note

Shift theorem

Real DFT from the geometric progression formula

from the binomial theorem

Discrete Fourier transform

154 is a rectangular window function of W points centered on n=0, where W is an odd integer, and is a sinc-like function (specifically,

is a Dirichlet

kernel) Discretization and periodic summation of the scaled Gaussian functions for

. Since either

or

is larger than one and thus warrants fast convergence of one of the two series, for large you may choose to compute the frequency spectrum and convert to the time domain using the discrete Fourier transform.

Generalizations Representation theory For more details on this topic, see Representation theory of finite groups § Discrete Fourier transform. The DFT can be interpreted as the complex-valued representation theory of the finite cyclic group. In other words, a sequence of n complex numbers can be thought of as an element of n-dimensional complex space Cn or equivalently a function f from the finite cyclic group of order n to the complex numbers, Zn → C. So f is a class function on the finite cyclic group, and thus can be expressed as a linear combination of the irreducible characters of this group, which are the roots of unity. From this point of view, one may generalize the DFT to representation theory generally, or more narrowly to the representation theory of finite groups. More narrowly still, one may generalize the DFT by either changing the target (taking values in a field other than the complex numbers), or the domain (a group other than a finite cyclic group), as detailed in the sequel.

Other fields Main articles: Discrete Fourier transform (general) and Number-theoretic transform Many of the properties of the DFT only depend on the fact that denoted

or

(so that

is a primitive root of unity, sometimes

). Such properties include the completeness, orthogonality,

Plancherel/Parseval, periodicity, shift, convolution, and unitarity properties above, as well as many FFT algorithms. For this reason, the discrete Fourier transform can be defined by using roots of unity in fields other than the complex numbers, and such generalizations are commonly called number-theoretic transforms (NTTs) in the case of finite fields. For more information, see number-theoretic transform and discrete Fourier transform (general).

Other finite groups Main article: Fourier transform on finite groups The standard DFT acts on a sequence x0, x1, …, xN−1 of complex numbers, which can be viewed as a function {0, 1, …, N − 1} → C. The multidimensional DFT acts on multidimensional sequences, which can be viewed as functions

This suggests the generalization to Fourier transforms on arbitrary finite groups, which act on functions G → C where G is a finite group. In this framework, the standard DFT is seen as the Fourier transform on a cyclic group, while the multidimensional DFT is a Fourier transform on a direct sum of cyclic groups.

Discrete Fourier transform

Alternatives Main article: Discrete wavelet transform For more details on this topic, see Discrete wavelet transform § Comparison with Fourier transform. There are various alternatives to the DFT for various applications, prominent among which are wavelets. The analog of the DFT is the discrete wavelet transform (DWT). From the point of view of time–frequency analysis, a key limitation of the Fourier transform is that it does not include location information, only frequency information, and thus has difficulty in representing transients. As wavelets have location as well as frequency, they are better able to represent location, at the expense of greater difficulty representing frequency. For details, see comparison of the discrete wavelet transform with the discrete Fourier transform.

Notes [1] Cooley et al., 1969 [2] In this context, it is common to define UNIQ-math-0-fdcc463dc40e329d-QINU to be the Nth primitive root of unity, UNIQ-math-1-fdcc463dc40e329d-QINU , to obtain the following form:

[3] As a linear transformation on a finite-dimensional vector space, the DFT expression can also be written in terms of a DFT matrix; when scaled appropriately it becomes a unitary matrix and the Xk can thus be viewed as coefficients of x in an orthonormal basis. [4] T. G. Stockham, Jr., " High-speed convolution and correlation (http:/ / dl. acm. org/ citation. cfm?id=1464209)," in 1966 Proc. AFIPS Spring Joint Computing Conf. Reprinted in Digital Signal Processing, L. R. Rabiner and C. M. Rader, editors, New York: IEEE Press, 1972. [5] Santhanam, Balu; Santhanam, Thalanayar S. "Discrete Gauss-Hermite functions and eigenvectors of the centered discrete Fourier transform" (http:/ / thamakau. usc. edu/ Proceedings/ ICASSP 2007/ pdfs/ 0301385. pdf), Proceedings of the 32nd IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2007, SPTM-P12.4), vol. III, pp. 1385-1388. [6] Akansu, Ali N.; Agirman-Tosun, Handan "Generalized Discrete Fourier Transform With Nonlinear Phase" (http:/ / web. njit. edu/ ~akansu/ PAPERS/ AkansuIEEE-TSP2010. pdf), IEEE Transactions on Signal Processing, vol. 58, no. 9, pp. 4547-4556, Sept. 2010.

Citations References • Brigham, E. Oran (1988). The fast Fourier transform and its applications. Englewood Cliffs, N.J.: Prentice Hall. ISBN 0-13-307505-2. • Oppenheim, Alan V.; Schafer, R. W.; and Buck, J. R. (1999). Discrete-time signal processing. Upper Saddle River, N.J.: Prentice Hall. ISBN 0-13-754920-2. • Smith, Steven W. (1999). "Chapter 8: The Discrete Fourier Transform" (http://www.dspguide.com/ch8/1. htm). The Scientist and Engineer's Guide to Digital Signal Processing (Second ed.). San Diego, Calif.: California Technical Publishing. ISBN 0-9660176-3-3. • Cormen, Thomas H.; Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein (2001). "Chapter 30: Polynomials and the FFT". Introduction to Algorithms (Second ed.). MIT Press and McGraw-Hill. pp. 822–848. ISBN 0-262-03293-7. esp. section 30.2: The DFT and FFT, pp. 830–838. • P. Duhamel, B. Piron, and J. M. Etcheto (1988). "On computing the inverse DFT". IEEE Trans. Acoust., Speech and Sig. Processing 36 (2): 285–286. doi: 10.1109/29.1519 (http://dx.doi.org/10.1109/29.1519). • J. H. McClellan and T. W. Parks (1972). "Eigenvalues and eigenvectors of the discrete Fourier transformation". IEEE Trans. Audio Electroacoust. 20 (1): 66–74. doi: 10.1109/TAU.1972.1162342 (http://dx.doi.org/10.1109/ TAU.1972.1162342). • Bradley W. Dickinson and Kenneth Steiglitz (1982). "Eigenvectors and functions of the discrete Fourier transform". IEEE Trans. Acoust., Speech and Sig. Processing 30 (1): 25–31. doi: 10.1109/TASSP.1982.1163843 (http://dx.doi.org/10.1109/TASSP.1982.1163843). (Note that this paper has an apparent typo in its table of the eigenvalue multiplicities: the +i/−i columns are interchanged. The correct table can be found in McClellan and

155

Discrete Fourier transform

• •

•

•

•

Parks, 1972, and is easily confirmed numerically.) F. A. Grünbaum (1982). "The eigenvectors of the discrete Fourier transform". J. Math. Anal. Appl. 88 (2): 355–363. doi: 10.1016/0022-247X(82)90199-8 (http://dx.doi.org/10.1016/0022-247X(82)90199-8). Natig M. Atakishiyev and Kurt Bernardo Wolf (1997). "Fractional Fourier-Kravchuk transform". J. Opt. Soc. Am. A 14 (7): 1467–1477. Bibcode: 1997JOSAA..14.1467A (http://adsabs.harvard.edu/abs/1997JOSAA..14. 1467A). doi: 10.1364/JOSAA.14.001467 (http://dx.doi.org/10.1364/JOSAA.14.001467). C. Candan, M. A. Kutay and H. M.Ozaktas (2000). "The discrete fractional Fourier transform". IEEE Trans. on Signal Processing 48 (5): 1329–1337. Bibcode: 2000ITSP...48.1329C (http://adsabs.harvard.edu/abs/ 2000ITSP...48.1329C). doi: 10.1109/78.839980 (http://dx.doi.org/10.1109/78.839980). Magdy Tawfik Hanna, Nabila Philip Attalla Seif, and Waleed Abd El Maguid Ahmed (2004). "Hermite-Gaussian-like eigenvectors of the discrete Fourier transform matrix based on the singular-value decomposition of its orthogonal projection matrices". IEEE Trans. Circ. Syst. I 51 (11): 2245–2254. doi: 10.1109/TCSI.2004.836850 (http://dx.doi.org/10.1109/TCSI.2004.836850). Shamgar Gurevich and Ronny Hadani (2009). "On the diagonalization of the discrete Fourier transform". Applied and Computational Harmonic Analysis 27 (1): 87–99. arXiv: 0808.3281 (http://arxiv.org/abs/0808.3281). doi: 10.1016/j.acha.2008.11.003 (http://dx.doi.org/10.1016/j.acha.2008.11.003). preprint at.

• Shamgar Gurevich, Ronny Hadani, and Nir Sochen (2008). "The finite harmonic oscillator and its applications to sequences, communication and radar". IEEE Transactions on Information Theory 54 (9): 4239–4253. arXiv: 0808.1495 (http://arxiv.org/abs/0808.1495). doi: 10.1109/TIT.2008.926440 (http://dx.doi.org/10.1109/ TIT.2008.926440). preprint at. • Juan G. Vargas-Rubio and Balu Santhanam (2005). "On the multiangle centered discrete fractional Fourier transform". IEEE Sig. Proc. Lett. 12 (4): 273–276. Bibcode: 2005ISPL...12..273V (http://adsabs.harvard.edu/ abs/2005ISPL...12..273V). doi: 10.1109/LSP.2005.843762 (http://dx.doi.org/10.1109/LSP.2005. 843762). • J. Cooley, P. Lewis, and P. Welch (1969). "The finite Fourier transform". IEEE Trans. Audio Electroacoustics 17 (2): 77–85. doi: 10.1109/TAU.1969.1162036 (http://dx.doi.org/10.1109/TAU.1969.1162036). • F.N. Kong (2008). "Analytic Expressions of Two Discrete Hermite-Gaussian Signals". IEEE Trans. Circuits and Systems –II: Express Briefs. 55 (1): 56–60. doi: 10.1109/TCSII.2007.909865 (http://dx.doi.org/10.1109/ TCSII.2007.909865).

External links • Matlab tutorial on the Discrete Fourier Transformation (http://www.nbtwiki.net/doku. php?id=tutorial:the_discrete_fourier_transformation_dft) • Interactive flash tutorial on the DFT (http://www.fourier-series.com/fourierseries2/DFT_tutorial.html) • Mathematics of the Discrete Fourier Transform by Julius O. Smith III (http://ccrma.stanford.edu/~jos/mdft/ mdft.html) • Fast implementation of the DFT - coded in C and under General Public License (GPL) (http://www.fftw.org) • The DFT “à Pied”: Mastering The Fourier Transform in One Day (http://www.dspdimension.com/admin/ dft-a-pied/) • Explained: The Discrete Fourier Transform (http://web.mit.edu/newsoffice/2009/explained-fourier.html) • wavetable Cooker (http://noisemakessound.com/blofeld-wavetable-cooker/) GPL application with graphical interface written in C, and implementing DFT IDFT to generate a wavetable set

156

Fast Fourier transform

157

Fast Fourier transform "FFT" redirects here. For other uses, see FFT (disambiguation). A fast Fourier transform (FFT) is an algorithm to compute the discrete Fourier transform (DFT) and its inverse. Fourier analysis converts time (or space) to frequency and vice versa; an FFT rapidly computes such transformations by factorizing the DFT matrix into a product of sparse (mostly zero) factors.[1] As a result, fast Fourier transforms are widely used for many applications in engineering, science, and mathematics. The basic ideas were popularized in 1965, but some FFTs had been previously known as early as 1805. Fast Fourier transforms have been described as "the most important numerical algorithm[s] of our lifetime".

Frequency and time domain for the same signal

Overview There are many different FFT algorithms involving a wide range of mathematics, from simple complex-number arithmetic to group theory and number theory; this article gives an overview of the available techniques and some of their general properties, while the specific algorithms are described in subsidiary articles linked below. The DFT is obtained by decomposing a sequence of values into components of different frequencies. This operation is useful in many fields (see discrete Fourier transform for properties and applications of the transform) but computing it directly from the definition is often too slow to be practical. An FFT is a way to compute the same result more quickly: computing the DFT of N points in the naive way, using the definition, takes O(N2) arithmetical operations, while a FFT can compute the same DFT in only O(N log N) operations. The difference in speed can be enormous, especially for long data sets where N may be in the thousands or millions. In practice, the computation time can be reduced by several orders of magnitude in such cases, and the improvement is roughly proportional to N / log(N). This huge improvement made the calculation of the DFT practical; FFTs are of great importance to a wide variety of applications, from digital signal processing and solving partial differential equations to algorithms for quick multiplication of large integers. The best-known FFT algorithms depend upon the factorization of N, but there are FFTs with O(N log N) complexity for all N, even for prime N. Many FFT algorithms only depend on the fact that is an N-th primitive root of unity, and thus can be applied to analogous transforms over any finite field, such as number-theoretic transforms. Since the inverse DFT is the same as the DFT, but with the opposite sign in the exponent and a 1/N factor, any FFT algorithm can easily be adapted for it.

Definition and speed An FFT computes the DFT and produces exactly the same result as evaluating the DFT definition directly; the most important difference is that an FFT is much faster. (In the presence of round-off error, many FFT algorithms are also much more accurate than evaluating the DFT definition directly, as discussed below.) Let x0, ...., xN-1 be complex numbers. The DFT is defined by the formula

Evaluating this definition directly requires O(N2) operations: there are N outputs Xk, and each output requires a sum of N terms. An FFT is any method to compute the same results in O(N log N) operations. More precisely, all known FFT algorithms require Θ(N log N) operations (technically, O only denotes an upper bound), although there is no

Fast Fourier transform known proof that a lower complexity score is impossible.(Johnson and Frigo, 2007) To illustrate the savings of an FFT, consider the count of complex multiplications and additions. Evaluating the DFT's sums directly involves N2 complex multiplications and N(N−1) complex additions [of which O(N) operations can be saved by eliminating trivial operations such as multiplications by 1]. The well-known radix-2 Cooley–Tukey algorithm, for N a power of 2, can compute the same result with only (N/2)log2(N) complex multiplications (again, ignoring simplifications of multiplications by 1 and similar) and Nlog2(N) complex additions. In practice, actual performance on modern computers is usually dominated by factors other than the speed of arithmetic operations and the analysis is a complicated subject (see, e.g., Frigo & Johnson, 2005), but the overall improvement from O(N2) to O(N log N) remains.

Algorithms Cooley–Tukey algorithm Main article: Cooley–Tukey FFT algorithm By far the most commonly used FFT is the Cooley–Tukey algorithm. This is a divide and conquer algorithm that recursively breaks down a DFT of any composite size N = N1N2 into many smaller DFTs of sizes N1 and N2, along with O(N) multiplications by complex roots of unity traditionally called twiddle factors (after Gentleman and Sande, 1966). This method (and the general idea of an FFT) was popularized by a publication of J. W. Cooley and J. W. Tukey in 1965, but it was later discovered (Heideman, Johnson, & Burrus, 1984) that those two authors had independently re-invented an algorithm known to Carl Friedrich Gauss around 1805 (and subsequently rediscovered several times in limited forms). The best known use of the Cooley–Tukey algorithm is to divide the transform into two pieces of size N/2 at each step, and is therefore limited to power-of-two sizes, but any factorization can be used in general (as was known to both Gauss and Cooley/Tukey). These are called the radix-2 and mixed-radix cases, respectively (and other variants such as the split-radix FFT have their own names as well). Although the basic idea is recursive, most traditional implementations rearrange the algorithm to avoid explicit recursion. Also, because the Cooley–Tukey algorithm breaks the DFT into smaller DFTs, it can be combined arbitrarily with any other algorithm for the DFT, such as those described below.

Other FFT algorithms Main articles: Prime-factor FFT algorithm, Bruun's FFT algorithm, Rader's FFT algorithm and Bluestein's FFT algorithm There are other FFT algorithms distinct from Cooley–Tukey. Cornelius Lanczos did pioneering work on the FFS and FFT with G.C. Danielson (1940). For N = N1N2 with coprime N1 and N2, one can use the Prime-Factor (Good-Thomas) algorithm (PFA), based on the Chinese Remainder Theorem, to factorize the DFT similarly to Cooley–Tukey but without the twiddle factors. The Rader-Brenner algorithm (1976) is a Cooley–Tukey-like factorization but with purely imaginary twiddle factors, reducing multiplications at the cost of increased additions and reduced numerical stability; it was later superseded by the split-radix variant of Cooley–Tukey (which achieves the same multiplication count but with fewer additions and without sacrificing accuracy). Algorithms that recursively factorize the DFT into smaller operations other than DFTs include the Bruun and QFT algorithms. (The Rader-Brenner and QFT algorithms were proposed for power-of-two sizes, but it is possible that they could be adapted to general composite n. Bruun's algorithm applies to arbitrary even composite sizes.) Bruun's algorithm, in particular, is based on interpreting the FFT as a recursive factorization of the polynomial zN−1, here into real-coefficient polynomials of the form zM−1 and z2M + azM + 1.

158

Fast Fourier transform

159

Another polynomial viewpoint is exploited by the Winograd algorithm, which factorizes zN−1 into cyclotomic polynomials—these often have coefficients of 1, 0, or −1, and therefore require few (if any) multiplications, so Winograd can be used to obtain minimal-multiplication FFTs and is often used to find efficient algorithms for small factors. Indeed, Winograd showed that the DFT can be computed with only O(N) irrational multiplications, leading to a proven achievable lower bound on the number of multiplications for power-of-two sizes; unfortunately, this comes at the cost of many more additions, a tradeoff no longer favorable on modern processors with hardware multipliers. In particular, Winograd also makes use of the PFA as well as an algorithm by Rader for FFTs of prime sizes. Rader's algorithm, exploiting the existence of a generator for the multiplicative group modulo prime N, expresses a DFT of prime size n as a cyclic convolution of (composite) size N−1, which can then be computed by a pair of ordinary FFTs via the convolution theorem (although Winograd uses other convolution methods). Another prime-size FFT is due to L. I. Bluestein, and is sometimes called the chirp-z algorithm; it also re-expresses a DFT as a convolution, but this time of the same size (which can be zero-padded to a power of two and evaluated by radix-2 Cooley–Tukey FFTs, for example), via the identity .

FFT algorithms specialized for real and/or symmetric data In many applications, the input data for the DFT are purely real, in which case the outputs satisfy the symmetry

and efficient FFT algorithms have been designed for this situation (see e.g. Sorensen, 1987). One approach consists of taking an ordinary algorithm (e.g. Cooley–Tukey) and removing the redundant parts of the computation, saving roughly a factor of two in time and memory. Alternatively, it is possible to express an even-length real-input DFT as a complex DFT of half the length (whose real and imaginary parts are the even/odd elements of the original real data), followed by O(N) post-processing operations. It was once believed that real-input DFTs could be more efficiently computed by means of the discrete Hartley transform (DHT), but it was subsequently argued that a specialized real-input DFT algorithm (FFT) can typically be found that requires fewer operations than the corresponding DHT algorithm (FHT) for the same number of inputs. Bruun's algorithm (above) is another method that was initially proposed to take advantage of real inputs, but it has not proved popular. There are further FFT specializations for the cases of real data that have even/odd symmetry, in which case one can gain another factor of (roughly) two in time and memory and the DFT becomes the discrete cosine/sine transform(s) (DCT/DST). Instead of directly modifying an FFT algorithm for these cases, DCTs/DSTs can also be computed via FFTs of real data combined with O(N) pre/post processing.

Computational issues Bounds on complexity and operation counts List of unsolved problems in computer science What is the lower bound on the complexity of fast Fourier transform algorithms? Can they be faster than Θ(N log N)?

A fundamental question of longstanding theoretical interest is to prove lower bounds on the complexity and exact operation counts of fast Fourier transforms, and many open problems remain. It is not even rigorously proved whether DFTs truly require Ω(N log(N)) (i.e., order N log(N) or greater) operations, even for the simple case of power of two sizes, although no algorithms with lower complexity are known. In particular, the count of arithmetic operations is usually the focus of such questions, although actual performance on modern-day computers is determined by many other factors such as cache or CPU pipeline optimization.

Fast Fourier transform

160

Following pioneering work by Winograd (1978), a tight Θ(N) lower bound is known for the number of real multiplications required by an FFT. It can be shown that only irrational real multiplications are required to compute a DFT of power-of-two length

. Moreover, explicit algorithms

that achieve this count are known (Heideman & Burrus, 1986; Duhamel, 1990). Unfortunately, these algorithms require too many additions to be practical, at least on modern computers with hardware multipliers.Wikipedia:Citation needed A tight lower bound is not known on the number of required additions, although lower bounds have been proved under some restrictive assumptions on the algorithms. In 1973, Morgenstern proved an Ω(N log(N)) lower bound on the addition count for algorithms where the multiplicative constants have bounded magnitudes (which is true for most but not all FFT algorithms). Pan (1986) proved an Ω(N log(N)) lower bound assuming a bound on a measure of the FFT algorithm's "asynchronicity", but the generality of this assumption is unclear. For the case of power-of-two N, Papadimitriou (1979) argued that the number of complex-number additions achieved by Cooley–Tukey algorithms is optimal under certain assumptions on the graph of the algorithm (his assumptions imply, among other things, that no additive identities in the roots of unity are exploited). (This argument would imply that at least real additions are required, although this is not a tight bound because extra additions are required as part of complex-number multiplications.) Thus far, no published FFT algorithm has achieved fewer than complex-number additions (or their equivalent) for power-of-two N. A third problem is to minimize the total number of real multiplications and additions, sometimes called the "arithmetic complexity" (although in this context it is the exact count and not the asymptotic complexity that is being considered). Again, no tight lower bound has been proven. Since 1968, however, the lowest published count for power-of-two N was long achieved by the split-radix FFT algorithm, which requires real multiplications and additions for N > 1. This was recently reduced to

(Johnson and Frigo, 2007;

Lundy and Van Buskirk, 2007). A slightly larger count (but still better than split radix for N≥256) was shown to be provably optimal for N≤512 under additional restrictions on the possible algorithms (split-radix-like flowgraphs with unit-modulus multiplicative factors), by reduction to a Satisfiability Modulo Theories problem solvable by brute force (Haynal & Haynal, 2011). Most of the attempts to lower or prove the complexity of FFT algorithms have focused on the ordinary complex-data case, because it is the simplest. However, complex-data FFTs are so closely related to algorithms for related problems such as real-data FFTs, discrete cosine transforms, discrete Hartley transforms, and so on, that any improvement in one of these would immediately lead to improvements in the others (Duhamel & Vetterli, 1990).

Accuracy and approximations All of the FFT algorithms discussed above compute the DFT exactly (in exact arithmetic, i.e. neglecting floating-point errors). A few "FFT" algorithms have been proposed, however, that compute the DFT approximately, with an error that can be made arbitrarily small at the expense of increased computations. Such algorithms trade the approximation error for increased speed or other properties. For example, an approximate FFT algorithm by Edelman et al. (1999) achieves lower communication requirements for parallel computing with the help of a fast multipole method. A wavelet-based approximate FFT by Guo and Burrus (1996) takes sparse inputs/outputs (time/frequency localization) into account more efficiently than is possibleWikipedia:Citation needed with an exact FFT. Another algorithm for approximate computation of a subset of the DFT outputs is due to Shentov et al. (1995). The Edelman algorithm works equally well for sparse and non-sparse data, since it is based on the compressibility (rank deficiency) of the Fourier matrix itself rather than the compressibility (sparsity) of the data. Conversely, if the data are sparse—that is, if only K out of N Fourier coefficients are nonzero—then the complexity can be reduced to O(Klog(N)log(N/K)), and this has been demonstrated to lead to practical speedups compared to an ordinary FFT for N/K>32 in a large-N example (N=222) using a probabilistic approximate algorithm (which estimates the largest K coefficients to several decimal places).[2]

Fast Fourier transform

161

Even the "exact" FFT algorithms have errors when finite-precision floating-point arithmetic is used, but these errors are typically quite small; most FFT algorithms, e.g. Cooley–Tukey, have excellent numerical properties as a consequence of the pairwise summation structure of the algorithms. The upper bound on the relative error for the Cooley–Tukey algorithm is O(ε log N), compared to O(εN3/2) for the naïve DFT formula (Gentleman and Sande, 1966), where ε is the machine floating-point relative precision. In fact, the root mean square (rms) errors are much better than these upper bounds, being only O(ε √log N) for Cooley–Tukey and O(ε √N) for the naïve DFT (Schatzman, 1996). These results, however, are very sensitive to the accuracy of the twiddle factors used in the FFT (i.e. the trigonometric function values), and it is not unusual for incautious FFT implementations to have much worse accuracy, e.g. if they use inaccurate trigonometric recurrence formulas. Some FFTs other than Cooley–Tukey, such as the Rader-Brenner algorithm, are intrinsically less stable. In fixed-point arithmetic, the finite-precision errors accumulated by FFT algorithms are worse, with rms errors growing as O(√N) for the Cooley–Tukey algorithm (Welch, 1969). Moreover, even achieving this accuracy requires careful attention to scaling in order to minimize the loss of precision, and fixed-point FFT algorithms involve rescaling at each intermediate stage of decompositions like Cooley–Tukey. To verify the correctness of an FFT implementation, rigorous guarantees can be obtained in O(Nlog(N)) time by a simple procedure checking the linearity, impulse-response, and time-shift properties of the transform on random inputs (Ergün, 1995).

Multidimensional FFTs As defined in the multidimensional DFT article, the multidimensional DFT

transforms an array xn with a d-dimensional vector of indices (over

for each j), where the division n/N, defined as

by a set of d nested summations , is

performed element-wise. Equivalently, it is the composition of a sequence of d sets of one-dimensional DFTs, performed along one dimension at a time (in any order). This compositional viewpoint immediately provides the simplest and most common multidimensional DFT algorithm, known as the row-column algorithm (after the two-dimensional case, below). That is, one simply performs a sequence of d one-dimensional FFTs (by any of the above algorithms): first you transform along the n1 dimension, then along the n2 dimension, and so on (or actually, any ordering will work). This method is easily shown to have the usual O(Nlog(N)) complexity, where is the total number of data points transformed. In particular, there are N/N1 transforms of size N1, etcetera, so the complexity of the sequence of FFTs is:

In two dimensions, the xk can be viewed as an

matrix, and this algorithm corresponds to first performing

the FFT of all the rows (resp. columns), grouping the resulting transformed rows (resp. columns) together as another matrix, and then performing the FFT on each of the columns (resp. rows) of this second matrix, and similarly grouping the results into the final result matrix. In more than two dimensions, it is often advantageous for cache locality to group the dimensions recursively. For example, a three-dimensional FFT might first perform two-dimensional FFTs of each planar "slice" for each fixed n1, and then perform the one-dimensional FFTs along the n1 direction. More generally, an asymptotically optimal cache-oblivious algorithm consists of recursively dividing the dimensions into two groups and

Fast Fourier transform

162

that are transformed recursively (rounding if d is not even) (see Frigo and Johnson, 2005). Still, this remain straightforward variation of the row-column algorithm that ultimately requires only a one-dimensional FFT algorithm as the base case, and still has O(Nlog(N)) complexity. Yet another variation is to perform matrix transpositions in between transforming subsequent dimensions, so that the transforms operate on contiguous data; this is especially important for out-of-core and distributed memory situations where accessing non-contiguous data is extremely time-consuming. There are other multidimensional FFT algorithms that are distinct from the row-column algorithm, although all of them have O(Nlog(N)) complexity. Perhaps the simplest non-row-column FFT is the vector-radix FFT algorithm, which is a generalization of the ordinary Cooley–Tukey algorithm where one divides the transform dimensions by a vector of radices at each step. (This may also have cache benefits.) The simplest case of vector-radix is where all of the radices are equal (e.g. vector-radix-2 divides all of the dimensions by two), but this is not necessary. Vector radix with only a single non-unit radix at a time, i.e. , is essentially a row-column algorithm. Other, more complicated, methods include polynomial transform algorithms due to Nussbaumer (1977), which view the transform in terms of convolutions and polynomial products. See Duhamel and Vetterli (1990) for more information and references.

Other generalizations An O(N5/2log(N)) generalization to spherical harmonics on the sphere S2 with N2 nodes was described by Mohlenkamp (1999), along with an algorithm conjectured (but not proven) to have O(N2 log2(N)) complexity; Mohlenkamp also provides an implementation in the libftsh library [3]. A spherical-harmonic algorithm with O(N2log(N)) complexity is described by Rokhlin and Tygert (2006). The Fast Folding Algorithm is analogous to the FFT, except that it operates on a series of binned waveforms rather than a series of real or complex scalar values. Rotation (which in the FFT is multiplication by a complex phasor) is a circular shift of the component waveform. Various groups have also published "FFT" algorithms for non-equispaced data, as reviewed in Potts et al. (2001). Such algorithms do not strictly compute the DFT (which is only defined for equispaced data), but rather some approximation thereof (a non-uniform discrete Fourier transform, or NDFT, which itself is often computed only approximately). More generally there are various other methods of spectral estimation.

References [1] Charles Van Loan, Computational Frameworks for the Fast Fourier Transform (SIAM, 1992). [2] Haitham Hassanieh, Piotr Indyk, Dina Katabi, and Eric Price, "Simple and Practical Algorithm for Sparse Fourier Transform" (http:/ / www. mit. edu/ ~ecprice/ papers/ sparse-fft-soda. pdf) (PDF), ACM-SIAM Symposium On Discrete Algorithms (SODA), Kyoto, January 2012. See also the sFFT Web Page (http:/ / groups. csail. mit. edu/ netmit/ sFFT/ ). [3] http:/ / www. math. ohiou. edu/ ~mjm/ research/ libftsh. html

• Brenner, N.; Rader, C. (1976). "A New Principle for Fast Fourier Transformation". IEEE Acoustics, Speech & Signal Processing 24 (3): 264–266. doi: 10.1109/TASSP.1976.1162805 (http://dx.doi.org/10.1109/TASSP. 1976.1162805). • Brigham, E. O. (2002). The Fast Fourier Transform. New York: Prentice-Hall • Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein, 2001. Introduction to Algorithms, 2nd. ed. MIT Press and McGraw-Hill. ISBN 0-262-03293-7. Especially chapter 30, "Polynomials and the FFT." • Duhamel, Pierre (1990). "Algorithms meeting the lower bounds on the multiplicative complexity of lengthDFTs and their connection with practical algorithms". IEEE Trans. Acoust. Speech. Sig. Proc. 38 (9): 1504–151. doi: 10.1109/29.60070 (http://dx.doi.org/10.1109/29.60070). • P. Duhamel and M. Vetterli, 1990, Fast Fourier transforms: a tutorial review and a state of the art (http://dx.doi. org/10.1016/0165-1684(90)90158-U), Signal Processing 19: 259–299.

Fast Fourier transform • A. Edelman, P. McCorquodale, and S. Toledo, 1999, The Future Fast Fourier Transform? (http://dx.doi.org/10. 1137/S1064827597316266), SIAM J. Sci. Computing 20: 1094–1114. • D. F. Elliott, & K. R. Rao, 1982, Fast transforms: Algorithms, analyses, applications. New York: Academic Press. • Funda Ergün, 1995, Testing multivariate linear functions: Overcoming the generator bottleneck (http://dx.doi. org/10.1145/225058.225167), Proc. 27th ACM Symposium on the Theory of Computing: 407–416. • M. Frigo and S. G. Johnson, 2005, " The Design and Implementation of FFTW3 (http://fftw.org/ fftw-paper-ieee.pdf)," Proceedings of the IEEE 93: 216–231. • Carl Friedrich Gauss, 1866. " Theoria interpolationis methodo nova tractata (http://lseet.univ-tln.fr/~iaroslav/ Gauss_Theoria_interpolationis_methodo_nova_tractata.php)," Werke band 3, 265–327. Göttingen: Königliche Gesellschaft der Wissenschaften. • W. M. Gentleman and G. Sande, 1966, "Fast Fourier transforms—for fun and profit," Proc. AFIPS 29: 563–578. doi: 10.1145/1464291.1464352 (http://dx.doi.org/10.1145/1464291.1464352) • H. Guo and C. S. Burrus, 1996, Fast approximate Fourier transform via wavelets transform (http://dx.doi.org/ 10.1117/12.255236), Proc. SPIE Intl. Soc. Opt. Eng. 2825: 250–259. • H. Guo, G. A. Sitton, C. S. Burrus, 1994, The Quick Discrete Fourier Transform (http://dx.doi.org/10.1109/ ICASSP.1994.389994), Proc. IEEE Conf. Acoust. Speech and Sig. Processing (ICASSP) 3: 445–448. • Steve Haynal and Heidi Haynal, " Generating and Searching Families of FFT Algorithms (http://jsat.ewi. tudelft.nl/content/volume7/JSAT7_13_Haynal.pdf)", Journal on Satisfiability, Boolean Modeling and Computation vol. 7, pp. 145–187 (2011). • Heideman, M. T.; Johnson, D. H.; Burrus, C. S. (1984). "Gauss and the history of the fast Fourier transform". IEEE ASSP Magazine 1 (4): 14–21. doi: 10.1109/MASSP.1984.1162257 (http://dx.doi.org/10.1109/MASSP. 1984.1162257). • Heideman, Michael T.; Burrus, C. Sidney (1986). "On the number of multiplications necessary to compute a lengthDFT". IEEE Trans. Acoust. Speech. Sig. Proc. 34 (1): 91–95. doi: 10.1109/TASSP.1986.1164785 • • • • •

• • •

(http://dx.doi.org/10.1109/TASSP.1986.1164785). S. G. Johnson and M. Frigo, 2007. " A modified split-radix FFT with fewer arithmetic operations (http://www. fftw.org/newsplit.pdf)," IEEE Trans. Signal Processing 55 (1): 111–119. T. Lundy and J. Van Buskirk, 2007. "A new matrix approach to real FFTs and convolutions of length 2k," Computing 80 (1): 23-45. Kent, Ray D. and Read, Charles (2002). Acoustic Analysis of Speech. ISBN 0-7693-0112-6. Cites Strang, G. (1994)/May–June). Wavelets. American Scientist, 82, 250-255. Morgenstern, Jacques (1973). "Note on a lower bound of the linear complexity of the fast Fourier transform". J. ACM 20 (2): 305–306. doi: 10.1145/321752.321761 (http://dx.doi.org/10.1145/321752.321761). Mohlenkamp, M. J. (1999). "A fast transform for spherical harmonics" (http://www.math.ohiou.edu/~mjm/ research/MOHLEN1999P.pdf). J. Fourier Anal. Appl. 5 (2-3): 159–184. doi: 10.1007/BF01261607 (http://dx. doi.org/10.1007/BF01261607). Nussbaumer, H. J. (1977). "Digital filtering using polynomial transforms". Electronics Lett. 13 (13): 386–387. doi: 10.1049/el:19770280 (http://dx.doi.org/10.1049/el:19770280). V. Pan, 1986, The trade-off between the additive complexity and the asyncronicity of linear and bilinear algorithms (http://dx.doi.org/10.1016/0020-0190(86)90035-9), Information Proc. Lett. 22: 11-14. Christos H. Papadimitriou, 1979, Optimality of the fast Fourier transform (http://dx.doi.org/10.1145/322108. 322118), J. ACM 26: 95-102.

• D. Potts, G. Steidl, and M. Tasche, 2001. " Fast Fourier transforms for nonequispaced data: A tutorial (http:// www.tu-chemnitz.de/~potts/paper/ndft.pdf)", in: J.J. Benedetto and P. Ferreira (Eds.), Modern Sampling Theory: Mathematics and Applications (Birkhauser).

163

Fast Fourier transform • Press, WH; Teukolsky, SA; Vetterling, WT; Flannery, BP (2007), "Chapter 12. Fast Fourier Transform" (http:// apps.nrbook.com/empanel/index.html#pg=600), Numerical Recipes: The Art of Scientific Computing (3rd ed.), New York: Cambridge University Press, ISBN 978-0-521-88068-8 • Rokhlin, Vladimir; Tygert, Mark (2006). "Fast algorithms for spherical harmonic expansions". SIAM J. Sci. Computing 27 (6): 1903–1928. doi: 10.1137/050623073 (http://dx.doi.org/10.1137/050623073). • James C. Schatzman, 1996, Accuracy of the discrete Fourier transform and the fast Fourier transform (http:// portal.acm.org/citation.cfm?id=240432), SIAM J. Sci. Comput. 17: 1150–1166. • Shentov, O. V.; Mitra, S. K.; Heute, U.; Hossen, A. N. (1995). "Subband DFT. I. Definition, interpretations and extensions". Signal Processing 41 (3): 261–277. doi: 10.1016/0165-1684(94)00103-7 (http://dx.doi.org/10. 1016/0165-1684(94)00103-7). • Sorensen, H. V.; Jones, D. L.; Heideman, M. T.; Burrus, C. S. (1987). "Real-valued fast Fourier transform algorithms". IEEE Trans. Acoust. Speech Sig. Processing 35 (35): 849–863. doi: 10.1109/TASSP.1987.1165220 (http://dx.doi.org/10.1109/TASSP.1987.1165220). See also Sorensen, H.; Jones, D.; Heideman, M.; Burrus, C. (1987). "Corrections to "Real-valued fast Fourier transform algorithms"". IEEE Transactions on Acoustics, Speech, and Signal Processing 35 (9): 1353–1353. doi: 10.1109/TASSP.1987.1165284 (http://dx.doi.org/10. 1109/TASSP.1987.1165284). • Welch, Peter D. (1969). "A fixed-point fast Fourier transform error analysis". IEEE Trans. Audio Electroacoustics 17 (2): 151–157. doi: 10.1109/TAU.1969.1162035 (http://dx.doi.org/10.1109/TAU.1969.1162035). • Winograd, S. (1978). "On computing the discrete Fourier transform". Math. Computation 32 (141): 175–199. doi: 10.1090/S0025-5718-1978-0468306-4 (http://dx.doi.org/10.1090/S0025-5718-1978-0468306-4). JSTOR 2006266 (http://www.jstor.org/stable/2006266).

External links • Fast Fourier Algorithm (http://www.cs.pitt.edu/~kirk/cs1501/animations/FFT.html) • Fast Fourier Transforms (http://cnx.org/content/col10550/), Connexions online book edited by C. Sidney Burrus, with chapters by C. Sidney Burrus, Ivan Selesnick, Markus Pueschel, Matteo Frigo, and Steven G. Johnson (2008). • Links to FFT code and information online. (http://www.fftw.org/links.html) • National Taiwan University – FFT (http://www.cmlab.csie.ntu.edu.tw/cml/dsp/training/coding/transform/ fft.html) • FFT programming in C++ — Cooley–Tukey algorithm. (http://www.librow.com/articles/article-10) • Online documentation, links, book, and code. (http://www.jjj.de/fxt/) • Using FFT to construct aggregate probability distributions (http://www.vosesoftware.com/ModelRiskHelp/ index.htm#Aggregate_distributions/Aggregate_modeling_-_Fast_Fourier_Transform_FFT_method.htm) • Sri Welaratna, " Thirty years of FFT analyzers (http://www.dataphysics.com/ 30_Years_of_FFT_Analyzers_by_Sri_Welaratna.pdf)", Sound and Vibration (January 1997, 30th anniversary issue). A historical review of hardware FFT devices. • FFT Basics and Case Study Using Multi-Instrument (http://www.virtins.com/doc/D1002/ FFT_Basics_and_Case_Study_using_Multi-Instrument_D1002.pdf) • FFT Textbook notes, PPTs, Videos (http://numericalmethods.eng.usf.edu/topics/fft.html) at Holistic Numerical Methods Institute. • ALGLIB FFT Code (http://www.alglib.net/fasttransforms/fft.php) GPL Licensed multilanguage (VBA, C++, Pascal, etc.) numerical analysis and data processing library. • MIT's sFFT (http://groups.csail.mit.edu/netmit/sFFT/) MIT Sparse FFT algorithm and implementation. • VB6 FFT (http://www.borgdesign.ro/fft.zip) VB6 optimized library implementation with source code.

164

Cooley-Tukey FFT algorithm

165

Cooley-Tukey FFT algorithm The Cooley–Tukey algorithm, named after J.W. Cooley and John Tukey, is the most common fast Fourier transform (FFT) algorithm. It re-expresses the discrete Fourier transform (DFT) of an arbitrary composite size N = N1N2 in terms of smaller DFTs of sizes N1 and N2, recursively, in order to reduce the computation time to O(N log N) for highly-composite N (smooth numbers). Because of the algorithm's importance, specific variants and implementation styles have become known by their own names, as described below. Because the Cooley-Tukey algorithm breaks the DFT into smaller DFTs, it can be combined arbitrarily with any other algorithm for the DFT. For example, Rader's or Bluestein's algorithm can be used to handle large prime factors that cannot be decomposed by Cooley–Tukey, or the prime-factor algorithm can be exploited for greater efficiency in separating out relatively prime factors. See also the fast Fourier transform for information on other FFT algorithms, specializations for real and/or symmetric data, and accuracy in the face of finite floating-point precision.

History This algorithm, including its recursive application, was invented around 1805 by Carl Friedrich Gauss, who used it to interpolate the trajectories of the asteroids Pallas and Juno, but his work was not widely recognized (being published only posthumously and in neo-Latin).[1][2] Gauss did not analyze the asymptotic computational time, however. Various limited forms were also rediscovered several times throughout the 19th and early 20th centuries. FFTs became popular after James Cooley of IBM and John Tukey of Princeton published a paper in 1965 reinventing the algorithm and describing how to perform it conveniently on a computer. Tukey reportedly came up with the idea during a meeting of a US presidential advisory committee discussing ways to detect nuclear-weapon tests in the Soviet Union.[3] Another participant at that meeting, Richard Garwin of IBM, recognized the potential of the method and put Tukey in touch with Cooley, who implemented it for a different (and less-classified) problem: analyzing 3d crystallographic data (see also: multidimensional FFTs). Cooley and Tukey subsequently published their joint paper, and wide adoption quickly followed. The fact that Gauss had described the same algorithm (albeit without analyzing its asymptotic cost) was not realized until several years after Cooley and Tukey's 1965 paper. Their paper cited as inspiration only work by I. J. Good on what is now called the prime-factor FFT algorithm (PFA); although Good's algorithm was initially mistakenly thought to be equivalent to the Cooley–Tukey algorithm, it was quickly realized that PFA is a quite different algorithm (only working for sizes that have relatively prime factors and relying on the Chinese Remainder Theorem, unlike the support for any composite size in Cooley–Tukey).[4]

The radix-2 DIT case A radix-2 decimation-in-time (DIT) FFT is the simplest and most common form of the Cooley–Tukey algorithm, although highly optimized Cooley–Tukey implementations typically use other forms of the algorithm as described below. Radix-2 DIT divides a DFT of size N into two interleaved DFTs (hence the name "radix-2") of size N/2 with each recursive stage. The discrete Fourier transform (DFT) is defined by the formula:

where

is an integer ranging from

to

.

Radix-2 DIT first computes the DFTs of the even-indexed inputs odd-indexed inputs

and of the

, and then combines those two results to produce the DFT of

Cooley-Tukey FFT algorithm

166

the whole sequence. This idea can then be performed recursively to reduce the overall runtime to O(N log N). This simplified form assumes that N is a power of two; since the number of sample points N can usually be chosen freely by the application, this is often not an important restriction. The Radix-2 DIT algorithm rearranges the DFT of the function indices and a sum over the odd-numbered indices

One can factor a common multiplier

into two parts: a sum over the even-numbered :

out of the second sum, as shown in the equation below. It is then clear

that the two sums are the DFT of the even-indexed part

and the DFT of odd-indexed part

function

by

by

. Denote the DFT of the Even-indexed inputs

of the

and the DFT of the Odd-indexed inputs

and we obtain:

Thanks to the periodicity of the DFT, we know that and

.

Therefore, we can rewrite the above equation as

We also know that the twiddle factor

obeys the following relation:

This allows us to cut the number of "twiddle factor" calculations in half also. For

, we have

This result, expressing the DFT of length N recursively in terms of two DFTs of size N/2, is the core of the radix-2 DIT fast Fourier transform. The algorithm gains its speed by re-using the results of intermediate computations to compute multiple DFT outputs. Note that final outputs are obtained by a +/− combination of and , which is simply a size-2 DFT (sometimes called a butterfly in this context); when this is generalized to larger radices below, the size-2 DFT is replaced by a larger DFT (which itself can be evaluated with an FFT).

Cooley-Tukey FFT algorithm

167

This process is an example of the general technique of divide and conquer algorithms; in many traditional implementations, however, the explicit recursion is avoided, and instead one traverses the computational tree in breadth-first fashion. The above re-expression of a size-N DFT as two size-N/2 DFTs is sometimes called the Danielson–Lanczos lemma, since the identity was noted by those two authors in 1942[5] (influenced by Runge's 1903 work). They applied their lemma in a "backwards" recursive fashion, repeatedly doubling the DFT size until the transform spectrum converged (although they apparently didn't Data flow diagram for N=8: a decimation-in-time radix-2 FFT breaks a length-N realize the linearithmic [i.e., order N log N] DFT into two length-N/2 DFTs followed by a combining stage consisting of many asymptotic complexity they had achieved). size-2 DFTs called "butterfly" operations (so-called because of the shape of the The Danielson–Lanczos work predated data-flow diagrams). widespread availability of computers and required hand calculation (possibly with mechanical aids such as adding machines); they reported a computation time of 140 minutes for a size-64 DFT operating on real inputs to 3–5 significant digits. Cooley and Tukey's 1965 paper reported a running time of 0.02 minutes for a size-2048 complex DFT on an IBM 7094 (probably in 36-bit single precision, ~8 digits). Rescaling the time by the number of operations, this corresponds roughly to a speedup factor of around 800,000. (To put the time for the hand calculation in perspective, 140 minutes for size 64 corresponds to an average of at most 16 seconds per floating-point operation, around 20% of which are multiplications.)

Pseudocode In pseudocode, the above procedure could be written: X0,...,N−1 ← ditfft2(x, N, s):

DFT of (x0, xs, x2s, ..., x(N-1)s):

if N = 1 then X0 ← x0

trivial size-1 DFT base case

X0,...,N/2−1 ← ditfft2(x, N/2, 2s)

DFT of (x0, x2s, x4s, ...)

XN/2,...,N−1 ← ditfft2(x+s, N/2, 2s)

DFT of (xs, xs+2s, xs+4s, ...)

for k = 0 to N/2−1

combine DFTs of two halves into full DFT:

else

t ← Xk Xk ← t + exp(−2πi k/N) Xk+N/2 Xk+N/2 ← t − exp(−2πi k/N) Xk+N/2 endfor endif

Here, ditfft2(x,N,1), computes X=DFT(x) out-of-place by a radix-2 DIT FFT, where N is an integer power of 2 and s=1 is the stride of the input x array. x+s denotes the array starting with xs. (The results are in the correct order in X and no further bit-reversal permutation is required; the often-mentioned necessity of a separate bit-reversal stage only arises for certain in-place algorithms, as described below.)

Cooley-Tukey FFT algorithm High-performance FFT implementations make many modifications to the implementation of such an algorithm compared to this simple pseudocode. For example, one can use a larger base case than N=1 to amortize the overhead of recursion, the twiddle factors can be precomputed, and larger radices are often used for cache reasons; these and other optimizations together can improve the performance by an order of magnitude or more.[] (In many textbook implementations the depth-first recursion is eliminated entirely in favor of a nonrecursive breadth-first approach, although depth-first recursion has been argued to have better memory locality.) Several of these ideas are described in further detail below.

General factorizations More generally, Cooley–Tukey algorithms recursively re-express a DFT of a composite size N = N1N2 as:[6] 1. Perform N1 DFTs of size N2. 2. Multiply by complex roots of unity called twiddle factors. 3. Perform N2 DFTs of size N1. Typically, either N1 or N2 is a small factor (not necessarily prime), called the radix (which can differ between stages of the recursion). If N1 is the radix, it is called a decimation in time The basic step of the Cooley–Tukey FFT for general factorizations can be viewed as (DIT) algorithm, whereas if N2 is the re-interpreting a 1d DFT as something like a 2d DFT. The 1d input array of length N = radix, it is decimation in frequency N1N2 is reinterpreted as a 2d N1×N2 matrix stored in column-major order. One performs (DIF, also called the Sande-Tukey smaller 1d DFTs along the N2 direction (the non-contiguous direction), then multiplies by phase factors (twiddle factors), and finally performs 1d DFTs along the N1 direction. The algorithm). The version presented transposition step can be performed in the middle, as shown here, or at the beginning or above was a radix-2 DIT algorithm; in end. This is done recursively for the smaller transforms. the final expression, the phase multiplying the odd transform is the twiddle factor, and the +/- combination (butterfly) of the even and odd transforms is a size-2 DFT. (The radix's small DFT is sometimes known as a butterfly, so-called because of the shape of the dataflow diagram for the radix-2 case.) There are many other variations on the Cooley–Tukey algorithm. Mixed-radix implementations handle composite sizes with a variety of (typically small) factors in addition to two, usually (but not always) employing the O(N2) algorithm for the prime base cases of the recursion (it is also possible to employ an N log N algorithm for the prime base cases, such as Rader's or Bluestein's algorithm). Split radix merges radices 2 and 4, exploiting the fact that the first transform of radix 2 requires no twiddle factor, in order to achieve what was long the lowest known arithmetic operation count for power-of-two sizes, although recent variations achieve an even lower count.[7][8] (On present-day computers, performance is determined more by cache and CPU pipeline considerations than by strict operation counts; well-optimized FFT implementations often employ larger radices and/or hard-coded base-case transforms of significant size.) Another way of looking at the Cooley–Tukey algorithm is that it re-expresses a size N one-dimensional DFT as an N1 by N2 two-dimensional DFT (plus twiddles), where the output matrix is transposed. The net result of all of these transpositions, for a radix-2 algorithm, corresponds to a bit reversal of the input (DIF) or output (DIT) indices. If, instead of using a small radix, one employs a radix of roughly √N and explicit input/output matrix transpositions, it is called a four-step algorithm (or six-step, depending on the number of transpositions), initially proposed to improve memory locality,[9][10] e.g. for cache optimization or out-of-core operation, and was

168

Cooley-Tukey FFT algorithm

169

later shown to be an optimal cache-oblivious algorithm.[11] The general Cooley–Tukey factorization rewrites the indices k and n as

and

,

respectively, where the indices ka and na run from 0..Na-1 (for a of 1 or 2). That is, it re-indexes the input (n) and output (k) as N1 by N2 two-dimensional arrays in column-major and row-major order, respectively; the difference between these indexings is a transposition, as mentioned above. When this re-indexing is substituted into the DFT formula for nk, the cross term vanishes (its exponential is unity), and the remaining terms give

where each inner sum is a DFT of size N2, each outer sum is a DFT of size N1, and the [...] bracketed term is the twiddle factor. An arbitrary radix r (as well as mixed radices) can be employed, as was shown by both Cooley and Tukey as well as Gauss (who gave examples of radix-3 and radix-6 steps). Cooley and Tukey originally assumed that the radix butterfly required O(r2) work and hence reckoned the complexity for a radix r to be O(r2 N/r logrN) = O(N log2(N) r/log2r); from calculation of values of r/log2r for integer values of r from 2 to 12 the optimal radix is found to be 3 (the closest integer to e, which minimizes r/log2r).[12] This analysis was erroneous, however: the radix-butterfly is also a DFT and can be performed via an FFT algorithm in O(r log r) operations, hence the radix r actually cancels in the complexity O(r log(r) N/r logrN), and the optimal r is determined by more complicated considerations. In practice, quite large r (32 or 64) are important in order to effectively exploit e.g. the large number of processor registers on modern processors, and even an unbounded radix r=√N also achieves O(N log N) complexity and has theoretical and practical advantages for large N as mentioned above.

Data reordering, bit reversal, and in-place algorithms Although the abstract Cooley–Tukey factorization of the DFT, above, applies in some form to all implementations of the algorithm, much greater diversity exists in the techniques for ordering and accessing the data at each stage of the FFT. Of special interest is the problem of devising an in-place algorithm that overwrites its input with its output data using only O(1) auxiliary storage. The most well-known reordering technique involves explicit bit reversal for in-place radix-2 algorithms. Bit reversal is the permutation where the data at an index n, written in binary with digits b4b3b2b1b0 (e.g. 5 digits for N=32 inputs), is transferred to the index with reversed digits b0b1b2b3b4 . Consider the last stage of a radix-2 DIT algorithm like the one presented above, where the output is written in-place over the input: when and are combined with a size-2 DFT, those two values are overwritten by the outputs. However, the two output values should go in the first and second halves of the output array, corresponding to the most significant bit b4 (for N=32); whereas the two inputs and are interleaved in the even and odd elements, corresponding to the least significant bit b0. Thus, in order to get the output in the correct place, these two bits must be swapped. If you include all of the recursive stages of a radix-2 DIT algorithm, all the bits must be swapped and thus one must pre-process the input (or post-process the output) with a bit reversal to get in-order output. (If each size-N/2 subtransform is to operate on contiguous data, the DIT input is pre-processed by bit-reversal.) Correspondingly, if you perform all of the steps in reverse order, you obtain a radix-2 DIF algorithm with bit reversal in post-processing (or pre-processing, respectively). Alternatively, some applications (such as convolution) work equally well on bit-reversed data, so one can perform forward transforms, processing, and then inverse transforms all without bit reversal to produce final results in the natural order. Many FFT users, however, prefer natural-order outputs, and a separate, explicit bit-reversal stage can have a non-negligible impact on the computation time, even though bit reversal can be done in O(N) time and has been the

Cooley-Tukey FFT algorithm subject of much research. Also, while the permutation is a bit reversal in the radix-2 case, it is more generally an arbitrary (mixed-base) digit reversal for the mixed-radix case, and the permutation algorithms become more complicated to implement. Moreover, it is desirable on many hardware architectures to re-order intermediate stages of the FFT algorithm so that they operate on consecutive (or at least more localized) data elements. To these ends, a number of alternative implementation schemes have been devised for the Cooley–Tukey algorithm that do not require separate bit reversal and/or involve additional permutations at intermediate stages. The problem is greatly simplified if it is out-of-place: the output array is distinct from the input array or, equivalently, an equal-size auxiliary array is available. The Stockham auto-sort algorithm[13] performs every stage of the FFT out-of-place, typically writing back and forth between two arrays, transposing one "digit" of the indices with each stage, and has been especially popular on SIMD architectures.[] Even greater potential SIMD advantages (more consecutive accesses) have been proposed for the Pease algorithm, which also reorders out-of-place with each stage, but this method requires separate bit/digit reversal and O(N log N) storage. One can also directly apply the Cooley–Tukey factorization definition with explicit (depth-first) recursion and small radices, which produces natural-order out-of-place output with no separate permutation step (as in the pseudocode above) and can be argued to have cache-oblivious locality benefits on systems with hierarchical memory.[14] A typical strategy for in-place algorithms without auxiliary storage and without separate digit-reversal passes involves small matrix transpositions (which swap individual pairs of digits) at intermediate stages, which can be combined with the radix butterflies to reduce the number of passes over the data.

References [1] Gauss, Carl Friedrich, " Theoria interpolationis methodo nova tractata (http:/ / lseet. univ-tln. fr/ ~iaroslav/ Gauss_Theoria_interpolationis_methodo_nova_tractata. php)", Werke, Band 3, 265–327 (Königliche Gesellschaft der Wissenschaften, Göttingen, 1866) [2] Heideman, M. T., D. H. Johnson, and C. S. Burrus, " Gauss and the history of the fast Fourier transform (http:/ / ieeexplore. ieee. org/ xpls/ abs_all. jsp?arnumber=1162257)," IEEE ASSP Magazine, 1, (4), 14–21 (1984) [3] Rockmore, Daniel N. , Comput. Sci. Eng. 2 (1), 60 (2000). The FFT — an algorithm the whole family can use (http:/ / www. cs. dartmouth. edu/ ~rockmore/ cse-fft. pdf) Special issue on "top ten algorithms of the century " (http:/ / amath. colorado. edu/ resources/ archive/ topten. pdf) [4] James W. Cooley, Peter A. W. Lewis, and Peter W. Welch, "Historical notes on the fast Fourier transform," Proc. IEEE, vol. 55 (no. 10), p. 1675–1677 (1967). [5] Danielson, G. C., and C. Lanczos, "Some improvements in practical Fourier analysis and their application to X-ray scattering from liquids," J. Franklin Inst. 233, 365–380 and 435–452 (1942). [6] Duhamel, P., and M. Vetterli, "Fast Fourier transforms: a tutorial review and a state of the art," Signal Processing 19, 259–299 (1990) [7] Lundy, T., and J. Van Buskirk, "A new matrix approach to real FFTs and convolutions of length 2k," Computing 80, 23-45 (2007). [8] Johnson, S. G., and M. Frigo, " A modified split-radix FFT with fewer arithmetic operations (http:/ / www. fftw. org/ newsplit. pdf)," IEEE Trans. Signal Processing 55 (1), 111–119 (2007). [9] Gentleman W. M., and G. Sande, "Fast Fourier transforms—for fun and profit," Proc. AFIPS 29, 563–578 (1966). [10] Bailey, David H., "FFTs in external or hierarchical memory," J. Supercomputing 4 (1), 23–35 (1990) [11] M. Frigo, C.E. Leiserson, H. Prokop, and S. Ramachandran. Cache-oblivious algorithms. In Proceedings of the 40th IEEE Symposium on Foundations of Computer Science (FOCS 99), p.285-297. 1999. Extended abstract at IEEE (http:/ / ieeexplore. ieee. org/ iel5/ 6604/ 17631/ 00814600. pdf?arnumber=814600), at Citeseer (http:/ / citeseer. ist. psu. edu/ 307799. html). [12] Cooley, J. W., P. Lewis and P. Welch, "The Fast Fourier Transform and its Applications", IEEE Trans on Education 12, 1, 28-34 (1969) [13] Originally attributed to Stockham in W. T. Cochran et al., What is the fast Fourier transform? (http:/ / dx. doi. org/ 10. 1109/ PROC. 1967. 5957), Proc. IEEE vol. 55, 1664–1674 (1967). [14] A free (GPL) C library for computing discrete Fourier transforms in one or more dimensions, of arbitrary size, using the Cooley–Tukey algorithm

170

Cooley-Tukey FFT algorithm

171

External links • a simple, pedagogical radix-2 Cooley–Tukey FFT algorithm in C++. (http://www.librow.com/articles/ article-10) • KISSFFT (http://sourceforge.net/projects/kissfft/): a simple mixed-radix Cooley–Tukey implementation in C (open source)

Butterfly diagram This article is about butterfly diagrams in FFT algorithms; for the sunspot diagrams of the same name, see Solar cycle. In the context of fast Fourier transform algorithms, a butterfly is a portion of the computation that combines the results of smaller discrete Fourier transforms (DFTs) into a larger DFT, or vice versa (breaking a larger DFT up into subtransforms). The name "butterfly" comes from the shape of the data-flow diagram in the radix-2 case, as described below.[1] The same structure can also be found in the Viterbi algorithm, used for finding the most likely sequence of hidden states. Most commonly, the term "butterfly" appears in the context of the Cooley–Tukey FFT algorithm, which recursively breaks down a DFT of composite size n = rm into r smaller transforms of size m where r is the "radix" of the transform. These smaller DFTs are then combined via size-r butterflies, which themselves are DFTs of size r (performed m times on corresponding outputs of the sub-transforms) pre-multiplied by roots of unity (known as twiddle factors). (This is the "decimation in time" case; one can also perform the steps in reverse, known as "decimation in frequency", where the butterflies come first and are post-multiplied by twiddle factors. See also the Cooley–Tukey FFT article.)

Data flow diagram connecting the inputs x (left) to the outputs y that depend on them (right) for a "butterfly" step of a radix-2 Cooley–Tukey FFT. This diagram resembles a butterfly (as in the Morpho butterfly shown for comparison), hence the name.

Radix-2 butterfly diagram In the case of the radix-2 Cooley–Tukey algorithm, the butterfly is simply a DFT of size-2 that takes two inputs (x0, x1) (corresponding outputs of the two sub-transforms) and gives two outputs (y0, y1) by the formula (not including twiddle factors):

If one draws the data-flow diagram for this pair of operations, the (x0, x1) to (y0, y1) lines cross and resemble the wings of a butterfly, hence the name (see also the illustration at right).

Butterfly diagram

172

More specifically, a decimation-in-time FFT algorithm on n = 2 p inputs with respect to a primitive n-th root of unity relies on O(n log n) butterflies of the form:

where k is an integer depending on the part of the transform being computed. Whereas the corresponding inverse transform can mathematically be performed by replacing ω with ω−1 (and possibly multiplying by an overall scale factor, depending on the normalization convention), one may also directly invert the butterflies: A decimation-in-time radix-2 FFT breaks a length-N DFT into two length-N/2 DFTs followed by a combining stage consisting of many butterfly operations.

corresponding to a decimation-in-frequency FFT algorithm.

Other uses The butterfly can also be used to improve the randomness of large arrays of partially random numbers, by bringing every 32 or 64 bit word into causal contact with every other word through a desired hashing algorithm, so that a change in any one bit has the possibility of changing all the bits in the large array.

References [1] Alan V. Oppenheim, Ronald W. Schafer, and John R. Buck, Discrete-Time Signal Processing, 2nd edition (Upper Saddle River, NJ: Prentice Hall, 1989)

External links • explanation of the FFT and butterfly diagrams (http://www.relisoft.com/Science/Physics/fft.html). • butterfly diagrams of various FFT implementations (Radix-2, Radix-4, Split-Radix) (http://www.cmlab.csie. ntu.edu.tw/cml/dsp/training/coding/transform/fft.html).

Codec

Codec This article is about encoding and decoding a digital data stream. For other uses, see Codec (disambiguation). Further information: List of codecs and Video codecs A codec is a device or computer program capable of encoding or decoding a digital data stream or signal. The word codec is a portmanteau of "coder-decoder" or, less commonly, "compressor-decompressor". A codec (the program) should not be confused with a coding or compression format or standard – a format is a document (the standard), a way of storing data, while a codec is a program (an implementation) which can read or write such files. In practice, however, "codec" is sometimes used loosely to refer to formats. A codec encodes a data stream or signal for transmission, storage or encryption, or decodes it for playback or editing. Codecs are used in videoconferencing, streaming media and video editing applications. A video camera's analog-to-digital converter (ADC) converts its analog signals into digital signals, which are then passed through a video compressor for digital transmission or storage. A receiving device then runs the signal through a video decompressor, then a digital-to-analog converter (DAC) for analog display. The term codec is also used as a generic name for a videoconferencing unit.

Related concepts An endec (encoder/decoder) is a similar yet different concept mainly used for hardware. In the mid 20th century, a "codec" was hardware that coded analog signals into pulse-code modulation (PCM) and decoded them back. Late in the century the name came to be applied to a class of software for converting among digital signal formats, and including compander functions. A modem is a contraction of modulator/demodulator (although they were referred to as "datasets" by telcos) and converts digital data from computers to analog for phone line transmission. On the receiving end the analog is converted back to digital. Codecs do the opposite (convert audio analog to digital and then computer digital sound back to audio). An audio codec converts analog audio signals into digital signals for transmission or storage. A receiving device then converts the digital signals back to analog using an audio decompressor, for playback. An example of this is the codecs used in the sound cards of personal computers. A video codec accomplishes the same task for video signals.

Compression quality • Lossy codecs: Many of the more popular codecs in the software world are lossy, meaning that they reduce quality by some amount in order to achieve compression. Often, this type of compression is virtually indistinguishable from the original uncompressed sound or images, depending on the codec and the settings used. Smaller data sets ease the strain on relatively expensive storage sub-systems such as non-volatile memory and hard disk, as well as write-once-read-many formats such as CD-ROM, DVD and Blu-ray Disc. Lower data rates also reduce cost and improve performance when the data is transmitted. • Lossless codecs: There are also many lossless codecs which are typically used for archiving data in a compressed form while retaining all of the information present in the original stream. If preserving the original quality of the stream is more important than eliminating the correspondingly larger data sizes, lossless codecs are preferred. This is especially true if the data is to undergo further processing (for example editing) in which case the repeated application of processing (encoding and decoding) on lossy codecs will degrade the quality of the resulting data such that it is no longer identifiable (visually, audibly or both). Using more than one codec or encoding scheme successively can also degrade quality significantly. The decreasing cost of storage capacity and network bandwidth has a tendency to reduce the need for lossy codecs for some media.

173

Codec

Media codecs Two principal techniques are used in codecs, pulse-code modulation and delta modulation. Codecs are often designed to emphasize certain aspects of the media to be encoded. For example, a digital video (using a DV codec) of a sports event needs to encode motion well but not necessarily exact colors, while a video of an art exhibit needs to encode color and surface texture well. Audio codecs for cell phones need to have very low latency between source encoding and playback. In contrast, audio codecs for recording or broadcast can use high-latency audio compression techniques to achieve higher fidelity at a lower bit-rate. There are thousands of audio and video codecs, ranging in cost from free to hundreds of dollars or more. This variety of codecs can create compatibility and obsolescence issues. The impact is lessened for older formats, for which free or nearly-free codecs have existed for a long time. The older formats are often ill-suited to modern applications, however, such as playback in small portable devices. For example, raw uncompressed PCM audio (44.1 kHz, 16 bit stereo, as represented on an audio CD or in a .wav or .aiff file) has long been a standard across multiple platforms, but its transmission over networks is slow and expensive compared with more modern compressed formats, such as MP3. Many multimedia data streams contain both audio and video, and often some metadata that permit synchronization of audio and video. Each of these three streams may be handled by different programs, processes, or hardware; but for the multimedia data streams to be useful in stored or transmitted form, they must be encapsulated together in a container format. Lower bitrate codecs allow more users, but they also have more distortion. Beyond the initial increase in distortion, lower bit rate codecs also achieve their lower bit rates by using more complex algorithms that make certain assumptions, such as those about the media and the packet loss rate. Other codecs may not make those same assumptions. When a user with a low bitrate codec talks to a user with another codec, additional distortion is introduced by each transcoding. AVI is sometimes erroneously described as a codec, but AVI is actually a container format, while a codec is a software or hardware tool that encodes or decodes audio or video into or from some audio or video format. Audio and video encoded with many codecs might be put into an AVI container, although AVI is not an ISO standard. There are also other well-known container formats, such as Ogg, ASF, QuickTime, RealMedia, Matroska, and DivX Media Format. Some container formats which are ISO standards are MPEG transport stream, MPEG program stream, MP4 and ISO base media file format.

References

174

FFTW

175

FFTW FFTW Developer(s)

Matteo Frigo and Steven G. Johnson

Initial release 24 March 1997 Stable release 3.3.4 / 16 March 2014 Written in

C, OCaml

Type

Numerical software

License

GPL, commercial

Website

www.fftw.org

[1]

The Fastest Fourier Transform in the West (FFTW) is a software library for computing discrete Fourier transforms (DFTs) developed by Matteo Frigo and Steven G. Johnson at the Massachusetts Institute of Technology. FFTW is known as the fastest free software implementation of the Fast Fourier transform (FFT) algorithm (upheld by regular benchmarks[2]). It can compute transforms of real and complex-valued arrays of arbitrary size and dimension in O(n log n) time. It does this by supporting a variety of algorithms and choosing the one (a particular decomposition of the transform into smaller transforms) it estimates or measures to be preferable in the particular circumstances. It works best on arrays of sizes with small prime factors, with powers of two being optimal and large primes being worst case (but still O(n log n)). To decompose transforms of composite sizes into smaller transforms, it chooses among several variants of the Cooley–Tukey FFT algorithm (corresponding to different factorizations and/or different memory-access patterns), while for prime sizes it uses either Rader's or Bluestein's FFT algorithm. Once the transform has been broken up into subtransforms of sufficiently small sizes, FFTW uses hard-coded unrolled FFTs for these small sizes that were produced (at compile time, not at run time) by code generation; these routines use a variety of algorithms including Cooley–Tukey variants, Rader's algorithm, and prime-factor FFT algorithms. For a sufficiently large number of repeated transforms it is advantageous to measure the performance of some or all of the supported algorithms on the given array size and platform. These measurements, which the authors refer to as "wisdom", can be stored in a file or string for later use. FFTW has a "guru interface" that intends "to expose as much as possible of the flexibility in the underlying FFTW architecture". This allows, among other things, multi-dimensional transforms and multiple transforms in a single call (e.g., where the data is interleaved in memory). FFTW has limited support for out-of-order transforms (using the MPI version). The data reordering incurs an overhead, which for in-place transforms of arbitrary size and dimension is non-trivial to avoid. It is undocumented for which transforms this overhead is significant. FFTW is licensed under the GNU General Public License. It is also licensed commercially by MIT and is used in the commercial MATLAB[3] matrix package for calculating FFTs. FFTW is written in the C language, but Fortran and Ada interfaces exist, as well as interfaces for a few other languages. While the library itself is C, the code is actually generated from a program called 'genfft', which is written in OCaml.[4] In 1999, FFTW won the J. H. Wilkinson Prize for Numerical Software.

FFTW

References [1] http:/ / www. fftw. org/ [2] Homepage, second paragraph (http:/ / www. fftw. org/ ), and benchmarks page (http:/ / www. fftw. org/ benchfft/ ) [3] Faster Finite Fourier Transforms: MATLAB 6 incorporates FFTW (http:/ / www. mathworks. com/ company/ newsletters/ articles/ faster-finite-fourier-transforms-matlab. html) [4] "FFTW FAQ" (http:/ / www. fftw. org/ faq/ section2. html#languages)

External links • Official website (http://www.fftw.org/)

176

177

Wavelets Wavelet A wavelet is a wave-like oscillation with an amplitude that begins at zero, increases, and then decreases back to zero. It can typically be visualized as a "brief oscillation" like one might see recorded by a seismograph or heart monitor. Generally, wavelets are purposefully crafted to have specific properties that make them useful for signal processing. Wavelets can be combined, using a "reverse, shift, multiply and integrate" technique called convolution, with portions of a known signal to extract information from the unknown signal. For example, a wavelet could be created to have a frequency of Middle C and a short duration of roughly a 32nd note. If this wavelet was to be convolved with a signal created from the recording of a song, then the resulting signal would be useful for determining when the Middle C note was being played in the song. Mathematically, the wavelet will correlate with the signal if the unknown signal contains information of similar frequency. This concept of correlation is at the core of many practical applications of wavelet theory. As a mathematical tool, wavelets can be used to extract information from many different kinds of data, including – but certainly not limited to – audio signals and images. Sets of wavelets are generally needed to analyze data fully. A set of "complementary" wavelets will decompose data without gaps or overlap so that the decomposition process is mathematically reversible. Thus, sets of complementary wavelets are useful in wavelet based compression/decompression algorithms where it is desirable to recover the original information with minimal loss. In formal terms, this representation is a wavelet series representation of a square-integrable function with respect to either a complete, orthonormal set of basis functions, or an overcomplete set or frame of a vector space, for the Hilbert space of square integrable functions.

Seismic wavelet

Name The word wavelet has been used for decades in digital signal processing and exploration geophysics. The equivalent French word ondelette meaning "small wave" was used by Morlet and Grossmann in the early 1980s.

Wavelet theory Wavelet theory is applicable to several subjects. All wavelet transforms may be considered forms of time-frequency representation for continuous-time (analog) signals and so are related to harmonic analysis. Almost all practically useful discrete wavelet transforms use discrete-time filterbanks. These filter banks are called the wavelet and scaling coefficients in wavelets nomenclature. These filterbanks may contain either finite impulse response (FIR) or infinite impulse response (IIR) filters. The wavelets forming a continuous wavelet transform (CWT) are subject to the

Wavelet

178

uncertainty principle of Fourier analysis respective sampling theory: Given a signal with some event in it, one cannot assign simultaneously an exact time and frequency response scale to that event. The product of the uncertainties of time and frequency response scale has a lower bound. Thus, in the scaleogram of a continuous wavelet transform of this signal, such an event marks an entire region in the time-scale plane, instead of just one point. Also, discrete wavelet bases may be considered in the context of other forms of the uncertainty principle. Wavelet transforms are broadly divided into three classes: continuous, discrete and multiresolution-based.

Continuous wavelet transforms (continuous shift and scale parameters) In continuous wavelet transforms, a given signal of finite energy is projected on a continuous family of frequency bands (or similar subspaces of the Lp function space L2(R) ). For instance the signal may be represented on every frequency band of the form [f, 2f] for all positive frequencies f > 0. Then, the original signal can be reconstructed by a suitable integration over all the resulting frequency components. The frequency bands or subspaces (sub-bands) are scaled versions of a subspace at scale 1. This subspace in turn is in most situations generated by the shifts of one generating function ψ in L2(R), the mother wavelet. For the example of the scale one frequency band [1, 2] this function is

with the (normalized) sinc function. That, Meyer's, and two other examples of mother wavelets are:

Meyer

Morlet

Mexican Hat

The subspace of scale a or frequency band [1/a, 2/a] is generated by the functions (sometimes called child wavelets)

where a is positive and defines the scale and b is any real number and defines the shift. The pair (a, b) defines a point in the right halfplane R+ × R. The projection of a function x onto the subspace of scale a then has the form

with wavelet coefficients

See a list of some Continuous wavelets. For the analysis of the signal x, one can assemble the wavelet coefficients into a scaleogram of the signal.

Wavelet

179

Discrete wavelet transforms (discrete shift and scale parameters) It is computationally impossible to analyze a signal using all wavelet coefficients, so one may wonder if it is sufficient to pick a discrete subset of the upper halfplane to be able to reconstruct a signal from the corresponding wavelet coefficients. One such system is the affine system for some real parameters a > 1, b > 0. The corresponding discrete subset of the halfplane consists of all the points (am, namb) with m, n in Z. The corresponding baby wavelets are now given as

A sufficient condition for the reconstruction of any signal x of finite energy by the formula

form a tight frame of L2(R).

is that the functions

Multiresolution based discrete wavelet transforms In any discretised wavelet transform, there are only a finite number of wavelet coefficients for each bounded rectangular region in the upper halfplane. Still, each coefficient requires the evaluation of an integral. In special situations this numerical complexity can be avoided if the scaled and shifted wavelets form a multiresolution analysis. This means that there has to exist an auxiliary function, the father wavelet φ in L2(R), and that a is an integer. A typical choice is a = 2 and b = 1. The most famous pair of father and mother wavelets is the Daubechies 4-tap wavelet. Note that not every orthonormal discrete wavelet basis can be associated to a multiresolution analysis; for example, the Journe wavelet admits no multiresolution analysis.

D4 wavelet

From the mother and father wavelets one constructs the subspaces

The mother wavelet

keeps the time domain properties, while the father wavelets

keeps the frequency

domain properties. From these it is required that the sequence forms a multiresolution analysis of L2 and that the subspaces

are the orthogonal

"differences" of the above sequence, that is, Wm is the orthogonal complement of Vm inside the subspace Vm−1, In analogy to the sampling theorem one may conclude that the space Vm with sampling distance 2m more or less covers the frequency baseband from 0 to 2−m-1. As orthogonal complement, Wm roughly covers the band [2−m−1, 2−m]. From those inclusions and orthogonality relations, especially sequences

and so that so that

, follows the existence of

that satisfy the identities and

Wavelet

180

The second identity of the first pair is a refinement equation for the father wavelet φ. Both pairs of identities form the basis for the algorithm of the fast wavelet transform. From the multiresolution analysis derives the orthogonal decomposition of the space L2 as

For any signal or function

this gives a representation in basis functions of the corresponding subspaces as

where the coefficients are and .

Mother wavelet For practical applications, and for efficiency reasons, one prefers continuously differentiable functions with compact support as mother (prototype) wavelet (functions). However, to satisfy analytical requirements (in the continuous WT) and in general for theoretical reasons, one chooses the wavelet functions from a subspace of the space This is the space of measurable functions that are absolutely and square integrable: and Being in this space ensures that one can formulate the conditions of zero mean and square norm one: is the condition for zero mean, and is the condition for square norm one. For ψ to be a wavelet for the continuous wavelet transform (see there for exact statement), the mother wavelet must satisfy an admissibility criterion (loosely speaking, a kind of half-differentiability) in order to get a stably invertible transform. For the discrete wavelet transform, one needs at least the condition that the wavelet series is a representation of the identity in the space L2(R). Most constructions of discrete WT make use of the multiresolution analysis, which defines the wavelet by a scaling function. This scaling function itself is solution to a functional equation. In most situations it is useful to restrict ψ to be a continuous function with a higher number M of vanishing moments, i.e. for all integer m < M

The mother wavelet is scaled (or dilated) by a factor of a and translated (or shifted) by a factor of b to give (under Morlet's original formulation):

For the continuous WT, the pair (a,b) varies over the full half-plane R+ × R; for the discrete WT this pair varies over a discrete subset of it, which is also called affine group. These functions are often incorrectly referred to as the basis functions of the (continuous) transform. In fact, as in the continuous Fourier transform, there is no basis in the continuous wavelet transform. Time-frequency interpretation uses a subtly different formulation (after Delprat). Restriction：

Wavelet

181

(1) (2)

when a1 = a and b1 = b, has a finite time interval

Comparisons with Fourier transform (continuous-time) The wavelet transform is often compared with the Fourier transform, in which signals are represented as a sum of sinusoids. In fact, the Fourier transform can be viewed as a special case of the continuous wavelet transform with the choice of the mother wavelet . The main difference in general is that wavelets are localized in both time and frequency whereas the standard Fourier transform is only localized in frequency. The Short-time Fourier transform (STFT) is similar to the wavelet transform, in that it is also time and frequency localized, but there are issues with the frequency/time resolution trade-off. In particular, assuming a rectangular window region, one may think of the STFT as a transform with a slightly different kernel

where

can often be written as

, where

and u respectively denote the length and

temporal offset of the windowing function. Using Parseval’s theorem, one may define the wavelet’s energy as = From this, the square of the temporal support of the window offset by time u is given by

and the square of the spectral support of the window acting on a frequency

As stated by the Heisenberg uncertainty principle, the product of the temporal and spectral supports for any given time-frequency atom, or resolution cell. The STFT windows restrict the resolution cells to spectral and temporal supports determined by . Multiplication with a rectangular window in the time domain corresponds to convolution with a function in the frequency domain, resulting in spurious ringing artifacts for short/localized temporal windows. With the continuous-time Fourier Transform, and this convolution is with a delta function in Fourier space, resulting in the true Fourier transform of the signal

. The window function may be some other apodizing filter,

such as a Gaussian. The choice of windowing function will affect the approximation error relative to the true Fourier transform. A given resolution cell’s time-bandwidth product may not be exceeded with the STFT. All STFT basis elements maintain a uniform spectral and temporal support for all temporal shifts or offsets, thereby attaining an equal resolution in time for lower and higher frequencies. The resolution is purely determined by the sampling width. In contrast, the wavelet transform’s multiresolutional properties enables large temporal supports for lower frequencies while maintaining short temporal widths for higher frequencies by the scaling properties of the wavelet transform. This property extends conventional time-frequency analysis into time-scale analysis.[1]

Wavelet

The discrete wavelet transform is less computationally complex, taking O(N) time as compared to O(N log N) for the fast Fourier transform. This computational advantage is not inherent to the transform, but reflects the choice of a logarithmic division of frequency, in contrast to the equally spaced frequency divisions of the FFT (Fast Fourier Transform) which uses the same basis functions as DFT (Discrete Fourier Transform).[2] It is also important to note that this complexity only applies when the filter size has no relation to the signal size. A wavelet without compact support such as the Shannon wavelet would require O(N2). (For instance, a logarithmic Fourier Transform also exists with O(N) complexity, but the original signal must be sampled logarithmically in time, which is only useful for certain types of signals.[3])

182

STFT time-frequency atoms (left) and DWT time-scale atoms (right). The time-frequency atoms are four different basis functions used for the STFT (i.e. four separate Fourier transforms required). The time-scale atoms of the DWT achieve small temporal widths for high frequencies and good temporal widths for low frequencies with a single transform basis set.

Definition of a wavelet There are a number of ways of defining a wavelet (or a wavelet family).

Scaling filter An orthogonal wavelet is entirely defined by the scaling filter – a low-pass finite impulse response (FIR) filter of length 2N and sum 1. In biorthogonal wavelets, separate decomposition and reconstruction filters are defined. For analysis with orthogonal wavelets the high pass filter is calculated as the quadrature mirror filter of the low pass, and reconstruction filters are the time reverse of the decomposition filters. Daubechies and Symlet wavelets can be defined by the scaling filter.

Scaling function Wavelets are defined by the wavelet function ψ(t) (i.e. the mother wavelet) and scaling function φ(t) (also called father wavelet) in the time domain. The wavelet function is in effect a band-pass filter and scaling it for each level halves its bandwidth. This creates the problem that in order to cover the entire spectrum, an infinite number of levels would be required. The scaling function filters the lowest level of the transform and ensures all the spectrum is covered. See [4] for a detailed explanation. For a wavelet with compact support, φ(t) can be considered finite in length and is equivalent to the scaling filter g. Meyer wavelets can be defined by scaling functions

Wavelet function The wavelet only has a time domain representation as the wavelet function ψ(t). For instance, Mexican hat wavelets can be defined by a wavelet function. See a list of a few Continuous wavelets.

History The development of wavelets can be linked to several separate trains of thought, starting with Haar's work in the early 20th century. Later work by Dennis Gabor yielded Gabor atoms (1946), which are constructed similarly to wavelets, and applied to similar purposes. Notable contributions to wavelet theory can be attributed to Zweig’s discovery of the continuous wavelet transform in 1975 (originally called the cochlear transform and discovered while

Wavelet studying the reaction of the ear to sound),[5] Pierre Goupillaud, Grossmann and Morlet's formulation of what is now known as the CWT (1982), Jan-Olov Strömberg's early work on discrete wavelets (1983), Daubechies' orthogonal wavelets with compact support (1988), Mallat's multiresolution framework (1989), Akansu's Binomial QMF (1990), Nathalie Delprat's time-frequency interpretation of the CWT (1991), Newland's harmonic wavelet transform (1993) and many others since.

Timeline • First wavelet (Haar wavelet) by Alfréd Haar (1909) • Since the 1970s: George Zweig, Jean Morlet, Alex Grossmann • Since the 1980s: Yves Meyer, Stéphane Mallat, Ingrid Daubechies, Ronald Coifman, Ali Akansu, Victor Wickerhauser,

Wavelet transforms A wavelet is a mathematical function used to divide a given function or continuous-time signal into different scale components. Usually one can assign a frequency range to each scale component. Each scale component can then be studied with a resolution that matches its scale. A wavelet transform is the representation of a function by wavelets. The wavelets are scaled and translated copies (known as "daughter wavelets") of a finite-length or fast-decaying oscillating waveform (known as the "mother wavelet"). Wavelet transforms have advantages over traditional Fourier transforms for representing functions that have discontinuities and sharp peaks, and for accurately deconstructing and reconstructing finite, non-periodic and/or non-stationary signals. Wavelet transforms are classified into discrete wavelet transforms (DWTs) and continuous wavelet transforms (CWTs). Note that both DWT and CWT are continuous-time (analog) transforms. They can be used to represent continuous-time (analog) signals. CWTs operate over every possible scale and translation whereas DWTs use a specific subset of scale and translation values or representation grid. There are a large number of wavelet transforms each suitable for different applications. For a full list see list of wavelet-related transforms but the common ones are listed below: • • • • • • • •

Continuous wavelet transform (CWT) Discrete wavelet transform (DWT) Fast wavelet transform (FWT) Lifting scheme & Generalized Lifting Scheme Wavelet packet decomposition (WPD) Stationary wavelet transform (SWT) Fractional Fourier transform (FRFT) Fractional wavelet transform (FRWT)

Generalized transforms There are a number of generalized transforms of which the wavelet transform is a special case. For example, Joseph Segman introduced scale into the Heisenberg group, giving rise to a continuous transform space that is a function of time, scale, and frequency. The CWT is a two-dimensional slice through the resulting 3d time-scale-frequency volume. Another example of a generalized transform is the chirplet transform in which the CWT is also a two dimensional slice through the chirplet transform. An important application area for generalized transforms involves systems in which high frequency resolution is crucial. For example, darkfield electron optical transforms intermediate between direct and reciprocal space have been widely used in the harmonic analysis of atom clustering, i.e. in the study of crystals and crystal defects.[6] Now

183

Wavelet that transmission electron microscopes are capable of providing digital images with picometer-scale information on atomic periodicity in nanostructure of all sorts, the range of pattern recognition[7] and strain[8]/metrology[9] applications for intermediate transforms with high frequency resolution (like brushlets[10] and ridgelets[11]) is growing rapidly. Fractional wavelet transform (FRWT) is a generalization of the classical wavelet transform in the fractional Fourier transform domains. This transform is capable of providing the time- and fractional-domain information simultaneously and representing signals in the time-fractional-frequency plane.[12]

Applications of Wavelet Transform Generally, an approximation to DWT is used for data compression if a signal is already sampled, and the CWT for signal analysis.[13] Thus, DWT approximation is commonly used in engineering and computer science, and the CWT in scientific research. Like some other transforms, wavelet transforms can be used to transform data, then encode the transformed data, resulting in effective compression. For example, JPEG 2000 is an image compression standard that uses biorthogonal wavelets. This means that although the frame is overcomplete, it is a tight frame (see types of Frame of a vector space), and the same frame functions (except for conjugation in the case of complex wavelets) are used for both analysis and synthesis, i.e., in both the forward and inverse transform. For details see wavelet compression. A related use is for smoothing/denoising data based on wavelet coefficient thresholding, also called wavelet shrinkage. By adaptively thresholding the wavelet coefficients that correspond to undesired frequency components smoothing and/or denoising operations can be performed. Wavelet transforms are also starting to be used for communication applications. Wavelet OFDM is the basic modulation scheme used in HD-PLC (a power line communications technology developed by Panasonic), and in one of the optional modes included in the IEEE 1901 standard. Wavelet OFDM can achieve deeper notches than traditional FFT OFDM, and wavelet OFDM does not require a guard interval (which usually represents significant overhead in FFT OFDM systems).[14]

As a representation of a signal Often, signals can be represented well as a sum of sinusoids. However, consider a non-continuous signal with an abrupt discontinuity; this signal can still be represented as a sum of sinusoids, but requires an infinite number, which is an observation known as Gibbs phenomenon. This, then, requires an infinite number of Fourier coefficients, which is not practical for many applications, such as compression. Wavelets are more useful for describing these signals with discontinuities because of their time-localized behavior (both Fourier and wavelet transforms are frequency-localized, but wavelets have an additional time-localization property). Because of this, many types of signals in practice may be non-sparse in the Fourier domain, but very sparse in the wavelet domain. This is particularly useful in signal reconstruction, especially in the recently popular field of compressed sensing. (Note that the Short-time Fourier transform (STFT) is also localized in time and frequency, but there are often problems with the frequency-time resolution trade-off. Wavelets are better signal representations because of multiresolution analysis.) This motivates why wavelet transforms are now being adopted for a vast number of applications, often replacing the conventional Fourier Transform. Many areas of physics have seen this paradigm shift, including molecular dynamics, ab initio calculations, astrophysics, density-matrix localisation, seismology, optics, turbulence and quantum mechanics. This change has also occurred in image processing, EEG, EMG,[15] ECG analyses, brain rhythms, DNA analysis, protein analysis, climatology, human sexual response analysis,[16] general signal processing, speech recognition, acoustics, vibration signals,[17] computer graphics, multifractal analysis, and sparse coding. In computer vision and image processing, the notion of scale space representation and Gaussian derivative operators is

184

Wavelet

185

regarded as a canonical multi-scale representation.

Wavelet Denoising Suppose we measure a noisy signal

. Assume s has a sparse representation in a certain wavelet bases,

and So

.

Most elements in p are 0 or close to 0, and Since W is orthogonal, the estimation problem amounts to recovery of a signal in iid Gaussian noise. As p is sparse, one method is to apply a gaussian mixture model for p. Assume a prior

,

is the variance of "significant" coefficients, and

is the variance of "insignificant" coefficients. Then

,

is called the shrinkage factor, which depends on the prior variances

and

. The effect of the shrinkage factor is that small coefficients are set early to 0, and large coefficients are unaltered. Small coefficients are mostly noises, and large coefficients contain actual signal. At last, apply the inverse wavelet transform to obtain

List of wavelets Discrete wavelets • • • • • • • • • • •

Beylkin (18) BNC wavelets Coiflet (6, 12, 18, 24, 30) Cohen-Daubechies-Feauveau wavelet (Sometimes referred to as CDF N/P or Daubechies biorthogonal wavelets) Daubechies wavelet (2, 4, 6, 8, 10, 12, 14, 16, 18, 20, etc.) Binomial-QMF (Also referred to as Daubechies wavelet) Haar wavelet Mathieu wavelet Legendre wavelet Villasenor wavelet Symlet[18]

Continuous wavelets Real-valued • • • • • •

Beta wavelet Hermitian wavelet Hermitian hat wavelet Meyer wavelet Mexican hat wavelet Shannon wavelet

Wavelet Complex-valued • • • • •

Complex Mexican hat wavelet fbsp wavelet Morlet wavelet Shannon wavelet Modified Morlet wavelet

Notes [1] Mallat, Stephane. "A wavelet tour of signal processing. 1998." 250-252. [2] The Scientist and Engineer's Guide to Digital Signal Processing By Steven W. Smith, Ph.D. chapter 8 equation 8-1: http:/ / www. dspguide. com/ ch8/ 4. htm [3] http:/ / homepages. dias. ie/ ~ajones/ publications/ 28. pdf [4] http:/ / www. polyvalens. com/ blog/ ?page_id=15#7. + The+ scaling+ function+ %5B7%5D [5] http:/ / scienceworld. wolfram. com/ biography/ Zweig. html Zweig, George Biography on Scienceworld.wolfram.com [6] P. Hirsch, A. Howie, R. Nicholson, D. W. Pashley and M. J. Whelan (1965/1977) Electron microscopy of thin crystals (Butterworths, London/Krieger, Malabar FLA) ISBN 0-88275-376-2 [7] P. Fraundorf, J. Wang, E. Mandell and M. Rose (2006) Digital darkfield tableaus, Microscopy and Microanalysis 12:S2, 1010–1011 (cf. arXiv:cond-mat/0403017 (http:/ / arxiv. org/ abs/ cond-mat/ 0403017)) [8] M. J. Hÿtch, E. Snoeck and R. Kilaas (1998) Quantitative measurement of displacement and strain fields from HRTEM micrographs, Ultramicroscopy 74:131-146. [9] Martin Rose (2006) Spacing measurements of lattice fringes in HRTEM image using digital darkfield decomposition (M.S. Thesis in Physics, U. Missouri – St. Louis) [10] F. G. Meyer and R. R. Coifman (1997) Applied and Computational Harmonic Analysis 4:147. [11] A. G. Flesia, H. Hel-Or, A. Averbuch, E. J. Candes, R. R. Coifman and D. L. Donoho (2001) Digital implementation of ridgelet packets (Academic Press, New York). [12] J. Shi, N.-T. Zhang, and X.-P. Liu, "A novel fractional wavelet transform and its applications," Sci. China Inf. Sci., vol. 55, no. 6, pp. 1270–1279, June 2012. URL: http:/ / www. springerlink. com/ content/ q01np2848m388647/ [13] A.N. Akansu, W.A. Serdijn and I.W. Selesnick, Emerging applications of wavelets: A review (http:/ / web. njit. edu/ ~akansu/ PAPERS/ ANA-IWS-WAS-ELSEVIER PHYSCOM 2010. pdf), Physical Communication, Elsevier, vol. 3, issue 1, pp. 1-18, March 2010. [14] An overview of P1901 PHY/MAC proposal. [15] J. Rafiee et al. Feature extraction of forearm EMG signals for prosthetics, Expert Systems with Applications 38 (2011) 4058–67. [16] J. Rafiee et al. Female sexual responses using signal processing techniques, The Journal of Sexual Medicine 6 (2009) 3086–96. (pdf) (http:/ / rafiee. us/ files/ JSM_2009. pdf) [17] J. Rafiee and Peter W. Tse, Use of autocorrelation in wavelet coefficients for fault diagnosis, Mechanical Systems and Signal Processing 23 (2009) 1554–72. [18] Matlab Toolbox – URL: http:/ / matlab. izmiran. ru/ help/ toolbox/ wavelet/ ch06_a32. html

References • Paul S. Addison, The Illustrated Wavelet Transform Handbook, Institute of Physics, 2002, ISBN 0-7503-0692-0 • Ali Akansu and Richard Haddad, Multiresolution Signal Decomposition: Transforms, Subbands, Wavelets, Academic Press, 1992, ISBN 0-12-047140-X • B. Boashash, editor, "Time-Frequency Signal Analysis and Processing – A Comprehensive Reference", Elsevier Science, Oxford, 2003, ISBN 0-08-044335-4. • Tony F. Chan and Jackie (Jianhong) Shen, Image Processing and Analysis – Variational, PDE, Wavelet, and Stochastic Methods, Society of Applied Mathematics, ISBN 0-89871-589-X (2005) • Ingrid Daubechies, Ten Lectures on Wavelets, Society for Industrial and Applied Mathematics, 1992, ISBN 0-89871-274-2 • Ramazan Gençay, Faruk Selçuk and Brandon Whitcher, An Introduction to Wavelets and Other Filtering Methods in Finance and Economics, Academic Press, 2001, ISBN 0-12-279670-5 • Haar A., Zur Theorie der orthogonalen Funktionensysteme, Mathematische Annalen, 69, pp 331–371, 1910. • Barbara Burke Hubbard, "The World According to Wavelets: The Story of a Mathematical Technique in the Making", AK Peters Ltd, 1998, ISBN 1-56881-072-5, ISBN 978-1-56881-072-0

186

Wavelet • Gerald Kaiser, A Friendly Guide to Wavelets, Birkhauser, 1994, ISBN 0-8176-3711-7 • Stéphane Mallat, "A wavelet tour of signal processing" 2nd Edition, Academic Press, 1999, ISBN 0-12-466606-X • Donald B. Percival and Andrew T. Walden, Wavelet Methods for Time Series Analysis, Cambridge University Press, 2000, ISBN 0-521-68508-7 • Press, WH; Teukolsky, SA; Vetterling, WT; Flannery, BP (2007), "Section 13.10. Wavelet Transforms" (http:// apps.nrbook.com/empanel/index.html#pg=699), Numerical Recipes: The Art of Scientific Computing (3rd ed.), New York: Cambridge University Press, ISBN 978-0-521-88068-8 • P. P. Vaidyanathan, Multirate Systems and Filter Banks, Prentice Hall, 1993, ISBN 0-13-605718-7 • Mladen Victor Wickerhauser, Adapted Wavelet Analysis From Theory to Software, A K Peters Ltd, 1994, ISBN 1-56881-041-5 • Martin Vetterli and Jelena Kovačević, "Wavelets and Subband Coding", Prentice Hall, 1995, ISBN 0-13-097080-8

External links • Hazewinkel, Michiel, ed. (2001), "Wavelet analysis" (http://www.encyclopediaofmath.org/index.php?title=p/ w097160), Encyclopedia of Mathematics, Springer, ISBN 978-1-55608-010-4 • OpenSource Wavelet C# Code (http://www.waveletstudio.net/) • JWave – Open source Java implementation of several orthogonal and non-orthogonal wavelets (https://code. google.com/p/jwave/) • Wavelet Analysis in Mathematica (http://reference.wolfram.com/mathematica/guide/Wavelets.html) (A very comprehensive set of wavelet analysis tools) • 1st NJIT Symposium on Wavelets (April 30, 1990) (First Wavelets Conference in USA) (http://web.njit.edu/ ~ali/s1.htm) • Binomial-QMF Daubechies Wavelets (http://web.njit.edu/~ali/NJITSYMP1990/ AkansuNJIT1STWAVELETSSYMPAPRIL301990.pdf) • Wavelets (http://www-math.mit.edu/~gs/papers/amsci.pdf) by Gilbert Strang, American Scientist 82 (1994) 250–255. (A very short and excellent introduction) • Wavelet Digest (http://www.wavelet.org) • NASA Signal Processor featuring Wavelet methods (http://www.grc.nasa.gov/WWW/OptInstr/ NDE_Wave_Image_ProcessorLab.html) Description of NASA Signal & Image Processing Software and Link to Download • Course on Wavelets given at UC Santa Barbara, 2004 (http://wavelets.ens.fr/ENSEIGNEMENT/COURS/ UCSB/index.html) • The Wavelet Tutorial by Polikar (http://users.rowan.edu/~polikar/WAVELETS/WTtutorial.html) (Easy to understand when you have some background with fourier transforms!) • OpenSource Wavelet C++ Code (http://herbert.the-little-red-haired-girl.org/en/software/wavelet/) • Wavelets for Kids (PDF file) (http://www.isye.gatech.edu/~brani/wp/kidsA.pdf) (Introductory (for very smart kids!)) • Link collection about wavelets (http://www.cosy.sbg.ac.at/~uhl/wav.html) • Gerald Kaiser's acoustic and electromagnetic wavelets (http://wavelets.com/pages/center.html) • A really friendly guide to wavelets (http://perso.wanadoo.fr/polyvalens/clemens/wavelets/wavelets.html) • Wavelet-based image annotation and retrieval (http://www.alipr.com) • Very basic explanation of Wavelets and how FFT relates to it (http://www.relisoft.com/Science/Physics/ sampling.html) • A Practical Guide to Wavelet Analysis (http://paos.colorado.edu/research/wavelets/) is very helpful, and the wavelet software in FORTRAN, IDL and MATLAB are freely available online. Note that the biased wavelet power spectrum needs to be rectified (http://ocgweb.marine.usf.edu/~liu/wavelet.html).

187

Wavelet

188

• WITS: Where Is The Starlet? (http://www.laurent-duval.eu/siva-wits-where-is-the-starlet.html) A dictionary of tens of wavelets and wavelet-related terms ending in -let, from activelets to x-lets through bandlets, contourlets, curvelets, noiselets, wedgelets. • Python Wavelet Transforms Package (http://www.pybytes.com/pywavelets/) OpenSource code for computing 1D and 2D Discrete wavelet transform, Stationary wavelet transform and Wavelet packet transform. • Wavelet Library (http://pages.cs.wisc.edu/~kline/wvlib) GNU/GPL library for n-dimensional discrete wavelet/framelet transforms. • The Fractional Spline Wavelet Transform (http://bigwww.epfl.ch/publications/blu0001.pdf) describes a fractional wavelet transform based on fractional b-Splines. • A Panorama on Multiscale Geometric Representations, Intertwining Spatial, Directional and Frequency Selectivity (http://dx.doi.org/10.1016/j.sigpro.2011.04.025) provides a tutorial on two-dimensional oriented wavelets and related geometric multiscale transforms. • HD-PLC Alliance (http://www.hd-plc.org/) • Signal Denoising using Wavelets (http://tx.technion.ac.il/~rc/SignalDenoisingUsingWavelets_RamiCohen. pdf) • A Concise Introduction to Wavelets (http://www.docstoc.com/docs/160022503/ A-Concise-Introduction-to-Wavelets) by René Puchinger.

Discrete wavelet transform In numerical analysis and functional analysis, a discrete wavelet transform (DWT) is any wavelet transform for which the wavelets are discretely sampled. As with other wavelet transforms, a key advantage it has over Fourier transforms is temporal resolution: it captures both frequency and location information (location in time).

Examples Haar wavelets Main article: Haar wavelet The first DWT was invented by the Hungarian mathematician Alfréd Haar. For an input represented by a list of numbers, the Haar wavelet transform may be considered to simply pair up input values, storing the difference and passing the sum. This process is repeated recursively, pairing up the sums to provide the next scale: finally resulting in differences and one final sum.

An example of the 2D discrete wavelet transform that is used in JPEG2000. The original image is high-pass filtered, yielding the three large images, each describing local changes in brightness (details) in the original image. It is then low-pass filtered and downscaled, yielding an approximation image; this image is high-pass filtered to produce the three smaller detail images, and low-pass filtered to produce the final approximation image in the upper-left.

Discrete wavelet transform

Daubechies wavelets Main article: Daubechies wavelet The most commonly used set of discrete wavelet transforms was formulated by the Belgian mathematician Ingrid Daubechies in 1988. This formulation is based on the use of recurrence relations to generate progressively finer discrete samplings of an implicit mother wavelet function; each resolution is twice that of the previous scale. In her seminal paper, Daubechies derives a family of wavelets, the first of which is the Haar wavelet. Interest in this field has exploded since then, and many variations of Daubechies' original wavelets were developed.[1]

The Dual-Tree Complex Wavelet Transform (ℂWT) The Dual-Tree Complex Wavelet Transform (ℂWT) is a relatively recent enhancement to the discrete wavelet transform (DWT), with important additional properties: It is nearly shift invariant and directionally selective in two and higher dimensions. It achieves this with a redundancy factor of only substantially lower than the undecimated DWT. The multidimensional (M-D) dual-tree ℂWT is nonseparable but is based on a computationally efficient, separable filter bank (FB).[2]

Others Other forms of discrete wavelet transform include the non- or undecimated wavelet transform (where downsampling is omitted), the Newland transform (where an orthonormal basis of wavelets is formed from appropriately constructed top-hat filters in frequency space). Wavelet packet transforms are also related to the discrete wavelet transform. Complex wavelet transform is another form.

Properties The Haar DWT illustrates the desirable properties of wavelets in general. First, it can be performed in operations; second, it captures not only a notion of the frequency content of the input, by examining it at different scales, but also temporal content, i.e. the times at which these frequencies occur. Combined, these two properties make the Fast wavelet transform (FWT) an alternative to the conventional Fast Fourier Transform (FFT).

Time Issues Due to the rate-change operators in the filter bank, the discrete WT is not time-invariant but actually very sensitive to the alignment of the signal in time. To address the time-varying problem of wavelet transforms, Mallat and Zhong proposed a new algorithm for wavelet representation of a signal, which is invariant to time shifts.[3] According to this algorithm, which is called a TI-DWT, only the scale parameter is sampled along the dyadic sequence 2^j (j∈Z) and the wavelet transform is calculated for each point in time.[4][5]

Applications The discrete wavelet transform has a huge number of applications in science, engineering, mathematics and computer science. Most notably, it is used for signal coding, to represent a discrete signal in a more redundant form, often as a preconditioning for data compression. Practical applications can also be found in signal processing of accelerations for gait analysis,[6] in digital communications and many others.[7] [8][9] It is shown that discrete wavelet transform (discrete in scale and shift, and continuous in time) is successfully implemented as analog filter bank in biomedical signal processing for design of low-power pacemakers and also in ultra-wideband (UWB) wireless communications.[10]

189

Discrete wavelet transform

Comparison with Fourier transform See also: Discrete Fourier transform To illustrate the differences and similarities between the discrete wavelet transform with the discrete Fourier transform, consider the DWT and DFT of the following sequence: (1,0,0,0), a unit impulse. The DFT has orthogonal basis (DFT matrix):

while the DWT with Haar wavelets for length 4 data has orthogonal basis in the rows of:

(To simplify notation, whole numbers are used, so the bases are orthogonal but not orthonormal.) Preliminary observations include: • Wavelets have location – the (1,1,–1,–1) wavelet corresponds to “left side” versus “right side”, while the last two wavelets have support on the left side or the right side, and one is a translation of the other. • Sinusoidal waves do not have location – they spread across the whole space – but do have phase – the second and third waves are translations of each other, corresponding to being 90° out of phase, like cosine and sine, of which these are discrete versions. Decomposing the sequence with respect to these bases yields:

The DWT demonstrates the localization: the (1,1,1,1) term gives the average signal value, the (1,1,–1,–1) places the signal in the left side of the domain, and the (1,–1,0,0) places it at the left side of the left side, and truncating at any stage yields a downsampled version of the signal:

190

Discrete wavelet transform

191

The DFT, by contrast, expresses the sequence by the interference of waves of various frequencies – thus truncating the series yields a low-pass filtered version of the series:

The sinc function, showing the time domain artifacts (undershoot and ringing) of truncating a Fourier series.

Notably, the middle approximation (2-term) differs. From the frequency domain perspective, this is a better approximation, but from the time domain perspective it has drawbacks – it exhibits undershoot – one of the values is negative, though the original series is non-negative everywhere – and ringing, where the right side is non-zero, unlike in the wavelet transform. On the other hand, the Fourier approximation correctly shows a peak, and all points are within of their correct value, though all points have error. The wavelet approximation, by contrast, places a peak on the left half, but has no peak at the first point, and while it is exactly correct for half the values (reflecting location), it has an error of

for the other values.

This illustrates the kinds of trade-offs between these transforms, and how in some respects the DWT provides preferable behavior, particularly for the modeling of transients.

Definition One level of the transform The DWT of a signal is calculated by passing it through a series of filters. First the samples are passed through a low pass filter with impulse response resulting in a convolution of the two:

The signal is also decomposed simultaneously using a high-pass filter

. The outputs giving the detail coefficients

(from the high-pass filter) and approximation coefficients (from the low-pass). It is important that the two filters are related to each other and they are known as a quadrature mirror filter. However, since half the frequencies of the signal have now been removed, half the samples can be discarded according to Nyquist’s rule. The filter outputs are then subsampled by 2 (Mallat's and the common notation is the opposite, g- high pass and h- low pass):

Discrete wavelet transform

192

This decomposition has halved the time resolution since only half of each filter output characterises the signal. However, each output has half the frequency band of the input so the frequency resolution has been doubled.

Block diagram of filter analysis

With the subsampling operator

the above summation can be written more concisely.

However computing a complete convolution

with subsequent downsampling would waste computation time.

The Lifting scheme is an optimization where these two computations are interleaved.

Cascading and Filter banks This decomposition is repeated to further increase the frequency resolution and the approximation coefficients decomposed with high and low pass filters and then down-sampled. This is represented as a binary tree with nodes representing a sub-space with a different time-frequency localisation. The tree is known as a filter bank.

A 3 level filter bank

At each level in the above diagram the signal is decomposed into low and high frequencies. Due to the decomposition process the input signal must be a multiple of where is the number of levels. For example a signal with 32 samples, frequency range 0 to

and 3 levels of decomposition, 4 output scales are

produced:

Frequency domain representation of the DWT

Level

Frequencies

Samples

3

to

4 to

4

2

to

8

1

to

16

Discrete wavelet transform

193

Relationship to the Mother Wavelet The filterbank implementation of wavelets can be interpreted as computing the wavelet coefficients of a discrete set of child wavelets for a given mother wavelet . In the case of the discrete wavelet transform, the mother wavelet is shifted and scaled by powers of two

where

is the scale parameter and

Recall that the wavelet coefficient signal of length

Now fix

is the shift parameter, both which are integers. of a signal

is the projection of

onto a wavelet, and let

be a

. In the case of a child wavelet in the discrete family above,

at a particular scale, so that

viewed as a convolution of

is a function of

can be

with a dilated, reflected, and normalized version of the mother wavelet,

, sampled at the points coefficients give at level

only. In light of the above equation,

. But this is precisely what the detail

of the discrete wavelet transform. Therefore, for an appropriate choice of

and

, the detail coefficients of the filter bank correspond exactly to a wavelet coefficient of a discrete set of child wavelets for a given mother wavelet . As an example, consider the discrete Haar wavelet, whose mother wavelet is reflected, and normalized version of this wavelet is

. Then the dilated,

, which is, indeed, the highpass

decomposition filter for the discrete Haar wavelet transform

Time Complexity The filterbank implementation of the Discrete Wavelet Transform takes only O(N) in certain cases, as compared to O(N log N) for the fast Fourier transform. Note that if

and

are both a constant length (i.e. their length is independent of N), then

and

each take O(N) time. The wavelet filterbank does each of these two O(N) convolutions, then splits the signal into two branches of size N/2. But it only recursively splits the upper branch convolved with (as contrasted with the FFT, which recursively splits both the upper branch and the lower branch). This leads to the following recurrence relation which leads to an O(N) time for the entire operation, as can be shown by a geometric series expansion of the above relation. As an example, the Discrete Haar Wavelet Transform is linear, since in that case 2.

and

are constant length

Discrete wavelet transform

Other transforms See also: Adam7 algorithm The Adam7 algorithm, used for interlacing in the Portable Network Graphics (PNG) format, is a multiscale model of the data which is similar to a DWT with Haar wavelets. Unlike the DWT, it has a specific scale – it starts from an 8×8 block, and it downsamples the image, rather than decimating (low-pass filtering, then downsampling). It thus offers worse frequency behavior, showing artifacts (pixelation) at the early stages, in return for simpler implementation.

Code example In its simplest form, the DWT is remarkably easy to compute. The Haar wavelet in Java: public static int[] discreteHaarWaveletTransform(int[] input) { // This function assumes that input.length=2^n, n>1 int[] output = new int[input.length]; for (int length = input.length >> 1; ; length >>= 1) { // length = input.length / 2^n, WITH n INCREASING to log(input.length) / log(2) for (int i = 0; i < length; ++i) { int sum = input[i * 2] + input[i * 2 + 1]; int difference = input[i * 2] - input[i * 2 + 1]; output[i] = sum; output[length + i] = difference; } if (length == 1) { return output; } //Swap arrays to do next iteration System.arraycopy(output, 0, input, 0, length

Kami adalah komuniti perkongsian. Sila muat naik ** 1 ** dokumen sebelum muat turun:

ATAU FACE TO DOWNLOAD