Cognitive Studies/Psychology/Visual Studies 201:
COGNITIVE SCIENCE IN CONTEXT

Auditory Signal Detection (One-Day Experiment)

Laboratory Module

by Douglas R. Elrod, Cornell University

Introduction

What is now known as Signal Detection Theory (SDT) got its start in radar research in the mid 1950's. Soon other fields, particularly psychology, had adopted its methods. Swets, Tanner, and Birdsall (1961) has a good overview of its history, and the mathematical details underlying it.

We won't have time to cover its full scope, but using its framework, one can generate an "ideal" response pattern for any detection task, if one is given the various costs and benefits of the particular detection and non-detection outcomes possible.

It uses two parameters, d' (d-prime) and β (beta), to describe the performance of any binary signal-detection system. D-prime reflects the sensitivity of the detector, and β is a criterion that reflects the tradeoff chosen between the goals of detecting the signal, when present, and not reporting the signal, when absent.

By applying a probabilistic model to detection, it provided a better description of human perception, particularly for marginal cases when one performs better than chance, yet one doesn't "feel" that one is seeing/hearing the stimulus.

Because of SDT, the use of forced-choice experiments, where one must produce an answer, even if one doesn't "consciously" perceive a stimulus, was given new support. Prior to this, it was often assumed that in that case, one just randomly chooses a response. SDT has been used to show that even in these cases some information is picked up from the environment.

A brief overview of Signal Detection Theory

The statistical model

Signal Detection diagram
Figure 1. Probability diagram for the general Signal Detection Theory model

We look at one stimulus dimension, represented by the x-axis, horizontally across the page. In the absence of a signal, we assume that the value along this axis that is used to make a judgment is distributed in a "normal curve" around the mean of a "noise distribution", represented in Fig. 1 by the blue-shaded area (the bell curve on the left). (See the footnote for how the curve was constructed.)

The presence of a signal shifts the resultant distribution to the right a certain amount, D (the Difference between the means of the two distributions), producing the pink-shaded signal+noise distribution shown on the right in Fig. 1.

In the simplest case, which we assume here, the "signal+noise" distribution has the same shape as the "noise" distribution. In particular, σ (sigma), the standard deviation, which reflects the "spread" of a normal distribution, is the same between the two distributions (see Fig. 1).

Signal Detection Theory assumes that, given this situation, we make our judgment of whether the signal is present, or not, by setting up a Criterion value, β (beta). In Fig. 1, this criterion line separates the probability space into two parts. When a value is picked up that exceeds β, we respond that the signal is present. When a value is picked up that doesn't exceed β, we respond that the signal is absent.

An important point is that the actual "comparison against β" takes place "internally", because some noise comes from the perceptual system itself. This also means that the easiest way to think of β is with no units -- as a "normally scaled" quantity. By convention, &beta is defined relative to the noise distribution, with 0 being at the center, and 1 being one standard deviation (σ) to the right in Fig. 1 (in the direction of the signal). So a β of 1.75 always means that there is a 4% chance of responding "Signal", when it is not present (a "false alarm") because 1.75 is the 96th percentile of a standard normal distribution. (To get a non-normalized version of β one multiplies by an estimate of σ.) For simplicity, we will use the normalized version here.

It also follows that, if we try to measure σ directly, in terms of the variability of external stimuli, we will underestimate it, because we aren't taking into account the "noise" (inefficiencies) of the perceptual system. Here we will focus on the more meaningful quantity, d' (d-prime) (see below).

Depending on the task demands, we may move β right or left. If we need to be conservative, and only respond that a signal is present when we are "really sure", it has been found that people act as though β were moved in the "signal" direction (right in Fig. 1). On the other hand, if it is more important to not miss any instances of a real signal, people tend to act as though β were moved in the "noise" direction (left in Fig. 1).

Task demands don't change the essential "detectability" of the stimulus, though. We measure this, relative to a given detector system, by the parameter d'. This is defined to be D divided by σ. When the noise and signal+noise distributions are far apart, relative to their spread, then it is easier to distinguish whether the signal is present or not. In this case, d' is high. But when the two distributions overlap greatly, d' approaches zero, and one's pattern of responses will be less related to whether or not a signal is actually present.

When comparing detector systems (on the same stimuli, for instance), d' can also be thought of as the "sensitivity" of the detector.

Possible response-outcomes to signals

Table 1 enumerates the possible ways that a binary response can be related to the presence or absence of a stimulus. The colored patterns in the cells correspond to the regions in the probability diagram of Fig. 1. The blue patterned areas are situations where the stimuli were actually produced by noise (lack of signal), and the pink-patterned areas are where the stimuli were actually produced by a signal (along with ever-present noise, of course).

The areas where the blue or pink pattern has lines going from upper left to lower right are those where a correct response was given (called "Correct Rejection" when noise leads to a "No Signal" response, and "Hit" when signal+noise leads to a "Signal" response). The areas where the blue or pink pattern has lines going from upper right to lower left are those where an incorrect response was given (called "False alarm" when noise leads to a "Signal" response, and "Miss" when signal+noise leads to a "No Signal" response).

Stimulus
Noise Signal+Noise
Response No Signal Correct Rejection Miss
Signal False Alarm Hit
Table 1. Possible responses to stimuli, for Signal Detection.

Referring back to Fig. 1, one should note that SDT predicts that, if d' is fixed, as one moves β to the right to reduce False Alarms, one necessarily increases the proportion of Misses, and vice-versa.

The Two-Signal Variant: Signal Discrimination

Discriminating between two signals can be seen as a variation on the classic signal detection problem. In the simplest case, signals A and B both have the noise that would be present in the "no signal" case added to them, as in Fig. 2.

Signal Discrimination diagram
Figure 2. Probability diagram for the Signal Discrimination Case

The possible outcomes of an observation are then shown in Table 2.

Stimulus
Signal A+Noise Signal B+Noise
Response Signal A Correct Discrimination (A) Mistake (A for B)
Signal B Mistake (B for A) Correct Discrimination (B)
Table 2. Possible responses to stimuli, for Signal Discrimination.

This model, then, can be used in the same way as the classic Signal Detection model. We will be applying it to the auditory domain to model discrimination between tones of various frequencies.

Reading Assignment

Laboratory Exercise

This laboratory exercise is designed to measure how sensitive we are to small changes in auditory frequency (which is experienced as "pitch").

It is adapted from Rosenblith & Stevens (1953), to use modern technology (such as computers), and be runnable in one day. As in their experiment, pure-tones are used, with rise and decay times of 40 ms.

Covariates such as loudness are controlled for by using a within-subject design. Each participant experiences all auditory stimuli at one computer.

Instructions

The Mac OS computers where the experiment can be run should each have a set of headphones attached. You should endeavor to sit at a computer as far away from other people as possible, to avoid distractions. If possible, arrange the moveable wall-sections to isolate the experimental area further.

Setting up the Headphones

If they aren't already, plug the headphones into the headphone jack on the left side of the monitor (this has a headphone symbol above it).

Make sure that volume is set to a reasonable level. For standardization, check that the computer volume control (In "System Preferences:Sound: Output Volume") is set to maximum, and the left-right balance is set to the midpoint.

Because the headphones will be plugged into the monitor jacks, the volume setting on the monitor will also affect the headphones. Set this to 50 (midway on the 0-100 scale). On the Panasonic monitors this is set with the "-" and "+" buttons, and on the Mitsubishi monitors this is set by pressing the "Menu" button to bring up the On Screen Display (OSD), then using the "-" and "+" buttons to change the volume, then pushing "Menu" several times until "OSD OFF" is highlighted, and then pushing "-" or "+" one more time.

Running the experiment

The experiment, in the basic form described here, should take about forty minutes to complete (plus the time for rest periods) for each participant.

To run the experiment, first quit out of other applications, and disable the screen saver, if one is present. Make sure the monitor is set to 640x480 resolution (otherwise the instructions will either not fit on the screen, or will be displayed in a frame in the middle of the screen).

Open the "Signal Detection" folder on the desktop, and then double-click on the "Auditory.script" file inside that folder. This should start SuperLab running the signal detection experiment. (If the computer is low on memory, you may find that the "Experiment Editor" display is empty at this point. You would then have to go to the File menu in SuperLab, and Open the "Auditory.script" from there.)

Note: DO NOT MAKE ANY CHANGES within the SuperLab Experiment Editor display and DO NOT press on the buttons at the bottom of the window.

When the SuperLab Experiment Editor window opens, go to the Experiment pull-down menu, and select Run. A different window will now open. This window allows you to enter the participant's name. Type in a unique name or designation for the participant and press the Run button in the window.

Another window will appear. This window allows you to specify where the results of the laboratory exercise will be saved, and what will be the name of the file in which the results will be saved. You can select the hard disk (on the Desktop, for instance, in a folder labeled with the experimenter's name) for now, but you should remember to copy the results file to a backup location (a writable CD, for instance) as soon as possible. Data files and other personal files on the computers' hard drives are liable to be erased.

Next, type in a unique name for the data file that will be created, replacing the default file name that automatically appears. Each data file must have a different, unique name. If two files have identical names, the more recent file will replace and destroy the older file.

After the location for saving the data file and the data file's name have been entered, the experiment automatically begins. Instructions will appear on the screen.

Note: If for any reason you need to terminate the experiment prematurely, hold down the command-key (Apple key) and press the period-key.

Method

What follows is a description of the SuperLab script, "Auditory.script":

First of all, the participant is asked how many years of musical training he or she has had (0=no training, 9=Nine or more years).

Then the participant is exposed to a tone, to make sure that the volume is reasonable, then to an example of a low-then-high tone pair, then an example of a high-then-low tone pair.

After that the participant does 12 practice trials, using tones just below and just above the experimental trial range.

Then the participant does the 403 experimental trials, which should take about 40 minutes to run through. In each, the task is to indicate if the second tone is higher or lower than the first tone, by hitting the h or l keys, respectively.

Data Analysis

Overview

There are two independent variables that were manipulated in the experiment. One is the initial frequency of the tone-pair. The other is the frequency-offset of the second tone, relative to the first. Thirteen different initial tones, from 247 Hz to 253 Hz in 0.5-Hz increments, were used. This range was chosen because it is centered around 250 Hz, one of the frequencies that Rosenblith & Stevens (1953) looked at, and it's in the range of normal speech-sounds.

The second tone was located at one of thirty-one different offsets from the initial tone, from minus 3.75 Hz to plus 3.75 Hz, in increments of 0.25 Hz, including the case where the initial and final tones were the same. (The identical-tone case will be excluded from the analyses below.) This range was chosen so that all participants would have a good chance of finding an offset where their correct-performance was midrange (compare with Fig. 2 in Rosenblith & Stevens).

There are two dependent variables that are measured during the experiment, the response ("Higher" or "Lower"), and the reaction time, RT. We will be mainly interested in the former of these, but you may also look at RT (see "Other Possible Analyses", below).

We will be making the assumption that the stimulus range, from 243.25 Hz to 256.75 Hz, is small enough that no substantial change in d' exists across it (i.e. that the auditory system isn't appreciably more sensitive to particular frequencies in this band). But we will expect that d' will go up as tones that are farther apart are compared (because D will be greater).

Note: You should always work on a copy of your original data, so that if something goes wrong, you can re-do your analyses, rather than having to re-do the experiment.

For more detailed information on the format of the data file generated by SuperLab, see the Appendix.

There is now an Excel template "SDT formulas template.xls" which automates the calculation of d' and β. By pasting your data into this (so the upper left of the data file is on the upper left of the template), d' and β should be computed automatically for each of the 15 different tone-differences. When the data is correctly aligned the cell to the right of "Calibration check:" should say "Aligned OK".

In some cases, the estimators for d' or β may not be finite -- this occurs when there are zero cell-counts (the task was too easy, or the participants were giving opposite responses to those called for by the task), see the discussion below regarding Special Cases.

If the Excel file is not available, you can duplicate its action by going through the following steps:

Recommended steps:

First, record the participant's "years of musical training". This is the Response to the "Intro1" trial. Then remove the "Intro" lines and the practice lines (the following twelve lines that deal with tones in the set {215Hz, 225Hz, 275Hz, and 285Hz}.

Also remove the other non-trial lines, "Beforefirst", "Beforesecond", "Beforethird", and "Endtrial". What you should have left are the 403 experimental trials.

Now you need to generate columns for the frequencies of the FirstTone and the SecondTone. You can do this most easily by using Excel to calculate FirstTone = SUM(F1-Whole, F1-Fract) and SecondTone = SUM(F2-Wh1, F2-Wh2, F2-Fract). If you don't know how to use a spreadsheet to compute these, ask the Lab Coordinator.

Then you can compute a new column, AbsToneDiff = ABS(SecondTone - FirstTone).

At this point, it would be useful to compute a column for "Correct Response" ("h" if ToneDiff > 0, "l" if ToneDiff < 0). We will be comparing this to the actual Response column. Note that "l" is a lower-case "L".

Now, we can make a new column that says what type of Response was obtained on each trial (which cell of Fig. 2 applies). A nested IF statement is useful here:

IF(Response="h" AND CorrectResponse="h") then "Correct Discrimination (H)" else IF(Response="h" AND CorrectResponse="l") then "Mistake (H for L)" else IF(Response="l" AND CorrectResponse="h") then "Mistake (L for H)" else IF(Response="l" AND CorrectResponse="l") then "Correct Discrimination (L)"

Now, you ought to sort the columns by the AbsToneDiff column (make sure you are working on a COPY of your original data, so you can get back the original order the trials were run in, if necessary).

Next, ignoring the "0" ToneDiff trials (which are the ones where the same tone was presented twice), for each of the 15 absolute differences, compute the four cells of Table 2. The number of trials in each condition goes into the cells (Excel has a "COUNTIF" function which helps with this). You ought to use an empty part of the spreadsheet laid out as in Table 3.

Abs. ToneDiff Correct Discrimination (L) Mistake (L for H) Mistake (H for L) Correct Discrimination (H)
15 3 8 10 5
14 4 6 9 7
etc...
1 2 5 11 8
Table 3. Example of Response-Outcomes vs. ToneDiff table.

Note that for each absolute value of ToneDiff, you use the data for the positive difference (where the second tone is higher than the first) for the "Mistake (L for H)" and "Correct Discrimination (H)" cells, and you use the data for the negative difference (where the second tone is lower than the first) for the "Mistake (H for L)" and "Correct Discrimination (L)" cells. Because 13 trials have a particular positive Tonediff, and 13 trials have a particular negative ToneDiff, the counts in the virtual 2x2 table for for each value of AbsToneDiff will add up to 26.

Now, you can estimate a separate d' and B for each of the AbsToneDiff values using the formulas shown below (using the count data you computed above).

d'-estimate
= NORMINV("Correct Discrimination (H)"/ ("Correct Discrimination (H)" + "Mistake (L for H)"))
- NORMINV("Mistake (H for L)"/ ("Mistake (H for L) + "Correct Discrimination (L)"))
and
β-estimate
= -NORMINV("Mistake (H for L)"/ ("Mistake (H for L) + "Correct Discrimination (L)"))
where NORMINV is the inverse of the cumulative normal distribution function (specified to be the standard version, with 0 mean, and 1 standard deviation). (For the case where only one d' and β are being estimated, these are the "maximum likelihood" estimators (MLE), see Dorfman & Alf (1968).)

Because NORMINV is only defined between 0 and 1, excluding the endpoints, a zero in any of the four cells, "Correct Discrimination (L)", "Mistake (L for H)", "Mistake (H for L)", or "Correct Discrimination (H)" will produce an undefined d'-estimate. Because lim(NORMINV(x), x -> 0) = -∞ (negative infinity), and lim(NORMINV(x), x -> 1) = ∞ (positive infinity), this should be interpreted either as an infinite, or undetermined, estimate of d' (see the Special Cases below).

Special cases: Zero cell counts

The special case of one zero indicates that this particular tone-difference was too easy to distinguish to get a good estimate of d' (either by producing no errors, or nothing but errors in one of the columns of Table 2). The first leads to an estimate for d' of ∞, the second leads to an estimate of d' of -∞ (see below for the meaning of negative d' estimates).

The special case of two zeros indicates either that the maximum-likelihood estimate of d' is undetermined (when the participant's responses are always "Higher" or always "Lower" -- these are cases when the β-estimate is -∞ or ∞, respectively), or &infin (perfect correct responses), or -∞ (perfect incorrect responses).

We don't need to consider the case of three cells with zero counts, because that can only occur when the subject is presented with only truly "Higher" pairs of stimuli, or only truly "Lower" pairs of stimuli.

Interpretation of negative estimate of d':

Although the parameter d' must be non-negative (given that it's merely the normally-scaled difference between the signal and noise distributions, see Fig. 1), our estimate of d' can be any positive, negative, or zero real value. Like large positive values, large negative d-prime estimates suggest that the detection system is sensitive to the signal, but in this case, in a way that tends to produce incorrect responses. In fact, it can be shown from the formula for our d' estimate that replacing correct responses by incorrect ones (and vice-versa) leads to a d'-estimate of the same absolute magnitude, but opposite sign.

Therefore, a large negative d' estimate could result from either

Analyses using d' and β

You can now test whether d' has the expected increase with AbsToneDiff by a linear regression. Treat d'-estimate as the dependent variable, and AbsToneDiff as the independent variable. Include a quadratic term, along with the linear one, to see if there is significant "nonlinearity" (a significant quadratic term means that you can reject that the data came from a simple linear relationship).

You can also examine whether β changes with ToneDiff.

Other possible analyses: Possible further lines of inquiry:

Laboratory Report for the Signal Detection Module

After you have completed the Laboratory Exercise, you will be ready to write a report in journal article format. Select a journal that represents an interest of yours. Follow its 'Instructions to Authors' and its general format in preparing your report. General guidelines can be found here.


Appendix

Guide to deciphering Results Files from the Auditory Signal Detection Experiment

Each participants' data will be stored in a file that can be opened using Microsoft Excel. It is also possible to read these files as text using other programs such as SimpleText or TeachText, or the statistics program StatView (which is particularly useful for statistical tests like t-tests and regression).

The participant code is at the top of the file, followed by the script name ("Auditory.script"), and the date and time when the experiment was run. Then there's a blank line, and the data columns follow, headed by the column-names which are listed below.

Each trial of the spreadsheet takes up one row of the spreadsheet, with successive trials going from top to bottom.

The first several trials are the information screens. The Response value for "Intro1" is the number of years that the participant had musical training. The next 12 trials are the practice trials (which should be omitted in experimental analyses), and the ones after that are the experimental trials. The columns in the datafile, from left to right, are as follows:

Note: It's best to edit a copy of the data file, rather than the original data file itself, given the risk of deleting data!


References:
This page maintained by Doug Elrod.
Note: If you are curious about how the normal-curve diagrams were created, see Doug Elrod's Scalable Normal Curve for an explanation.