The Psychophysics Symposium 1993

The Psychophysics Symposium was held in August 1993 as part of the New Zealand Psychological Society's Annual Conference. Papers were presented by John Whitmore, Vit Drga, Brian Scurfield, and Judi Lapsley. The following links are abstracts for these papers; more details are available from the authors.

John Whitmore
Vit Drga
Brian Scurfield
- The meaning of the area under the Receiver Operating Characteristic curve
Judi Lapsley
- Measures of Sensitivity in Single-Interval Forced-Choice and Two-Interval Forced-Choice tasks
- The Acoustical Uncertainty Principle and the Bandwidth-Duration Product

Abstracts

Why group operating characteristic (GOC) analysis?

John Whitmore

It is understood widely that experimental technique and research design reduce data varaibility or error, and that statistics measures the error which remains. The remaining error almost always confounds the comparison of data to theory. In psychoacoustics, GOC analysis statisically reduces data error further, providing a more precise and powerful way of testing theories of hearing.

Group operating characteristic (GOC) analysis of Type II decisions

John Whitmore and Susan Galvin

A few years ago, one of us (Susan Galvin) presented to the NZPS a theoretical analysis of Type I and Type II detection tasks in the theory of signal detectability. Type I tasks are concerned with how observers distinguish between environmental events, and Type II tasks are concerned with how observers distinguish between their own correct and incorrect decisions about those Type I events. We generated complete families of Type II ROC curves for strict, medium, and lax Type I criteria. Human GOC curves from Type II tasks are found to be in excellent agreement with their appropriate theoretical ROC curves.

A diotic amplitude discrimination experiment replicated 100 times

John Whitmore, Vit Drga, and Alan Taylor

The impact of large numbers of replications on the reduction of error in GOC analysis demonstrates the way in which substantive problems can be analysed. Individual and mean ROC curves are shown to be dramatically inferior to GOC curves for two observers. Both the locations and shapes of the GOC curves conform to the theoretical model for the discrimination task, whereas the ROC curves do not.

Theory of group operating characteristic (GOC) analysis

Vit Drga

Up to now, it has not been clear why a GOC curve should tend toward the appropriate theoretical ROC curve, nor has it been clear why a transform-averaged GOC curve should tend toward the theoretical ROC curve regardless of the particular transform chosen for calculation. A theory of GOC analysis will be presented which covers both transform-average GOC analysis and transfer-function analysis within the same theoretical framework. An ideal observer is modelled as a system consisting of a black box discriminator, a unique noise source, an additive unique and common noise mixer, followed by a transfer function onto a rating scale. The theory allows the unique noise distribution to vary as a function of common noise evidence value. The theory assumes strict stochastic ordering of the distributions of unique and common noise mixed together. It allows the specification of conditions under which GOC analysis will and will not work.

Transform-averaged group operating characteristic (GOC) analysis

Vit Drga

Transform-averaged GOC analysis is a new form of emperical ROC analysis based on the mean rating per stimulus across replications. Here, mean rating refers to a generalised, transform-averaged mean, which is calculated as follows: A strictly monotonic transform is applied to the set of ratings, the arithmetic mean (y) of the transformed ratings per stimulus is calculated, and the inverse transform is applied to y. This generalised GOC analysis can be interpreted in either of two equivalent ways: 1) as GOC analysis based on transform-averaged mean ratings, or 2) as GOC analysis based on arithmetic-mean ratings following a rescaling of the rating scale. Transform-averaged GOC curves from a simple frequency discrimination experiment will be presented along with their theoretical ROC curve. Each GOC curve is based on the same data set, but relies on a different choice of transform. Some transform-averaged GOC curves lie at least as close to the theoretical ROC curve as the arithmetic-mean GOC curve, implying there is nothing inherently special about the arithemetic-mean rating. GOC analysis based on the sum of ratings is seen to be a special case of transform-averaged GOC analysis.

Transfer-function analysis of rating scales in the theory of signal detectability

Vit Drga

In the context of signal detection and discrimination experiments, the transfer function is a one-to-one mapping between a decision axis variable and a rating scale. This concept is implicit in any theoretical interpretation of empirical ROC analysis. Given a pair of event-conditional theoretical distributions, a transfer function is estimated by pairing the theoretical criteria and empirical rating cutoffs that result in the same values of cumulative probability and cumulative proportion. Data from a frequency discrimination experiment will be used to illustrate the estimation of transfer functions. Given a transfer function, ratings can be converted into estimated decision axis pseudo-values. This allows estimation of the distribution of unique noise under the assumption that unique noise is additive with common noise on the decision axis.

The meaning of the area under the Receiver Operating Characteristic curve

Brian Scurfield

Many applications of the Theory of Signal Detectibility (TSD) use the parametric measure d' to determine observer sensitivity. This
measure is based on the assumption that the underlying distributions of the evidence variable conditional on signal-plus-noise (SN) and noise-alone (N) are Gaussian with equal variance. When this assumption is not met, there has been a belief expressed in the literature that TSD is not applicable. In truth, however, TSD provides a perfectly good non-parametric measure of observer sensitivity - namely the area, A, under the Receiver Operating Characteristic curve. To encourage researchers to use A, this paper reviews the meaning of A by relating it to the seperation of the SN and N probability functions. To overcome some problems to do with the scaling of A, another measure based on A is proposed. This measure quantifies the uncertainty, in the information theoretic sense, about whether the evidence conditional on SN will be greater than the evidence conditional on N. It is shown that A and the new measure are still appropriate sensitivity measures for tasks where the evidence is multidimensional (such as in 2IFC and Same-Different tasks) and where there are more than two events.

Measures of sensitivity in single-interval forced-choice and two-interval forced-choice tasks

Judi Lapsley, Brian Scurfield, Vit Drga, Susan Galvin, and John Whitmore

The relationship between the area under the ROC curve for the single-interval forced-choice (SIFC) task, A_SIFC, and the proportion correct in the two-interval forced-choice (2IFC) task, P(C)_2IFC, is well known. However A_SIFC=P(C)_2IFC has only been derived for the case of continuous probability functions. We have derived the relationship for discrete probability functions as well as relaxing a number of assumptions in the continuous case. To date, experimental research that tests the relationship has been equivocal, mainly because of observer inconsistency. Empirical results that are degraded by observer inconsistency cannot be used to justify theoretical relationships. We have overcome this by using a detection task with known, discrete, probability functions. By using known functions, the theoretical measures of sensitivity are also known. Observer inconsistency in our experimental data was removed by using group operating characteristic (GOC) analysis. The measures of sensitivity based on the GOC curves closely approximated the theoretical measures of sensitivity, whereas the measures based on mean ROC curves were poor approximations to the theoretical measures. The results indicated that A_SIFC=P(C)_2IFC empirically, once unique noise had been removed from the data. The implications of this result will be discussed.

The Acoustical Uncertainty Principle & WT

Judi Lapsley

Acoustic signals can be represented in both the time domain and the frequency domain. The Fourier transform is used to swap from one domain to the other. In modelling human hearing it is desirable to use input signals that are finite in both domains for they are more like naturally occurring sounds. However, it is theoretically impossible for a signal to have both a finite bandwidth (W) and a finite duration (T), for as resolution is increased in one domain, it is lost in the other. This trade-off is analogous to the Heisenberg uncertainty principle of quantum physics. Many mathematical models of human hearing assume that the bandwidth-duration product, WT, is the important parameter and not the actual bandwidth or duration. This means that a detector's performance would still be the same no matter how resolution in time, and in frequency, was traded-off. Due to physiological limitations of the human ear, this trade-off is likely to break down for very large, or very small, bandwidths and durations.

Last updated 08 Nov 2009 04:37 PM