The Psychophysics Symposium 1993
The Psychophysics Symposium was held in August 1993 as part of the New Zealand Psychological Society's Annual Conference. Papers were presented by John Whitmore, Vit Drga, Brian Scurfield, and Judi Lapsley. The following links are abstracts for these papers; more details are available from the authors.
Abstracts
Why group operating characteristic (GOC) analysis?
John Whitmore
It is understood widely that experimental technique and research design
reduce data varaibility or error, and that statistics measures the error which
remains. The remaining error almost always confounds the comparison of data to
theory. In psychoacoustics, GOC analysis statisically reduces data error
further, providing a more precise and powerful way of testing theories of
hearing.
Group operating characteristic (GOC) analysis of Type II decisions
John Whitmore and Susan Galvin
A few years ago, one of us (Susan Galvin) presented to the NZPS a theoretical
analysis of Type I and Type II detection tasks in the theory of signal
detectability. Type I tasks are concerned with how observers distinguish between
environmental events, and Type II tasks are concerned with how observers
distinguish between their own correct and incorrect decisions about those Type I
events. We generated complete families of Type II ROC curves for strict, medium,
and lax Type I criteria. Human GOC curves from Type II tasks are found to be in
excellent agreement with their appropriate theoretical ROC curves.
A diotic amplitude discrimination experiment replicated 100 times
John Whitmore, Vit Drga, and Alan Taylor
The impact of large numbers of replications on the reduction of error in GOC
analysis demonstrates the way in which substantive problems can be analysed.
Individual and mean ROC curves are shown to be dramatically inferior to GOC
curves for two observers. Both the locations and shapes of the GOC curves
conform to the theoretical model for the discrimination task, whereas the ROC
curves do not.
Theory of group operating characteristic (GOC) analysis
Vit Drga
Up to now, it has not been clear why a GOC curve should tend toward the appropriate theoretical ROC curve, nor has it been clear why a transform-averaged GOC curve should tend toward the theoretical ROC curve regardless of the particular transform chosen for calculation. A theory of GOC analysis will be presented which covers both transform-average GOC analysis and transfer-function analysis within the same theoretical framework. An ideal observer is modelled as a system consisting of a black box discriminator, a unique noise source, an additive unique and common noise mixer, followed by a transfer function onto a rating scale. The theory allows the unique noise distribution to vary as a function of common noise evidence value. The theory assumes strict stochastic ordering of the distributions of unique and common noise mixed together. It allows the specification of conditions under which GOC analysis will and will not work.
Transform-averaged group operating characteristic (GOC) analysis
Vit Drga
Transform-averaged GOC analysis is a new form of emperical ROC analysis based
on the mean rating per stimulus across replications. Here, mean rating refers to
a generalised, transform-averaged mean, which is calculated as follows: A
strictly monotonic transform is applied to the set of ratings, the arithmetic
mean (y) of the transformed ratings per stimulus is calculated, and the
inverse transform is applied to y. This generalised GOC analysis can be
interpreted in either of two equivalent ways: 1) as GOC analysis based on
transform-averaged mean ratings, or 2) as GOC analysis based on arithmetic-mean
ratings following a rescaling of the rating scale. Transform-averaged GOC curves
from a simple frequency discrimination experiment will be presented along with
their theoretical ROC curve. Each GOC curve is based on the same data set, but
relies on a different choice of transform. Some transform-averaged GOC curves
lie at least as close to the theoretical ROC curve as the arithmetic-mean GOC
curve, implying there is nothing inherently special about the arithemetic-mean
rating. GOC analysis based on the sum of ratings is seen to be a special case of
transform-averaged GOC analysis.
Transfer-function analysis of rating scales in the theory of signal detectability
Vit Drga
In the context of signal detection and discrimination experiments, the
transfer function is a one-to-one mapping between a decision axis variable and a
rating scale. This concept is implicit in any theoretical interpretation of
empirical ROC analysis. Given a pair of event-conditional theoretical
distributions, a transfer function is estimated by pairing the theoretical
criteria and empirical rating cutoffs that result in the same values of
cumulative probability and cumulative proportion. Data from a frequency
discrimination experiment will be used to illustrate the estimation of transfer
functions. Given a transfer function, ratings can be converted into estimated
decision axis pseudo-values. This allows estimation of the distribution of
unique noise under the assumption that unique noise is additive with common
noise on the decision axis.
The meaning of the area under the Receiver Operating Characteristic curve
Brian Scurfield
Many applications of the Theory of Signal Detectibility (TSD) use the
parametric measure d' to determine observer sensitivity. This
measure is based on the assumption that the underlying distributions of the
evidence variable conditional on signal-plus-noise (SN) and noise-alone
(N) are Gaussian with equal variance. When this assumption is not met,
there has been a belief expressed in the literature that TSD is not applicable.
In truth, however, TSD provides a perfectly good non-parametric measure of
observer sensitivity - namely the area, A, under the Receiver Operating
Characteristic curve. To encourage researchers to use A, this paper
reviews the meaning of A by relating it to the seperation of the SN
and N probability functions. To overcome some problems to do with the
scaling of A, another measure based on A is proposed. This
measure quantifies the uncertainty, in the information theoretic sense, about
whether the evidence conditional on SN will be greater than the
evidence conditional on N. It is shown that A and the new
measure are still appropriate sensitivity measures for tasks where the evidence
is multidimensional (such as in 2IFC and Same-Different tasks) and where there
are more than two events.
Measures of sensitivity in single-interval forced-choice and two-interval forced-choice tasks
Judi Lapsley, Brian Scurfield, Vit Drga, Susan Galvin, and John Whitmore
The relationship between the area under the ROC curve for the single-interval
forced-choice (SIFC) task, ASIFC, and the proportion
correct in the two-interval forced-choice (2IFC) task, P(C)2IFC,
is well known. However ASIFC=P(C)2IFC
has only been derived for the case of continuous probability functions. We have
derived the relationship for discrete probability functions as well as relaxing
a number of assumptions in the continuous case. To date, experimental research
that tests the relationship has been equivocal, mainly because of observer
inconsistency. Empirical results that are degraded by observer inconsistency
cannot be used to justify theoretical relationships. We have overcome this by
using a detection task with known, discrete, probability functions. By using
known functions, the theoretical measures of sensitivity are also known.
Observer inconsistency in our experimental data was removed by using group
operating characteristic (GOC) analysis. The measures of sensitivity based on
the GOC curves closely approximated the theoretical measures of sensitivity,
whereas the measures based on mean ROC curves were poor approximations to the
theoretical measures. The results indicated that ASIFC=P(C)2IFC
empirically, once unique noise had been removed from the data. The implications
of this result will be discussed.
The Acoustical Uncertainty Principle & WT
Judi Lapsley
Acoustic signals can be represented in both the time domain and the frequency domain. The Fourier transform is used to swap from one domain to the other. In modelling human hearing it is desirable to use input signals that are finite in both domains for they are more like naturally occurring sounds. However, it is theoretically impossible for a signal to have both a finite bandwidth (W) and a finite duration (T), for as resolution is increased in one domain, it is lost in the other. This trade-off is analogous to the Heisenberg uncertainty principle of quantum physics. Many mathematical models of human hearing assume that the bandwidth-duration product, WT, is the important parameter and not the actual bandwidth or duration. This means that a detector's performance would still be the same no matter how resolution in time, and in frequency, was traded-off. Due to physiological limitations of the human ear, this trade-off is likely to break down for very large, or very small, bandwidths and durations.
Last updated 08 Nov 2009