Even if our interest is in the study of UFO's as some sort of extraordinary physical phenomenon (whether of natural or, possibly, of intelligent, extraterrestrial origin), our study cannot ignore the inescapable fact that nearly all of our evidence comes -- not from physical measuring instruments -- but from human observers. So far, however, we have consistently sold the human observer short indeed, in neglecting to make use of psychologically oriented techniques that would more fully enable observers to bring to bear their really rather remarkable powers of perception and recognition, we may have been forfeiting the opportunity of obtaining evidence from independent observers that would be sufficiently convergent and well-defined to clarify the true nature of the phenomena.

The Extent to Which the Apparent Unpredictability of UFO Phenomena Hinders their Scientific Study

The scientific investigation of a set of phenomena becomes possible whenever those phenomena exhibit some discernible degree of order or pattern. Scientific study is of course greatly facilitated when, as in astronomy, the order strongly emerges in the form of a space-time pattern of the very occurrences themselves. For, only then, can we arrange to have suitably trained observers suitably "training" powerful and, hence necessarily, highly directional recording and measuring instruments on the right place at the right time.

Indeed, by contrast with astronomical phenomena, those loosely classified together as "Unidentified Flying Objects' ("UFO's") or (with perhaps somewhat less commitment implied as to their real nature) "Unidentified Aerial Phenomena" sometimes appear almost inaccessible to scientific study. Certainly phenomena that are rare and fleeting are difficult enough but, if in addition they are totally capricious and unpredictable when they do occur, then the scientific method is to no avail and we are reduced to awaiting each new happening in the same primitive state of uncomprehending docility.

The repeated successes of science, however, have encouraged us always to search for pattern and order even when none at first appears. And, although a scientific study does of course become enormously more difficult when the occurrences of the phenomena do not fall into any predictable pattern in space and time, it remains a possibility so long as some regularity exists within the phenomena themselves -- whenever they do happen to occur. Thus, in the field of psychopathology, even if it were the case that some psychological phenomena (such, say, as psychotic episodes) occurred wholly unpredictably -- striking any person at any time, quite at random, we could still study the internal patterns of such episodes when they do strike. We might for example find that when symptom A appears, it is usually accompanied by symptom B, but seldom by symptom C, and so on. This, too, is a kind of predictability and can even lead to a degree of understanding and perhaps, eventually, to a method of treatment.

Similarly, in the case of UFO episodes, it may be possible to discover some regularities or patterns within these episodes even though any clear overall pattern in their mere occurrences (except, possibly for the tendency to unpredicted local concentrations in space and time) continues to elude us. This is not to say that efforts -- such as those of Michel (1958) and of Vallee and Vallee (1966) -- to detect some overall pattern should not be pursued, but only that the attempt at a scientific study need not await a positive outcome of those efforts.

The Principal Sense in Which the Problem of UFO Phenomena is a Psychological Problem

In the meantime, a scientific study of these phenomena is not impossible -- just more difficult. For, we are faced for the most part with a problem -- not of making physical measurements -- but of interpreting verbal reports. We are faced, in short, with a problem amenable more to the methods of the psychologist than to those of the physical scientist.

It is the principal purpose of this note to propose that, despite the relatively primitive state of development of psychological science, psychological and social scientists and even, indeed, law enforcement specialists have devised some techniques that could as well be applied to further the scientific study of UFO's.

I do not mean to suggest by this that most reports of UFO's can probably be shown to arise from purely psychological aberrations such as illusions, hallucinations, delusions, after-images, and the like. On the contrary, a careful examination of most of the best-documented cases has convinced me -- as at least one psychologist who has studied rather extensively into the fields of normal and psychopathological perception -- that very few such cases can be explained along these lines. Indeed, I have the impression that the claims that the UFO's reported even by seemingly responsible citizens represent lapses of a basically psychopathological character have generally come from people who have neglected to study closely either into the literature on psychopathology, or into that on UFO's, or (in many cases, I fear) both.

The Desirability of Separating Three Psychologically Extreme Types of UFO Cases

I have so far ignored the reports of the so-called "contactees" and related cultists who seem to form a relatively distinct class and who are generally readily identifiable without the benefit of extensive psychological training. Insofar as possible, I should also like to disregard the reports of out-and-out hoaxters. Admittedly, however, these present a somewhat more troublesome problem to which we must return later -- particularly in connection with photographs and other types of alleged physical evidence.

Of course there always are ambiguous cases which are difficult to place certainly within the triangle defined by the three, psychologically distinct "corners" representing the deluded contactee, the conscious pranksters, and the involuntary but responsible witness of some real but puzzling phenomenon. However, as has often been remarked, the existence of twilight does not deter us from distinguishing between night and day -- nor should it.

In fact, science generally proceeds most rapidly by focusing first on the purest and most clear-cut cases, and by leaving for later any "mop-up" operation of dealing with the remaining cases that are to varying degrees complicated, mixed, messy, borderline, or obscure. Thus the social scientist who is primarily interested in studying the formation and perpetuation of delusional belief systems will do well to focus precisely on the members of the "contactee" cults, the archetypal examples of which are situated in the south-western United States and are heavily constituted by persons (often predominantly women rather past middle age) who have relatively little formal education together with a history of professed beliefs in the mystical, the spiritual, or the occult. Likewise, a clinical psychologist interested in the ways in which socially responsible adult behavior emerges or fails to emerge out of the play and testing behavior of childhood may learn something from an intensive study of any of the almost canonical cases of adolescent boys who, often in pairs and in accordance with an almost tiringly regular pattern, attain at least transitory notoriety by submitting their photograph of a "flying saucer" -- replete with dome, antenna, and, perhaps, portholes and fins.

Just so, if we are, as here, primarily concerned with the possibility of unexplained but objective phenomena taking place within our atmosphere, then we should eschew not only the two pure sorts of cases just mentioned, but also the

various more-or-less obscure or ill-defined cases falling somewhere within the "triangle." (Indeed, to throw all such intermediate cases together, without adequate regard for the reliability or credibility of each report -- as some investigators have tended to do for the purposes of compiling over-all statistics concerning "UFO activity" -- can, I think, lead to a largely uninterpretable picture. For, there is then no way of assessing or parceling out the "noise" contributed by the contactee, the prankster and, of course, the many well-meaning citizens who, under unusual circumstances, will continue to misidentify familiar phenomena.) Rather, we stand to learn most from an intensive study into those numerous cases represented by the remaining "corner" of the triangle in which converging evidence from apparently involuntary, independent, and responsible witnesses strongly points to the occurrence of an objective phenomenon of an unexplained character.

The potential contribution of psychological techniques to the study even of purely physical phenomena

Even though our primary interest is, then, on the unexplained objective and presumably physical phenomena that may give rise to such UFO reports, our problem remains as much a psychological as a physical one. For, the vast bulk of the data upon which we must base our scientific investigation comes -- not from physical recording or measuring devices -- but solely from one or more human observers. Moreover these are observers who were not, evidently, selected for their powers of observation or description and who have good reason to be reluctant as well -- particularly in view of the likelihood of ridicule, often encouraged, curiously, by the very investigators who profess to be seeking the truth (cf., Fuller, 1966, pp. 211-220; Weitzel, 1967).

It is here, surely, that we have been most glaringly remiss in our attempts to put the investigation of UFO's on a scientific footing. We have, simply, failed to make anywhere near full use of the one recording and measuring instrument at our disposal; namely, our unwitting human witness.

Now it is true that one of the more exotic psychological techniques, hypnotic regression, has already been attempted with interesting -- if considerably less than conclusive results in at least one UFO case of a rather sensational nature (Fuller, 1967). However, although astonishing claims have sometimes been made for the kind of detail that can be recovered under hypnosis (e.g., by McCulloch in von Foerster, 1952, p. 100), the results of controlled experiments on accuracy of recall have generally been less impressive (Reiff & Scheerer, 1959). More reliable, in my opinion, are some techniques based on certain psychological facts of a more mundane but better understood character.

The power of methods guided by recognition rather than description in the reconstruction of a fleeting event

It is, I suppose, a fact familiar to us all that we can take in and remember much more information than we can readily communicate to others. Contrast, for example, how easily we recognize the face of a friend in a crowd with how difficult it is to describe that face so that any other person could then do it for us. Quite generally, our powers of recognition exceed our powers of description (and, indeed, surpass anything that we have yet been able to accomplish by physical instrument or machine). In an experiment on recognition memory, I once presented human subjects with over 600 different pictures, one right after the other, and then found that they could immediately distinguish between, those "old" pictures and otherwise completely comparable "new" pictures with median accuracy of over 98% (Shepard, 1967). Even when the test was not given until a week later, the discriminations between "old" and "new" pictures were still 92% correct. Moreover, the advantage of recognition over verbal description, should become especially pronounced when the object or event to be remember is unfamiliar and, so, not uniquely or succinctly "captured" by readily available terms or labels.

Over and over again, witnesses of "UFO's" have provided descriptions that, while they strongly suggest that a clear view was obtained of some well-defined but extraordinary object or phenomenon, leave the investigator frustratingly in the dark as to its precise appearance or behavior. A closely viewed, "spinning metallic" object is said, for example, to have been "mushroom-shaped," or to have resembled "an inverted top." But what does this mean? What sort of mushroom? With or without the stalk? And what on earth (!) is referred to, precisely, by "a 30 foot inverted top?"

Some psychologists have been expressly studying the ways in which people come to describe nearly nondescript objects to others (e.g., Krauss & Weinheimer, 1964, 1966). Often a person will feel that the ambiguous term he comes up with (such, for example, as "an inverted top") does quite well. Possibly this is because he is picturing some particular interpretation (e.g. a particular toy that he played with as a child). For the listener who does not have that particular picture in mind, however, the description may prove either meaningless or, worse, completely misleading (cf., Glucksberg, Krauss, & Weisberg, 1966). An indication of the same sort of problem is the tendency of witnesses to say things like "it looked about the size of a football." Further circumstances make clear that they must have been referring to its apparent visual size rather than its real, physical size (which could, after all, hardly be estimated without also knowing its real, physical distance). More pertinently here, it appears that they were really talking more about its shape than its size. Possibly, the presence, so to speak, of a very vivid image in the mind of the witness causes him to lose sight of the total inadequacy of his verbal encoding of that image.

This problem is already implicitly recognized in certain situations of more obviously pressing practical concern. Investigators in cases of homicide do not rest content with the weak and fuzzy descriptions typically offered by a witness but, in addition, may employ a skilled artist (such as Richard Kenehan of the New York Police Department) to work with the witness in an attempt to reconstruct a usable likeness of the murderer's face. The witness may be asked to select eyes, eyebrows, nose, mouth, or ears from series that are systematically graded in size and shape. The witness can then help to adjust their positions on an outlined face. This will generally provide a sufficiently concrete stimulus to enable the witness, finally, to become reasonably explicit about further refinements concerning hair, complexions, lines, scars, asymmetries, and so on. In a number of cases (such as the recent one in which Richard Speck was charged with the murder of several nursing students in Chicago) a likeness constructed in this way from a single surviving witness has proved remarkably accurate and, in more than one instance, has actually led to the apprehension of the criminal (cf., Schumach, 1958).

This provides perhaps the most directly pertinent substantiation for the one central point that I want to leave with those concerned with the investigation of UFO's. Briefly, it is this: Even when an event occurs without warning, leaves little time for careful observation and, indeed, occasions extreme fear or anxiety, the average witness often retains an accurate, almost photographic record of the event -- a record, moreover, that can be largely recovered from him even though he lacks the words to describe it himself. Possibly then, in allowing our investigations to depend solely upon our informant's inadequate, his misleading and, yes, his sometimes even ludicrous choice of words, we have done both him -- and ourselves -- a telling disservice.

The desirability of establishing a system permitting convergence of independent reconstructions

Admittedly, in the case of UFO's, the value of information provided by a single, isolated witness -- however detailed that information may be -- is, by itself, always quite small (except, of course, for the witness himself!). For, from the standpoint of any other person, there is always at least the possibility of hallucination, delusion or, more likely, just plain fabrication. This is amply pointed up by the relative lack of evidential value of the many quite detailed photographs purported to have been taken of UFOs by solitary witnesses.

It is only when there turns up an otherwise inexplicably close correspondence between the information furnished by two or more independent witnesses that the evidence becomes at all compelling at the public or scientific level. But we do not provide for even the possibility of a close correspondence unless we elicit sufficiently detailed information. Thus, when one person reports a "glowing mushroom-shaped object" while another, remote witness refers to the passage of an "inverted top," we have little basis for evaluating the likelihood that they have both observed a physical object -- let alone the same physical object.

Now it is true that certain rather suggestive regularities have already emerged in the more or less spontaneous reports of observers. Among the very most common, for example, are the frequent references to disk-like shapes, to extraordinary velocities, to abrupt simultaneous changes of color and direction and, perhaps most strikingly, to the so-called "pendulum" or "falling-leaf" type

of descent. Still, much more detailed information concerning such things as shape could presumably be extracted by techniques (akin to those already used in criminal cases) designed to take fuller advantage of the witnesses' usually untapped but vastly more discriminating powers of recognition.

The point, here, is that such more detailed information is needed not merely for its own sake. It is needed, even before that, because the establishment of the very validity of the information in question hinges upon the demonstration of the kind of point-for-point correspondence between reports that becomes possible only when those reports are sufficiently detailed. If two, unrelated witnesses both claim to have seen a disk-shaped object at about the same time and place this is not sufficiently compelling. (Evidently! For it has already happened many times.) But, if artists working with the two witnesses, independently, construct pictures of what appears to be the very same object or, alternatively, if the two witnesses independently point to the very same drawing or photograph in an array of 50 or more different pictures of such objects, then the coincidence becomes more interesting. (And, of course, if the pictures reconstructed or singled out in this way just once turned out to coincide, also, with an actual photograph taken at the time, we should at last have opened the door for the more precise measurements of physical science-including the sophisticated and powerful photogrammetric methods being developed for the analysis and interpretation of lunar photographs.)

The establishment of a pre-tested and standardized procedure for reconstructing information by the sort of psychologically oriented techniques envisaged here, moreover, would be incomparably cheaper than the implementation of other more physically oriented schemes that have sometimes been proposed -- such as the construction of a far-flung network of automatic radar-and-camera stations. For, instead of having simultaneously to cover all possible sites in advance, we could simply move in to recover the desired information after an incident is first reported.

There is, however, one unavoidable aspect of the psychologically oriented type of approach proposed here that I, anyway, regard as quite regrettable. To the extent that any detailed pictures reconstructed by these techniques are made publicly available, we cannot guarantee that pictures obtained from subsequent witnesses will be suitably independent for our purposes. Consequently, rather tight security precautions would have to be imposed on the more detailed reconstructions, if the purely scientific purposes of the investigation are not to be compromised.

The use of concrete stimuli to provide a basis for the independent, recognition-guided process of reconstruction

The need for ensuring independence of information supplied by different witnesses is in fact so great that it is doubtful whether much reliance could safely be placed on different pictures reconstructed with the help of the same artist. Despite the best intentions of the artist, he might unwittingly guide different witnesses along somewhat similar channels by means of subtle, perhaps unconscious cues (cf., Rosenthal, 1966). Moreover, even if a different artist could be supplied for each witness, we would still be left with the problem of evaluating the likelihood that any two pictures constructed in this way could have turned out as similarly as they did by chance alone.

For these reasons and for reasons of feasibility, convenience, and economy, it would be preferable to develop a standardized set of materials containing suitably representative and graded series of shapes to which each witness could independently respond. Some such materials would be needed, in any case, in order to provide stimuli suitable for tapping the witnesses' powers of recognition. Possibly even separate arrays should be constructed for distinguishable parts such as "domes" or other projections (just as separate series of eyes, or mouths may be helpful to the witness in criminal cases). By means of suitable standardization and control in the preparation and presentation of such materials, then, we could be reasonably sure that the responses of one witness are not unduly influencing the choices of another. Moreover, since each witness would make his choices from a pre-tested, fixed set of alternatives of known size, we would be in a favorable position to assess the probability that any coincidences of choice might have occurred merely by chance.

All things considered, the best procedure might be to divide the questioning of a witness into the following three distinct phases: first, the recording of the

witness describing what he saw as completely as he can, in his own words, and without any cues (whether verbal or pictorial) that might bias him in one direction or another; second, the recording of his responses to the standardized, pictorial materials; and third (if the case seems to warrant it) the foil reconstruction of a new picture with the help of a suitably trained artist. Such a new picture, if sufficiently novel or well-defined, might then be incorporated in future revisions of the materials used for the second phase of the interview.

The effectiveness of the proposed procedure would depend very heavily upon the amount of thought, care, skill and, above all, pretesting that went into the preparation of the materials. The arrays of alternative shapes should of course include all types of shapes that have been clearly described, sketched, or (allegedly) actually photographed by some previous witness of at least reasonable reliability. One helpful attempt at systematizing the kinds of shapes that have been reported has in fact already been published (see Hall, 1964, p. 144). However, more extensive and refined work would be necessary in order to cover the great variety of reported shapes, and to do this in a sufficiently concrete and realistic manner to promote recognition and, possibly, further specification by witnesses.

The Use of Photographs of Alleged UFOs as a Source of Concrete Test Materials

Since photographs represent an especially tempting vehicle for the hoaxter and, in addition, are easily faked, they are individually of little value as evidence -- except in the rare cases in which there were independent, corroborating eyewitnesses. Photographs purporting to be of UFOs are, however, surprisingly numerous. (I myself have assembled well over 150 distinct such photographs merely from published reports.) Moreover, since at least one of these photographs might be authentic and since we have no sure way of knowing in advance which one it might be, we can not afford to eliminate any distinct type that hasn't already been proved to be a fraud. In the meantime, moreover, the other, spurious photographs can serve (in somewhat the same way as the non-suspects in a police "line-up") as a means of assessing consistency of choice or, contrary-wise, mere guessing on the part of different witnesses.

The accompanying figure reproduces drawings in which I have tried to portray, as accurately as I could, representative objects from 63 of these photographs. The greater contrast of the drawings renders them more readily duplicatable than the original photographs. Moreover, I was also able in this way to reduce them all to uniform size and to eliminate background details which, although useful for estimating size or gauging authenticity, are only distracting for present purposes. Some of the photographs are from well documented or widely publicized cases while others are of more obscure or dubious origin. At least two cases have subsequently been admitted to be hoaxes, while circumstances surrounding some of the others make them difficult to dismiss in this way.

Nevertheless, the single most striking tiling about these pictures -- far from [there] being any general uniformity in their appearance -- is their largely irreconcilable diversity. Whether or not this diversity is interpreted as detracting from their value as evidence, it surely cannot be taken as contributing to that value. It does, however, serve our immediate, rather different purpose of providing an initial sample from which to extrapolate and interpolate an eventual graded array of the sort that we seek for purposes of testing witnesses.

Some Incidental Comments on the Status of Photographs of Alleged UFOs as a Source of Evidence Themselves

Before leaving the photographs themselves, however, it should be noted that there are a few instances of rather suggestive similarities between these photographs. A frequently cited case is the striking resemblance between E6, reportedly taken by a farmer, Paul Trent, as he was returning home with his wife near McMinville, Oregon in June 1950, and ET, allegedly taken by a French military pilot near Rouen, France in March 1954.

Another example, involving several different sightings, is shown in the upper right. G1, G2, and. G3 present three successive views of the same object purportedly taken by a photographer, Ed Keffel, of the Brazilian publication "O Cruziero" while he was accompanied by a journalist, Joao Martins, near Rio de Janeiro in May, 1952. The edge-on view, G1, is almost indistinguishable from another photograph, represented in F1, allegedly taken from an Argentine pursuit plane in late 1954. The "top" view, G2, seemingly resembles F2 (the left edge of which was cut off by the boundary of the original photograph), which a 15-year-old boy, Michael Savage, claims to have taken near San Bernardino, California in July, 1956. It also somewhat resembles the lighted object, F3, allegedly appearing in a color photograph taken by Joseph Sigel near Waikiki, Hawaii in June, 1959. And, finally, the "bottom" view, G3, presents the same general sort of configuration as that shown in G4, which is based upon a photograph purportedly taken by Yukuse Matsumura outside his residence in Yokohama, Japan in January 1957 (although the relative dimensions of the features appear slightly different in these last two photographs).

There are several other instances in which photographs taken by apparently unrelated individuals might be of the same object. Another view (not included in the figure) showing more of the "bottom" of the object displayed in E4 resembles the object shown in G5 and, even more closely, an object apparently hovering over a seaplane in still another photograph (also not included) of unknown origin. In a number of instances (e.g., F5, 6, and 7 or D6, 7, and 8) the degree of correspondence is more difficult to assess owing to the relatively poorer definition of the images.

Of course even very close similarities do not in themselves guarantee authenticity. Consider, for example, C2 and C3 which are strikingly similar despite the fact that the object in C2 appears over a mountain near Riverside, California in the original photograph reportedly taken by a 21-year-old man and two friends in 1951, whereas the object in C3 appears over a flock of grazing sheep in the photograph submitted by an Australian rancher in 1954. But, since the object in C2 has subsequently been admitted to be none other than a 1967 Ford hubcap, the object in C3 is presumably the same. (Another photograph later confessed to be fraudulent is represented in B6, and somewhat suspicious circumstances also surround several other photographs, including those represented in A9, B1, B8, B9, C1, and G8.)

Perhaps the safest attitude to adopt is that recommended by the National Investigations Committee on Aerial Phenomena (cf., Hall, 1964, p. 86); namely,

that the evidential value at least of still photographs depends entirely upon the surrounding circumstances. An isolated photograph about which little is known, no matter how impressive it may appear in itself, is essentially worthless -- except, possibly, in cases in which sophisticated photogrammetric analysis yields further detail tending to confirm the verbal account of the photographer. (I have heard that this has happened in at least one case; viz., that of the controversial series of Polaroid photographs -- one of which is represented in A8 -- taken by a California state highway employee, Rex Heflin, near Santa Ana in August 1965.) The photographs most worthy of further intensive investigation would seem to be those for which there also were reported to be many eyewitnesses as well as other, corroborating photographs -- as in the celebrated case of the "saturn-shaped" object (A1) that was said to have been photographed from an oceanographic vessel near the Brazilian island of Trindade in January 1958 (see Lorenzen, 1966, pp. 145-153, 164-174).

The Assessment of the Representativeness of a Set of Recognition Test-materials.

Even though an extensive effort is made to represent every sort of shape that has been reliably described, sketched, or photographed, the possibility will remain that the collection of proposed test materials will not be sufficiently representative. Certain types of completely fraudulent shapes may unnecessarily inflate the already unwieldy collection and, more seriously, some significant types may still be missing. There are, however, ways of assessing the representativeness of any proposed collection of shapes. One is, simply, to have people describe these shapes and then to look for any pronounced departures of the relative frequencies of the various descriptive terms used from the corresponding relative frequencies in reports issuing from actual sightings of UFOs. My research assistant, Miss Shelley Meltzer, carried out an exploratory attempt at this sort of thing that may help to illustrate some of the relevant considerations.

From our total sample of photographs, 75 that seemed suitably representative were selected for this preliminary study. These included most of the 63 already portrayed in the accompanying figure, but those that were known or strongly suspected to be fraudulent were eliminated and a number of others of less sharply defined shape were added (since many reports indicate that the shape was not clearly visible). Each of 19 subjects, mostly students at Harvard University, then looked through one of three subsets of 25 of these photographs and, for each, attempted to describe the pictured object in their own words. (Immediately following that, each subject then looked through another subset of 25 and, this time, indicated the appropriateness for each photograph of each term in a fixed set that we had listed in advance on a standardized rating sheet. However this part of the experiment will not be considered in any detail here.) Of most immediate interest are the descriptive labels spontaneously produced in the 19 x 25 or 475 subject-photograph encounters.

These could now be compared with the descriptive labels appearing in a sample of 206 different representative reports of actual UFO sightings that Miss Meltzer had already extracted (for a different purpose) from a number of sources (mostly Edwards, 1966; Hall, 1964; Michel, 1967; Olsen, 1966; Ruppelt, 1956; and Vallée, 1965). The accompanying table lists those descriptive terms that pertain to visual appearance but, for purposes of comparison with the mostly black-and-white photographs, excludes the many references to (chromatic) color. With one exception (#33), only terms that appeared at least twice in the sample of 206 actual reports are included, and these arranged in order of decreasing frequency of occurrence in that sample.

The two columns of numbers, then, present the resulting frequencies of occurrence (a) in the 206 actual UFO reports and (b) in the 475 opportunities for these same descriptive terms to arise in the experiment with the photographs. Direct numerical comparisons are somewhat hazardous owing to the different circumstances in which the two sets of descriptive terms arose. In terms merely of opportunities, the numbers in the second column should be about twice as large as those in the first. However, the totals for the two columns are nearly equal and, so, the real encounters evidently were relatively more productive of descriptive terms on the average. Numerically small departures or departures in which the second number is somewhere between the size of the first number and twice that size are probably not very significant therefore.

The remaining positive and negative discrepancies of appreciable size are indicated by the plus and minus signs in the right-most column. Some of these are probably explainable in terms of the two-dimensional, achromatic, and stationary character of the photographs (e.g., #10 & 13), or in terms of differences in vocabulary to be expected between the unselected witnesses and the college-educated subjects of the experiment (e.g., #9 & 15). Other discrepancies, however, suggest either that some shapes, such as the so-called "cigar" (#7), were not adequately represented in the sample of photographs, or that some shapes, such as those most frequently said to resemble a "hat" (#33), are especially likely to have been of fraudulent origin. (Among the objects included in the above figure that were often said to be hat-like are C3, which we already noted is almost certainly a 1937 Ford hubcap; E4, which doesn't seem to fit very well with the usual descriptions of UFOs; and G2, which, although it too doesn't coincide with at least my notion of a "flying saucer," does however correspond rather closely with several other photographs.)

The Use of the Computer in the Design of Recognition Test-Materials and in the Analysis of Results

Our sample of only 206 actual UFO cases is really too small and haphazard for the purpose of ensuring that all types of reported shapes are adequately represented in any proposed recognition array. Many descriptive terms that have repeatedly been used (such as "doughnut," "ring," "mushroom," flattened ball," "double-convex lens," "bullet," "blimp," and "submarine") didn't happen to appear more than once in our particular sample). Ideally, for this work, one would like access to a centralized library of all reasonably documented cases -- suitably coded for retrieval via computer. Indeed, at present the scientific study of the UFO problem is greatly hampered by the circumstance that the thousands of reported sightings have not been adequately coded or systematized in any uniform way and are, in fact, still scattered among such diverse and often mutually hostile organizations as the U.S. Air Force, NICAP, APRO, and the University of Colorado Project -- not to mention a number of more-or-less private files assembled by individual investigators both here and abroad.

Recent developments in computer technology -- particularly in computer graphics -- could be utilized, also, in the construction of arrays of shapes for a recognition test. Thus for any specified shape, the computer (together with suitable graphical output equipment) could automatically generate alternative pictures of the same object as viewed from any desired angle (e.g., Noll, 1965; Zajac, 1964); and could even generate other test shapes intermediate between that shape and some other specified shape. (In fact, as on-line graphical facilities become more widely available, even a relatively unartistic witness, seated in front of a suitable display device, should be able to reconstruct his own object by techniques of these general sorts.)

For the present, however, perhaps the most promising use of the computer in this connection would be in finding an optimum arrangement of the alternative test shapes in the recognition array. This is a matter of real concern owing to the large number of shapes that should be included. (Even the 63 exhibited in the above figure fall far short of covering all the varieties that have been sketched or described.) If the alternatives could somehow be arranged so that similar shapes are close together, then the witness could quickly narrow down to the most relevant region of the array in order to make his final, most refined discriminations.

In order to do this we would first need to obtain some measure of the perceived similarity between any two shapes. One could of course obtain a direct, subjective judgment of similarity from experimental subjects. However, it might be more convenient to obtain a derived measure of similarity based upon the frequency with which different subjects will sort the two shapes into the same pile, or upon the overlap in their application of the same descriptive terms to the two shapes in the kind of task described in the preceding section (cf„ Rosenberg, Nelson, & Vivekananthan, in press).

Once we have any such measure of similarity for every pair, we can apply powerful new computer-based methods for mapping the objects into a two-dimensional arrangement in such a way that their similarities are preserved, in so far as possible, in the spatial proximities among them (Kruskal, 1964 Shepard, 1962; Shepard & Carroll, 1966). Moreover, these same methods could yield a quantitative metric of similarity that would then enable us to specify just how similar an object identified by one witness is to the object identified by another witness. Indeed, they could even tell us something about the basic dimensions along which UFO phenomena differ or, with the help of recently perfected methods for "hierarchical clustering" (Johnson, 1967), they could provide an indication of the basically different classes into which these phenomena undoubtedly fall.

Possibly, some of these classes of unidentified aerial phenomena will turn out to be of purely natural origin. I once even ventured to suggest this for certain puzzling types of cases myself (Shepard, 1967b) -- though, admittedly, attempts to develop such explanations in terms of known principles of atmospheric physics, generally, have run into competent and serious criticism (McDonald, 1968). Still. even if some of the phenomena are of natural origin, a more complete and accurate characterization of their appearance and behavior should be of some interest to the physical scientist -- indeed, all the more so to the extent that they appear to conflict with known physical principles.

In any case, it appears that techniques now exist that could provide the basis for a psychologically oriented, but genuinely scientific investigation into unidentified aerial phenomena, whatever their nature may ultimately prove to be.

The development of some of the techniques described here and the preliminary experimental tests of these techniques on UFO materials were carried out as one part of a more general project on psychological scaling and data analysis supported by Grant No. GS-1302 from the National Science Foundation to Harvard University. The author is indebted to the Foundation for its support and, also, to Miss Shelley Meltzer for her extensive assistance in the project

Edwards, F. Flying saucers-serious business. New York: Lyle Stuart, 1966, (Also Bantam paperback S3378).

Fuller, J. G. Incident at Exeter. New York: G. P. Putnam's Sons, 1966, (Also Berkeley Medallion Paperback, TM 757,375).

Fuller, J. G. The interrupted journey. New York: Dial Press, 1967, (Also Dell Paperback 4068).

Glucksberg, S., Krauss, R. M., & Weisberg, R. Referential communication in nursery school children: Method and some preliminary findings. Journal of Experimental Child Psychology, 1966, 3, 333-342.

Hall, R. H. (ED.) The UFO evidence. National Investigations Committee on Aerial Phenomena, 1530 Connecticut Ave., N.W., Washington, D.C., 1964.

Johnson, S. C. Hierarchical clustering schemes. Psychometrika, 1967, 32, 241-254.

Krauss, R. M., & Weinheimer, C. Changes in reference phrases as a function of frequency of usage in social interaction: a preliminary study. Psychonomic Science, 1964, 1, 113-114.

Krauss, R. M., & Weinheimer, S. Concurrent feedback, confirmation, and the encoding of referents in verbal communication. Journal of Personality and Social Psychology, 1966, 4, 343-346.

Kruskal, J. B. Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis. Psychometrika, 1964, 29, 1-27.

Lorenzen, C. B. Flying Saucers: The Startling Evidence of the Invasion from Outer Space. New York: Signet, 1966, (Signet Paperback T3058).

McDonald, J. E. UFOs -- atmospheric or extraterrestrial? Talk presented to the Chicago Chapter of the American Meteorological Society, May 31, 1968. (See, also, his contribution to this volume.)

Michel, A. The truth about flying saucers. New York: S. G. Phillips, 1956. (Also Pyramid Paperback T-1647)

Michel, A. Flying saucers and the straight-line mystery. New York: Criterion Books, 1958.

Noll, A. M. Stereographic projections by digital computer. Computers and Automation, 1965, 14, No. 5.

Olsen, T. M. (Ed.) The reference for outstanding UFO sighting reports. UFOIRC, Inc., Dept SM 518, P.O. Box 57, Riderwood, Maryland, 21139, 1966.

Reiff, R., and Scheerer, M. Memory and hypnotic age regression. New York: International Univ. Press, 1959.

Rosenberg, S., Nelson, C., and Vivekananthan, P. W. A multidimensional approach to the structure of personality impressions. Journal of Personality and Social Psychology, 1968 (in press).

Rosenthal, R. Experimenter effects in behavioral research. New York: Meredith Publishing Co., 1966.

Ruppelt, E.J. The report on unidentified flying objects. New York: Doubleday, 1956. (Also Ace Books paperback G-537)

Schumach, M. Palette-packing cop. New York Times Magazine, August 24, 1958.

Shepard, R. N. The analysis of proximities: Multidimensional scaling with an unknown distance function. I. Psychometrika, 1962, 27, 125-140. II. Psychometrika, 1962, 27, 219-246.

Shepard, R. N. Recognition memory for words, sentences and pictures. Journal of Verbal Learning and Verbal Behavior, 1967, 6, 156-163. (a)

Shepard, R. N. Tornadoes: Puzzling phenomena and photographs. (Letter to Editor). Science, 1967, 155, 27-28. (b)

Shepard, R. N., and Carroll, J. D. Parametric representation of nonlinear data structures. In P. R. Krishnaiah (Ed.), Multivariate analysis: Proceedings of an international symposium. New York: Academic Press, 1966. pp. 561-592.

Vallée, J. Anatomy of a Phenomenon. New York: Ace Books, 1965 (Paperback).

Vallée, J., and Vallée, J. Challenge to Science: The UFO Enigma. Chicago: Regnery, 1966.

Von Foerster, H. (Ed.) Cybernetics: Transactions of the eighth conference. New York: Josiah Macy, Jr. Foundation, 1952.

Weitzel, W. ... Into the Middle of Hell. Flying saucers: UFO reports #3. New York: Dell Publishing Co., 1967. pp. 38-49.

Zajac, E. E. Programmed pictorial displays. Proceedings of the 1964 symposium on digital computing. Bell Telephone Laboratories, 1964. pp. 33-44.