Anticipatory eye-movements in initially ambiguous sentences: There's more to it than meets the eye

Pia Knöferle(1), Matthew W. Crocker(1), Christoph Scheepers(1) and Martin Pickering(2)
University of the Saarland(1)
University of Edinburgh(2)

The monitoring of eye-movements in visual scenes during auditory sentence comprehension has shown rapid combination of visual and linguistic information (Tanenhaus et al., 1995). Indeed, recent studies of unambiguous sentences have even shown that subjects make anticipatory eye-movements to likely potential referents in the scene (Altmann & Kamide, 1999; Kamide et al. 2002). In the present studies, we investigate what such anticipatory eye-movements can reveal about the preferred interpretation of initially ambiguous sentences. Specifically, we examine the interaction of the purely structural SVO preference in German, visual scene constraints, and lexical (verb) information.

German, with its relatively flexible word order permits an ambiguously case-marked sentence-initial NP to be interpreted as the Subject or as the Object of a sentence. Despite the possibility of an Object-first reading, there is a strong preference to interpret the first NP as the Subject of the sentence (Hemforth, 1993), i.e. (1a) is preferred over (1b):

(1a) Die Krankenschwester schubst in diesem Moment den Sportler. The nurse (NOM, amb.) pushes at that moment the sportsman (ACC).

(1b) Die Krankenschwester fönt in diesem Moment der Priester. The nurse (ACC, amb.) blow-drys at that moment the priest (NOM).

28 subjects were simultaneously presentend with sentence conditions as above and an image showing a nurse, a sportsman and a priest carrying out the actions described by the sentences. We assume that a Subject typically corresponds to an Agent, and an Object to a Patient in the scene. In all sentences the initial noun-phrase is ambiguous (feminine) as Subject (NOM) or Object (ACC), and in all accompanying scenes the corresponding referent is both Agent and Patient. The other two referents are unambiguously masculine, one being an Agent and one being a Patient. Thus images had the following form:

SPORTSMAN <-pushing- NURSE <-blow-drying- PRIEST
(Patient) (Patient/Agent)
(Agent)

We predict that the SVO preference will trigger anticipatory eye-movements to the Patient (the sportsman) after the initial NP is processed for both conditions (1a, SVO & 1b, OVS), assuming 'the nurse' is understood as Subject/Agent. In addition, if the verb, combined with visual information, disambiguates towards the OVS interpretation (1b), we predict that participants will revise their interpretation, causing anticipatory eye-movements to the structurally disfavoured Subject/Agent (the priest) before the onset of the second NP. An analysis of gaze proportions confirmed both of these predictions were borne out and significant.

A second study used similar materials but varied the word order such that the main verb appeared at the end of the sentence, further delaying the availability of verb information. An analysis of eye-movements confirmed that the initial NP is understood as the Subject/Agent, triggering anticipatory eye-movements to the Object/Patient. These results show that anticipatory eye-movements can reveal preferred interpretations for ambiguous sentences, and that expectations can be
driven by structural as well as lexical information.

References

Hemforth, B. (1993). Kognitives Parsing: Repräsentation und Verarbeitung sprachlichen Wissens. Sankt Augustin: Infix.

Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M., & Sedivy, J. C. (1995). Integration of visual and linguistic information in spoken language comprehension. Science, 268, 1632-1634.

Altmann, G. T. M., & Kamide, Y. (1999). Incremental interpretation at verbs: Restricting the domain of subsequent reference. Cognition, 73, 247-264.

Kamide, Y., Scheepers, C., Altmann, G. T. M. & Crocker, M. W. (2002). Integration of syntactic and semantic information in predictive processing: Anticipatory eye-movements in German. CUNY, 2002.