TITLE: Corpus Approaches to Evaluation
SUBTITLE: Phraseology and Evaluative Language
SERIES TITLE: Routledge Advances in Corpus Linguistics
PUBLISHER: Routledge
YEAR: 2011
INTRODUCTION
‘Corpus Approaches to Evaluation’ by Susan Hunston is the thirteenth volume in the series of ‘Routledge Advances in Corpus Linguistics’, edited by Tony McEnery and Michael Hoey. Nine chapters, an appendix of concordance lines, notes, references and an index constitute the 199 numbered pages.
'Corpus Approaches to Evaluation' aims to combine the discourse analysis of evaluative language with a corpus linguistic approach, advocating the use of phraseology and pattern grammar to do so. Throughout, examples from naturally-occurring data are used to illustrate the various approaches, including concordance lines from a corpus of New Scientist articles and data from the Bank of English, as well different models of discourse analysis. The methods, texts, results and the variety of approaches examined are likely to appeal to anyone interested in corpus linguistics and discourse analysis, and to researchers in related areas such as computational linguistics, FrameNet, English for academic and specific purposes, and theories of embodiment and constructional grammar.
SUMMARY
The introductory Chapter One defines the major terms used in the study. Evaluative language is described as indexing an act of evaluation or stance taking. Evaluation and stance are further investigated in chapter two, but two important points must be made here. The first is that evaluation may be cumulative in a text and it may also be implicit. The second point, in fact the central challenge in this book, is that evaluation is typically analysed after a close reading of discourse. That is, an analysis of evaluative language is not normally carried out by computer through an automatic process. It is a main aim of this book to make the analysis of evaluation in discourse amenable to corpus analytical methods. Here, a corpus refers to a machine-readable collection of texts that can be exploited by data manipulation techniques. Data is often extracted from a corpus to identify the frequency of lexical items or grammatical classes, but Hunston advocates the use of this corpus data to study phraseology through the use of collocation, colligation and lexical priming (Hoey, 2005). Finally, chapter one outlines the remaining chapters, dividing the book into three parts – the first three chapters focusing on discoursal issues, chapters four to eight dedicated to corpus issues, and a concluding chapter that summarises and outlines future research.
Chapter Two investigates discourse analytical approaches to appraisal, stance and evaluation. Outlining four main approaches to the study of evaluative language in different traditions, Hunston observes six similarities. All approaches: emphasise the subjective and inter-subjective nature of evaluation; describe the role of reader and writer in jointly construing ideologies; recognize a broad range of indices for evaluative language, some of which have stable meanings while others are context-dependent; view evaluation as cumulative through text and dependent on context; have both a target and a source, and; recognize that evaluative language is relative, demanding that the point at which the analyst no longer recognizes evaluation needs to be made explicit. The four approaches to evaluation of Appraisal Theory, Status, Value & Relevance, Stance, and Metadiscourse are then reviewed in detail. In brief, Appraisal Theory (Martin & White, 2005) is a reading, rather than an analysis, because categorisation is still largely subjective. A typical category in Appraisal theory is Attitude (generally, feelings), which is subdivided into Affect (emotional responses of un/happiness, in/security or dis/satisfaction), Judgement (aesthetic view of object) and Appreciation (view of behavior). Three acts (moves) in discourse reveal Status. An object is identified for categorization (Status), it is accorded a Value and the finally the interpretation is given significance (Relevance). In some respects, Status, Value and Relevance have been superseded by Appraisal Theory. Stance, as evaluation, has been interpreted in two distinct ways. The first, from corpus linguistics, identifies and quantifies lexical items in a corpus as individual markers of evaluation. The second, from Conversation Analysis, views Stance (more accurately, stance-taking) as the active combination of evaluation (of an object), positioning (of subject) and alignment (with other subjects). Finally, the various discussions of metadiscourse characterize the degree to which a writer’s presence and use of evaluation of the text are made explicit.
Chapter Three further examines the construct of Status, and applies its analysis to multimodal texts. A writer avers status on propositions in a text, or attributes the status of a proposition to another source. Texts do not reflect the world but the writer’s construal of the world, including what is said, what may be, what is likely and what we think. Thus, “one function of evaluation is to reify texts and propositions by assigning them an epistemic status.” (p.25) The notion of status, described here and based on previous work by Hunston (e.g. 2000), can be usefully compared with the two concepts from Systemic Functional Linguistics (SFL) of Modality and Engagement. Modality is treated as the semantic space between ‘it is’ and ‘it isn’t,’ reflecting not just polarity but also obligation, possibility and desirability, most congruently realized in modal verbs but also allowing for metaphorical realizations such as ‘I think’ and ‘It is likely that.’ An Engagement analysis plots the writer’s negotiation of the (imagined) dialogue with the reader. Analyses of texts from different genres highlight the strengths and weaknesses of the Status and Engagement models of analysis.
The fourth chapter marks the transition of the book from discourse theories to corpus approaches, beginning with the contribution of corpus studies to understanding evaluation. Corpus approaches often identify a limited set of lexical items used for marking evaluation and examine their behavior in and across corpora. These Corpus studies permit a wide range of discourse-based findings, showing for instance that evaluation is relatively more frequent in spoken than written discourse, that metadiscourse is relatively more frequent in doctoral than masters theses and that some academic disciplines exhibit counter-intuitive patterns of evaluative language. Results from corpus and appraisal analyses have been integrated into sentiment analysis (or “opinion mining”), which attempts to automatically characterize attitudes and opinions in text. While automatic analysis is unable to recognize context-specific evaluation, one can expect no more than a ‘good enough’ indication of the polarity of the evaluation. This chapter also introduces the importance of phraseology, drawing inspiration from Sinclair’s (e.g. 1991) pioneering corpus work which emphasized that units of meaning are routinely realized by linguistic units beyond single words. The key concepts, developed in later chapters, are collocation (items that regularly co-occur with a core term), colligation (grammatical features associated with the core term), semantic preference (the set of words that regularly appear within the frame of a core term) and semantic prosody (the pragmatic meaning, or discourse function, of a core term). For instance, corpus analysis of the behavior of words reveals that negative evaluation may be a stable feature of a word and that its collocations may also be typically negative, or that a word may have a stable meaning that is neutral but that it collocates almost exclusively with items with negative evaluation. The chapter ends by previewing the combination of discourse and corpus analysis to shed light on evaluation in discourse.
The study of modal-like expressions in chapter five appears at first a digression, but later becomes a central aspect of the argument of the book. While the congruent meanings of modal auxiliaries express evaluation (typically obligation, possibility and desirability), other phrases and structures can be used in context or across contexts to express similar meanings. The challenge is to identify these phrases and structures without recourse to ‘guesswork.’ Hunston advocates using corpus results to identify lexical items which co-occur frequently with recognisably evaluative items such as modal auxiliary verbs. Starting with verbs, the words that appear relatively frequently in the vicinity of modal expressions, particularly verb-preposition combinations, are studied for frequent phrases. Concordance lines are produced for the analyst to then look for patterns in the data. For instance, ‘decide’ occurs regularly within the vicinity of modal auxiliary verbs. Examining the data for decide, the phrase ‘to decide wh-‘ (where wh- represents wh- question words, relative clause complementisers, or similar) reveals itself to carry a meaning similar to that typically carried by a modal auxiliary, even when there are no auxiliaries in the co-text: “almost all of the most frequent immediate left collocates of to decide whether turn out to be part of phraseologies that have a modal meaning although the grammatical functionality is different in each case.” (p.73) Expressions that carry a modal meaning are labelled Modal-Like Expressions, or MLEs. MLEs offer greater flexibility in distribution through the sentence, enabling greater flexibility to assign lexical items the status of Theme and New information. On further investigation, a range of selected verbs exhibited a tendency to appear as a base form (infinitive) and to be regularly associated with certain phraseologies. For instance 46% of all uses of ‘prefer’ occurred in the phrases ‘I prefer,’ ‘would prefer’ and ‘if you prefer’; 41% of all uses of ‘care’ were ‘don’t care,’ ‘didn’t care’ or ‘couldn’t care less.’ Hunston interpreted these phraseologies to carry evaluative meaning when examined in the context of concordance lines.
Chapter six investigates status, as described in chapter two, using the method described in chapter five. A previous study identified 11 nouns (including ‘idea’, ‘assumption’ and ‘conclusion’) that appeared in the ‘Noun + that’ structure to identify status. These were then examined using concordance lines from a corpus of New Scientist articles, and recurrent patterns revealed five groups of meanings for these nouns. Status nouns are used for the discourse functions of Existence, Evaluation (subdivided into agreement, affect and appreciation), Cause, Result, and Confirmation (subdivided into support, explanation, and consistency). For instance, when ‘assumption’ is used in the frequent pattern such as ‘this assumption,’ ‘had assumed that,’ or ‘the assumption that’, whatever is being evaluated is “far more likely to be evaluated negatively than positively” (p.100). Examination of concordanced examples suggest that ‘assumption‘ indicates a site of contention or an old assumption that has been overturned by a newer discovery. Another example, ‘fact’, is particularly problematic because, unlike the word hypothesis which always labels a hypothesis, facts are rarely labeled as such. Examination of ‘the fact that’ phrases in context results in three identifiable groups: those with a ‘cause motif’ (e.g. ‘rely on the fact that’, ‘problem lies in the fact that’), an ‘orientation motif’ (e.g. ‘account for the fact that’) or a ‘human response motif’ (e.g. ‘blind us to the fact that’, ‘draw attention to the fact that’, ‘draw comfort from the fact that’).
The aim of chapter seven is to use pattern grammar (Hunston and Francis, 1999) and local grammar (e.g. Barnbrook 2002) to identify evaluation in a corpus. Pattern Grammar uses Sinclair’s (1991) ‘Idiom Principle’ which observes that natural, fluent language consists far more of pre-assembled chunks than spontaneously constructed sequences predicted by a slot-and-filler model of generative linguistics. Pattern grammar urges us to look at larger units which are rarely ambiguous, especially in context. While we may be able to identify meanings for individual lexical items, these meanings are ad hoc and mutable and derived from repeated behaviour across phrases, and it seems to be the phrases that display the most stable meanings. Looking at evaluative phrases, Hunston notes that ‘that-‘, ‘to –‘ and ‘wh-‘ clauses package information for subjective comment, while prepositions typically classify, and prepositions like ‘about’, ‘as’ and ‘against’ may evaluate. Combining these cues for evaluation with gradable adjectives, which are also frequently but not exclusively associated with evaluation, Hunston produces very similar patterns to Martin and White’s ‘grammatical frames’ for evaluation of Affect, Appreciation and Judgement (as outlined in chapter two). Using the canonical grammatical frames for each type of evaluation (I feel, I consider it X, and It was X of him/her to, respectively), Hunston applies the methodology of chapter 6 by looking at concordances of the most frequent collocates of these phrases to discover the tendency of prepositions to recur in the patterns. This results in a variety of patterns, or frames, that typically evaluate: ‘it V-link Adj that’, ‘there V-link something Adj about n/Ving’. While some ‘Adj about N’ patterns depend on the choice of adjective, some of the patterns (‘V N as N’, ‘V N as Adj,’ ‘V N N’ and ‘V N Adj’) typically evaluate and are multilayered. Exemplifications include ‘represent the story as a steady progression,’ ‘consider him aloof,’ consider us a happy family,’ and ‘pronounced him sane.’ Finally, Hunston notes similar results between an approach using local grammar, which should be a fully functional grammatical description, and FrameNet in the categories for ‘difficult’ and similar gradable evaluating adjectives, even though local grammars start with a phraseological pattern while FrameNet approaches language from semantic elements.
Not all evaluations are equal, and so chapter eight attempts to measure the mutually-supporting intensity (strength) and density (extent or quantity) of evaluations through phraseology. One reason the identification of evaluation in a text can be problematic is that it tends to behave prosodically, rather than using particulate forms. Pulses of consistent evaluations through a text tend to support each other to create coherence, but as evaluations become more relatively frequent or increasingly forceful in a text, each instance becomes more redundant. Using corpus linguistic techniques Hunston is able to identify a range of phrases that are typically associated with intensifying evaluative meanings, such as ‘in an epoch of,’ ‘in the event’ and ‘as is humanly possible.’ Starting with individual texts, phrases such as ‘to the point of’ and ‘bordering on’ are investigated in a corpus for behaviour across texts revealing an important intensifying role. Hunston then outlines a process to identify similar intensifying phrases. Starting with typically evaluative lexical items (‘aggression,’ ‘endure,’ ‘suffer,’ ‘tragedy’ etc.), i.e. items with a ‘stable meaning’ for evaluation, a suitable corpus is searched for the most common collocates. After removing non-significant collocates, concordance lines are classified. For instance, starting with the phrase ‘tragedy,’ ‘of’ was added as a highly frequent collocate of tragedy and because it has already been identified as potentially contributing to evaluative meanings. The corpus process described for ‘of tragedy’ produced the phrases ‘in the face,’ ‘a time,’ ‘in the midst,’ all of which are used for negative evaluation, and ‘an air,’ ‘fair share,’ and ‘the seeds,’ which can be used for positive or negative evaluation. Taking other probes in place of ‘tragedy’, identified the intensifying phrases ‘to the point of,’ ‘on the verge of,’ a hint of,’ and ‘can’t emphasise enough’ among others. In the last two cases, the phrases increase intensity but not redundancy because their polarity is unpredictable.
In the concluding chapter Hunston reviews the corpus linguistic methods used in this study of evaluative phrases to stress that corpus linguistics is not defined by its object of study – a corpus of machine-readable texts – but by its methods of research and interrogation. Using some of the simplest corpus investigation tools available, (frequency data and concordance lines), Hunston has investigated one of the most elusive functions of language. There is still much work to do. As Hunston reminds us in chapter eight, “only the tip of the iceberg has been investigated here.” (p.165)
EVALUATION
‘Corpus Approaches to Evaluation’ is written in a clear style with very little assumed knowledge. The concepts and arguments are presented clearly and logically which should make it accessible to a wide audience. This is despite the rather elusive subject matter. Any researcher who becomes involved in language as it is produced on a day-to-day basis will recognize that evaluation is a ubiquitous part of language use in all contexts. While great strides have been made to understand and analyse this language (e.g. Martin and White, 2005), this title takes the project of analyzing evaluative language one step further by taking steps towards its reliable automatic recognition.
Hunston successfully demonstrates the identification of a wide range of language exponents of evaluation through very simple corpus tools. In fact, one of the most surprising aspects of Hunston’s approach is the lack of sophisticated statistical methods to identify significant phrases. It would be interesting to compare Hunston’s results, which depend heavily on the labour-intensive examination and analysis of concordance lines, with those from a statistical approach that utilises relative frequency, t-test, mutual information and other measures of variance from expectation. I believe that this would help to resolve the one issue I see with Hunston’s methodology. All of the examples shown in the book depend on a non-corpus source for the identification of an initial probe. The sources for the original evaluative term are previous studies by Hunston (often with Francis), suggestions from Martin & White’s appraisal framework or from FrameNet, sentiment analysis or similar. With an initial probe derived from these sources, Hunston is able to reliably identify a range of valid evaluative phrases. However, this appears to be an ad hoc methodology that could easily neglect many significant phrases – without a discourse analysis to identify a certain probe, the associated phrases will not be found, and the methodology is dependent on chance.
This volume succeeds in taking an aspect of language which does not appear to be amenable to corpus linguistic analysis and manages, as with all good corpus studies, to reveal language that we all know to be important in performing a certain function but which we would be unable to identify spontaneously. There is still a lot of work to do here, but Hunston shows how a suitable corpus linguistic methodology can be applied to validate theories of discourse analysis.
REFERENCES
Barnbrook, Geoff. 2002. Defining Language: A Local Grammar of Definition Sentences. Amsterdam: Benjamins
Hoey, Michael. 2005. Lexical Priming: A New Theory of Words and Language. London: Routledge.
Hunston, Susan. 2000. ‘Evaluation and the planes of discourse: Status and value in persuasive texts.’ In S. Hunston & G. Thompson (eds.) Evaluation in Text: Authorial Stance and the Construction of Discourse. Oxford: Oxford University Press
Hunston, Susan and Francis, Gill. 1999. Pattern Grammar: A Corpus-Driven Approach to the Lexical Grammar of English. Amsterdam: Benjamins
Martin, James R. & White, Peter R.R. 2005. The Language of Evaluation: Appraisal in English. Basingstoke: Palgrave Macmillan
Sinclair, John McH. 1991. Corpus Concordance Collocation. Oxford: Oxford University Press
ABOUT THE REVIEWER
Nick Moore has worked in Brazil, Oman, Turkey, UAE and UK with students and teachers of English as a foreign language, English for Specific and Academic Purposes and linguistics. He holds the RSA/UCLES Dip. TEFL and Aston University’s M.Sc. in Teaching English. His PhD in applied linguistics from the University of Liverpool addressed information structure in written English. Other research interests include systemic functional linguistics, corpus linguistics, theories of embodiment, lexis and skills in language teaching, and reading programmes. Dr. Moore is a reviewer for a number of journals and the co-editor of ‘READ.’ He currently coordinates and teaches undergraduate courses in English composition and technical writing, as well as an introductory linguistics course, at Khalifa University in the United Arab Emirates.
NOTE: This review was prepared for linguistlist.org and can also be viewed from here.
No comments:
Post a Comment