Abstract
This research problematises the visual-verbal semantic interface in a representative multimodal text from the Finance department of The Economist magazine. It proposes that the social semiotic, metafunctional view of communication advanced by systemic functional linguistics (SFL) is an effective theory for informing attempts to explicate the visual-verbal interface in page-based multimodal texts.This study shows that the SFL model has important and relevant applications to multimodal forms of communication. It is particularly useful in elucidating the intersemiotic relations between the linguistic and visual modes which commonly co-occur on page-based print media. In addressing the question of the ways that the visual and verbal modes work together in intersemiotic terms, this study tests the Hallidayan claim of the inter-relatedness of systems of meaning. It also explores the proposition that both the visual and verbal modes, while utilising the meaning-making features peculiar to their respective semiotic systems, work together in the particular context described to realise a unified, coherent multimodal text.
The SFL model interprets communication in social terms as a system of meaning potential. The speakers or writers make selections from the language system in response to various contexts of situation which are also instantiations of the wider context of culture. In accordance with the SFL view of context and language being in a dialectic relationship, the sample multimodal text from The Economist magazine is interpreted as a particular contextual configuration of the variables Field, Tenor and Mode which is situated in a particular context of creation, and which possesses an intertextual history. General background information on the context of creation is presented. This is concerned with The Economist magazine as an institution producing economic and financial journalism, its institutional history, and its editorial policy in terms of the ways that it aims to approach its readers through both verbal and visual means. Further, the sample text is examined in terms of its intertextual history, or the previously produced texts which contribute to its final form. These extra-visual and extra-linguistic variables are interpreted as having an important bearing on the production and subsequent interpretation of the sample text’s visual and verbal meanings.
Utilising this contextual information, it is demonstrated that in the representative Economist magazine text the visual and verbal modes complement each other semantically to produce a single textual phenomenon characterised by INTERSEMIOTIC COMPLEMENTARITY. A descriptive analytical framework is developed, and it is proposed that intersemiotic complementarity obtains when one or more of a range of criteria are met. These criteria are identified as intersemiotic ideational, interpersonal, or compositional metafunctional meanings. The results of this study demonstrate that the intersemiotic complementarity between the visual and verbal modes is realised in the sample text through three simultaneously-occurring intersemiotic resources. Firstly, the visual and verbal ideational meanings are lexico-semantically related by means of the intersemiotic sense relations of repetition, synonymy, antonymy, hyponymy, meronymy, and collocation. Secondly, the visual and verbal interpersonal meanings are related through intersemiotic reinforcement of address and intersemiotic attitudinal congruence. Thirdly, the visual-verbal compositional meanings are integrated through the compositional relations of information value, salience, and visual framing, as well as the influence of visual synonymy, and potential reading paths. These results are examined in terms of their implications for further research in the intersemiotic analysis of multimodal texts. Some educational implications are also discussed.
Chapters (for download)
Chapter 1 Introduction
Chapter 2 Review of Economics
Chapter 3 Semiotics and Linguistics
Chapter 4 Hallidayan approaches
Chapter 5 Theory and Framework
Chapter 6 Contextual Analysis
Chapter 7 Text Analysis
Chapter 8 Conclusions