BRIDIGING THE GAP BETWEEN HERE AND THERE – COMBINING MULTIMODAL ANALYSIS FROM INTERNATIONAL PERSPECTIVES
Dr. Janina Wildfeuer
Multimodality can today be seen as one of the most influential semiotic theories for analyzing media artefacts. However, the concepts of this theory are ambiguously and heterogeneously widespread, especially with regard to approaches in Germany on the one hand and those within the international context on the other. Definitions of modality and even mediality differ from each other in terms of their general basis which, from a national perspective, is too often focused on language as the main point of description. A more general and internationally already well-established basis in multimodal analysis, in contrast, takes language no longer as the main semiotic resource of communication, but sees it as one part of the multimodal ensemble carrying meaning.
This lecture takes these differences in national and international perspectives as a starting point of discussion and analysis. It combines talks given at the BreMM14-conference of the same name that took place in September 2014 at Bremen University (for further information, see http://www.mm2014.uni-bremen.de). The participants of this conference all elaborated on the topic by making explicit which aspects of description, terminology and methodology vary and which difficulties result from these variations in daily academic life.
The talks of the conference have been arranged for this lecture according to the various topics level of description. Besides theoretical reflections on the term ‘multimodality’ and its research in a first section, there is also a methodological section (II) as well as a section showing empirical and experimental approaches (III). A fourth section includes diverse example analyses of various multimodal artefacts and further issues of multimodal work.
All talks will be published as papers in the following collection to which we refer for further details and references: Wildfeuer, Janina (ed.): Building Bridges for Multimodal Research International Perspectives on Theories and Practices of Multimodal Analysis. Bern/New York: Peter Lang [in progress].
I THEORETICAL REFLECTIONS ON THE NOTION AND WORK OF MULTIMODALITY
From Text-Linguistics to Multimodality – Mapping Concepts and Methods across Domains Hartmut Stöckl, University of Salzburg, Austria lecture - discussion - slides
Reflecting on the Semiotic Work of Multimodal Research (slides only) Diane Mavers, MODE Center for Researching Data and Environments, London, UK slides
II METHODS FOR MULTIMODALITY RESEARCH
A Visual Communication Approach to Multimodal Document Analysis Ognyan Seizov, Bremen University, Germany lecture - discussion - slides
Cognitive Semiotics as a Common Descriptive Framework for Pictorial and Verbal Representation Alina Kwiatkowska, University of Lodz, Poland
Dos and Don’ts in Pictorial and Multimodal Consumer Information: Visual Rhetoric and vs./Cognitive Semantics Sandra Handl, Innsbruck University, Austria lecture - discussion - slides
Images and Agonalität (Polarity) in Discourse Analysis Anna Mattfeldt, University of Heidelberg, Germany
Issues and Suggestions for Describing the GSP of Page-Based Multimodal Texts Wendy Bowcher, Sun Yat-sen University, China lecture - discussion - slides
Critical Discourse Analysis and Multimodality: Tracing ‘New’ Domains and Genres Shaimaa El Naggar, Lancaster University, UK lecture - discussion - slides
III EMPIRICAL AND EXPERIMENTAL APPROACHES TO MULTIMODAL ANALYSIS
Automatic Classification of Iconic Images Based on a Multimodal Model. An Interdisciplinary Project. Simone Ponzetto, Hartmut Wessler, Heiner Stuckenschmidt, University of Mannheim, Germany lecture - discussion - slides
On the Use of Different Modalities in Political Communication: Evidence from German Election Manifestos Heiner Stuckenschmidt, Hartmut Wessler, University of Mannheim, Germany lecture - discussion - slides
Seeing the Unforeseen: Eye-Tracking and Multimodal Analysis Martin Kaltenbacher, University of Salzburg, Austria lecture - discussion - slides
An Experimental Approach to Multimodality. Investigating the Interactions between Musical and Architectural Styles in Aesthetic Perception Martin Siefkes, Bremen University, Germany lecture - discussion - slides
Narrative Process Annotation of Comic Strips for Corpus Analysis Elisa Vales, Bremen University, Germany lecture - discussion - slides
IV EXAMPLE ANALYSES OF VARIOUS MULTIMODAL ARTEFACTS
Amodality – Multimodality. Metaphoric Meaning-Making in Audiovisuals Christina Schmitt, Free University Berlin, Germany lecture
Towards an Argumentative Analysis of Multimodal Documents Assimakis Tseronis, University of Amsterdam, The Netherlands lecture - discussion - slides
Public Healthcare Posters: A Social Semiotic Approach to Cross-Cultural Studies Kaela Zhang, The Hongkong Polytechnic University, Hongkong lecture - discussion - slides
German Sign Language (DGS) as an instance of multimodal language? Ulrike Wrobel, University of Hamburg, Germany
Layout Symmetry in Bilingual Documents Tuomo Hippala, University of Helsinki, Finland lecture - discussion - slides
Bridging the Gap between Polyphony and Multimodality in Online Media Formats Jan Krasni, University of Belgrade, Serbia lecture - discussion - slides
News Items Possible Paradigm in Analyzing Online Editions for Romanian Newspapers – A Multimodal Approach? Ana-Maria Teodeorescu, University of Bucharest, Romania lecture - discussion - slides
When Here is Now and There is Then: Bridging the Gap in Time Kate Maxwell, Agder University, Norway lecture - discussion - slides
Visual Literacy in EAP: The Dialogue between Reader/Designer Features and Multimodal Text Dorra Moalla, Faculty of Arts and Humanities of Sfax, Tunisia lecture - discussion - slides
