Problems With The System Of Image-Text Relations

 Martin & Rose (2007: 329, 333n):

Image-text relations include expansion or projection, boundary strength and identification. These … options in image-text relations in Figure 9.14.⁹

 

⁹ We have used the term IMAGE-TEXT BOUNDARY whereas Kress and van Leeuwen (1996) use the term ‘framing’ for boundary strength, which conflicts with Bernstein’s (1971, 1996) use of ‘framing’ for control within a context.

Blogger Comments:

[1] As previously observed, expansion and projection do not feature in the discourse semantics of Martin & Rose. Of the expansion categories, the authors provided no example of an extension relation between image and text, and neither example of elaboration or enhancement withstood close scrutiny. With regard to projection relations between image and text, the authors provided no examples, and importantly, a text cannot project an image as a locution, because locutions are wordings, and images lack a lexicogrammatical stratum.

[2] Importantly, as previously observed, boundary strength is an expression plane distinction, but Martin & Rose here again misconstrue it as being of the same level of symbolic abstraction as (conjunctively related) content plane systems.

[3] As previously observed, the authors' example of image-text identification did not withstand close scrutiny, because it involved the false claim that people depicted in a photograph were gazing at the text to the left of the photograph, despite the fact that their gaze was actually directed to the stage in the background of the photograph.

[4] To be clear, this footnote at the end of the chapter acknowledges the intellectual source of the the authors' model, and identifies their own contribution as rebranding the original system, on the spurious pretext that the original term has a different meaning in a different field (which can also be said for many other terms. such as 'projection', for example).

Problems With The System Of Image Textual Organisation

Martin & Rose (2007: 329, 333n):

In sum, images and layouts are organised by their left-right, top-down and centre-margin axes, and by the relative salience of their elements. … These options in textual organisation are set out in Figure 9.13. … ⁹

 

⁹ Kress and van Leeuwen’s terms Given and New derive from Halliday’s description of the linguistic system of INFORMATION (Halliday and Matthiessen 2004). We have generalised their Given-New, Ideal-Real and Centre-Margin contrasts as options in INFORMATION DISTRIBUTION. 

The term ‘salience’ is used by Kress and van Leeuwen, but the SALIENCE values of high/neutral/low are our own. 

Blogger Comments:

[1] To be clear, this confuses the spatial dimensions of images with textual organisation: the relative textual highlighting of information in images. In doing so, it omits the depth dimension: from foreground to background, and misconstrues the centre-margin relation as an axis.

[2] To be clear, the validity of the network can be challenged on two grounds: its features and its wiring. 

With regard to its features, as previously observed, the 'horizontal' system makes the unwarranted assumption that Given always precedes New, whereas in the source model, New may precede Given, Given may be absent, or "surround" the New. 

With regard to the 'vertical' system, the features 'ideal' and 'real' are categories of ideational meaning, not categories of textual highlighting. 

With regard to the 'central' system, the centre of an image is not always the textual focus, as previously demonstrated for the case of maps. 

And with regard, to the 'SALIENCE' system, Martin & Rose have not demonstrated three levels of salience in their analyses.

With regard to the wiring of the network, it allows for the contradictory combinations of textual highlighting (New, Central) with low SALIENCE, and textual downplaying (Given, Marginal) with high SALIENCE.

[3] To be clear, this footnote at the end of the chapter acknowledges the intellectual source of the their model, and identifies their own contribution as a system name and feature scaling.

Enhancing Image-Text Relations

 Martin & Rose (2007: 323, 328-9):

The left-right axis of the page, the vectors in the inauguration-flag image, and its relations with preceding text, combine to construct an indexical temporal sequence. The gaze of people in the crowd is up to the stage and across to the left. Implicit in these gazes is the inauguration ceremony they are watching, and its central protagonist, Mandela. And their gaze is also towards Mandela’s life story that lies to the left of the image. These vectors realise implicit identification, all pointing anaphorically to ‘him’, Mandela. But Mandela himself is not in the picture. Counterbalancing this up and leftward gaze is the powerful vector in the flag, which points down and right towards the people who surround it, cataphorically identifying ‘them’. In sum, the layout and images indexically construe a complex activity sequence, in which not only apartheid belongs to the past, but also the struggle against it, and Mandela’s own life story. In contrast the future belongs to the people.

Blogger Comments:

[1] This is misleading, because it is not true. The two photographs construe the same temporal location — that of the inauguration of Mandela as President — but the text, an excerpt from the Meaning of Freedom text (reproduced below) is not located in time relative to the inauguration.

[2] This is misleading, because it is not true. The gaze of the crowd in the right-hand photograph is towards the background in the image, not left to the adjacent text.

[3] To be clear, this confuses paying attention (looking at phenomena) with directing the attention of others (pointing at phenomena).

[4] To be clear, this "powerful vector in the flag" is directed only to one anonymous head, in the foreground of the photograph. That is, every other person in the crowd is excluded by this vector.

[5] As demonstrated above, this conclusion is not warranted by the authors' analyses. The 'activity sequence' is confined to the text, and the two photographs, both of the same occasion, are not related temporally to the text.

∞

Meaning of Freedom

I was not born with a hunger to be free. I was born free - free in every way that I could know. Free to run in the fields near my mother's hut, free to swim in the clear stream that ran through my village, free to roast mealies under the stars and ride the broad backs of slow^moving bulls, As long as I obeyed my father and abided by the customs of my tribe, I was not troubled by the laws of man or God.

It was only when ! began to learn that my boyhood freedom was an illusion, when I discovered as a young man that my freedom had already been taken from me, that I began to hunger for it.

At first, as a student, I wanted freedom only for myself, the transitory freedoms of being able to stay out at night, read what I pleased and go where i chose. Later, as a young man in Johannesburg, I yearned for the basic and honourable freedoms of achieving my potential, of earning my keep, of marrying and having a family - the freedom not to be obstructed in a lawful life.

But then i slowly saw that not only was I not free, but my brothers and sisters were not free.

I saw that it was not just my freedom that was curtailed, but the freedom of everyone who looked like I did. That is when I joined the African National Congress and that is when the hunger for my own freedom became the greater hunger for the freedom of my people. It was this desire for the freedom of my people to live their lives with dignity and self-respect that animated my life, that transformed a frightened young man into a bold one, that drove a law-abiding attorney to become a criminal that turned a family-loving husband into a man without a home, that forced a life-loving man to live like a monk. I am no more virtuous or self-sacrificing than the next man, but I found that 1 could not even enjoy the poor and limited freedoms I was allowed when I knew my people were not free.

Freedom is indivisible; the chains on any one of my people were the chains on all of them, the chains on all of my people were the chains on me. 

It was during those long and lonely years that my hunger for the freedom of my own people became a hunger for the freedom of all people, white and black. 

I knew as well as I knew anything that the oppressor must be liberated just as surely as the oppressed. A man who takes away another man's freedom is a prisoner of hatred, he is locked behind the bars of prejudice and narrow-mindedness. I am not truly free if I am taking away someone else’s freedom, just as surely as I am not free when my freedom is taken from me. 

The oppressed and the oppressor alike are robbed of their humanity.

When I walked out of prison, that was my mission, to liberate the oppressed and the oppressor both. 

Some say that has now been achieved. But I know that this is not the case. The truth is that we are not yet free; we have merely achieved the freedom to be free, the right not to be oppressed. We have not taken the final step of our journey, but the first step on a longer and even more difficult road. For to be free is not merely to cast off one's chains, but to live in a way that respects and enhances the freedom of others.

The true test of our devotion to freedom is just beginning. 

I have walked that long road to freedom. I have tried not to falter; I have made missteps along the way. But I have discovered the secret that after climbing a great hill, one only finds that there are many more hills to climb. I have taken a moment here to rest, to steal a view of the glorious vista that surrounds me, to look back on the distance I have come. But I can only rest for a moment, for with freedom come responsibilities, and I dare not linger, for my long walk is not yet ended.

(Mandela 1995: 746-51)

Elaborating Image-Text Relations

Martin & Rose (2007: 323, 328):

In the horizontal triptych, image-text relations are both elaborating and enhancing. The image of the boy restates the words that begin Mandela’s story, I was not born with a hunger to be free . . . It was only when I began to learn that my boyhood freedom was an illusion ... that I began to hunger for it. The analogy with the text is signalled by the weak image-text boundary, with the photo intruding into the text. In contrast, the image of the inauguration is more strongly bounded from Mandela’s story, and Mandela himself is noticeably absent from the photo, replaced by the people of South Africa under the flag of their new nation. So this image is clearly marked off from the text as distinct new information.

Blogger Comments:

[1] As previously noted, the logico-semantic relations of elaboration and enhancement do not feature in Martin's model of logical discourse semantics, conjunction, because it is a rebranding of Halliday & Hasan's (1976) model of cohesive conjunction, which was written before they had formulated the most general categories of logical relations.

[2] To be clear, this is manifestly untrue. The image is a photograph of a boy at Mandela's inauguration as President. As such, it does not restate Mandela's words about himself. This can be demonstrated by presenting the photograph to someone and asking them what it "states".

[3] To be clear, this is a bare assertion, unsupported by argument as to why a "weak image-text boundary" should represent a relation of analogy, rather than anything else, or why a "strong image-text boundary" should not represent a relation of analogy. Moreover, it is not true that the image "intrudes" into the text, since there is no overlap of image with text. Instead, the text wraps around the image, each remaining distinct from each other.

[4] To be clear, this is bare assertion, unsupported by argument as why this image-text boundary is "more strongly bounded" than the other. On the contrary, both images are unframed, and both images are separate from the texts adjacent to them. This would suggest that the image-text boundaries are the same for both images.

[5] To be clear, Mandela is absent from both photographs. The boy depicted in the left-hand photograph is not Mandela, but a member of the crowd at Mandela's inauguration as President. Accordingly, Mandela has not been replaced in the right-hand photograph.

[6] To be clear, this conclusion is not validated by the propositions that precede it; see [4] and [5].

The Vertical Textual Organisation Of Layout: Ideal/Real

 Martin & Rose (2007: 323, 328):

The semantic contrast between the top and bottom texts is between the historical origins of apartheid (Ideal) and their outcomes in the recent past and present (Real);  the text above deals with both the inauguration of a new republic and erection of the former apartheid regime; the text below notes the effects on people, first of the old regime (harsh and inhumane), and then of the new (respect for the rights and freedoms of all peoples). The photo of the boy mediates this temporal succession: the top-bottom layout construes the apartheid regime as preceding the boy’s protest, and its overturning as following his protest. The relatively high salience of the image has the effect of emphasising the causative role of the people’s defiance, represented by the boy, in overturning the inhumane regime. This is a reading of the texts enhancing the image, but as we discussed above for appraisal, the boy’s tenacity can also be read as determination for the future, which is elaborated by the words in the text below (overturned forever).

Blogger Comments:

[1] To be clear, "the top and bottom texts" are one text, or more properly, an excerpt of one text, interrupted by the photograph of a boy in the crowd at Mandela's inauguration as President.

[2] The authors' claim here is that Mandela is presenting the erection of the Apartheid system as ideal, and that 'ideal' is textual meaning, rather than ideational meaning.

[3] The authors' claim here is that Mandela is presenting the overthrow of the Apartheid system as not ideal, and that 'real' is textual meaning, rather than ideational meaning.

[4] To be clear, this is the authors' interpretation of the ideational meaning of the text above and below the photograph. Martin & Rose present this as an analysis of the textual meaning of the layout.

[5] To be clear, this is absolute nonsense. The photograph simply depicts a boy celebrating — not protesting — at the inauguration of Mandela as President. That is, it depicts the Range of the opening thematic circumstance of the text: On the day of the inauguration.

[6] On the one hand, the relatively high salience of the image is in terms of its (iconic) expression, relative to the (symbolic) expression of language. Whether this salience translates to its content needs to be established by reasoned argument. 

On the other hand, as previously explained, the authors' claim that the depicted boy is expressing defiance derives from their interpretation of him as metaphorical for Mandela, and then transferring the defiance they attribute to Mandela onto the boy himself. The photograph simply depicts a boy celebrating the inauguration of Mandela as President.

[7] To be clear, as demonstrated above, neither of the authors' claims regarding enhancement, time [5] or causation [6], withstand close scrutiny.

[8] Again, the authors' claim that the depicted boy is expressing tenacity derives from their interpretation of him as metaphorical for Mandela, and then transferring the tenacity they attribute to Mandela onto the boy himself.

[9] On the one hand, experientially, the authors' claim here is that determination for the future is encoded by reference to the words "overturned forever":

On the other hand, logically, the authors' claim is that the words "overturned forever" either restates or exemplifies or specifies in greater detail determination for the future.

∞

Procrustean Bed: a scheme or pattern into which someone or something is arbitrarily forced

The Horizontal Textual Organisation Of Layout: Given/New

Martin & Rose (2007: 323, 328):

In the horizontal triptych on pages 202-3, the photo of the boy is Given and that of the inauguration is New.  The photos are more salient than the texts by virtue of their colour intensity and Given-New positions, with the inauguration image by far most salient. Our eye is attracted first to this large picture, then back to the photo of the boy, and then to the texts to explain the images for us. Within the left-hand vertical triptych, the image of the boy is more salient than the marginal texts above and below him, due to its size, colour and centrality. As a result we expect these texts to expand on the meaning of the photograph, and indeed they do, with enhancement.

Blogger Comments:

[1] To be clear, Martin & Rose provide no argument here for the applicability of Halliday's information unit to images and page layout. The authors merely accept its use by Kress & van Leeuwen (1996) without question. Moreover, there is no argument for treating the unmarked sequence, Given^New, as the only possible ordering. For information units, the structure may be:

• Given^New
• Given^New^Given
• New^Given
• New

Here the left image is claimed to be Given information merely because it is positioned to the left of the other image, which is claimed to be (the focus of) New information merely because it is positioned to the right of the other image. Martin & Rose provide no argument as to why a photograph of a boy in the crowd (who is not mentioned in the text) should be Given information relative to the photograph of the flag in the same crowd. It might be argued that the new South African flag is presented as New, but Martin & Rose do not make that argument.

[2] To be clear, here Martin & Rose have (unwittingly) switched their attention from the content plane to the expression plane, confusing salience of expression (size, colour, layout position) with salience of content.

[3] This is misleading, because it is untrue. While it is true that the photograph provides an instance of 'the dark-skinned peoples' mentioned in the texts, the texts above and below the photograph say nothing whatsoever about the meaning of the photograph:

See further in the following post. 

Image-Text Relations

Martin & Rose (2007: 327-8, 333n):

Image-text relations include their logical relations, the boundaries between text and image, and identification. Logical relationships can be mapped in terms of expansion or projection, as we described for texts in macrogenres in Chapter 8. For example, images and texts can restate, specify or summarise each other (elaborating), they can be added to each other (extending), or explain or follow each other in time (enhancing). Images can also project wordings as thought or speech bubbles, and the reverse is also possible. Boundaries between image and text may be weak or strong: images may intrude into text, and text may overlap images, or there may be strong demarcation. And finally elements of images may be identified explicitly in accompanying texts (e.g. in captions), and elements of text or other images may be referred to in accompanying images, for example by vectors that point to them.⁸

⁸ Kress and van Leeuwen draw attention to vectors, which can be constructed through the gaze of participants or lines formed by the position of people and things. Whereas they interpret vectors in ideational terms, its seems to us that vectors are realisational strategies for ideational or textual functions.

Blogger Comments:

[1] Importantly, and not acknowledged here, these logical and identification relations obtain at the level of content, whereas the text-image boundary relations obtain at the level of expression.

[2] To be clear, the authors' model of logical discourse semantics, the system of conjunction (now rebranded 'connexion'), does not use the general category 'expansion', and 'projection' is entirely absent. This is because Martin's model is his rebranding of cohesive conjunction (Halliday & Hasan 1976), in which the general category of expansion had not yet been theorised by Halliday, and in which projection plays no cohesive function.

[3] To be clear, this would involve instances like he said <image>, and she thought <image>.

[4] To be clear, as previously demonstrated in the examination of Chapter 5, the authors' model of identification is a confusion of 'reference' in the textually cohesive sense, and 'reference' in the ideational sense of denotation. It will be seen that this misunderstanding is maintained in their discussion of image-text relations.

Problems With The Model Of Textual Organisation Of Images

Martin & Rose (2007: 326-7):

To interpret the semantic relations of images to texts in the layout of pages 202-3, we need to introduce several dimensions of textual organisation and image-text relations. Kress and van Leeuwen (1996) suggest two forms of textual organisation for images, ‘polarised’ and ‘centred’. On the one hand there is polarisation along horizontal and/or vertical axes. For images that are horizontally polarised, the lefthand side is glossed as Given and the right as New — organisation comparable to that outlined by Halliday for the English clause, as introduced in Chapter 6 above. For vertically polarised images Kress and van Leeuwen suggest the terms Ideal and Real, where Ideal may be characterised as a more general or abstract category, and Real as more specific or concrete. Alternatively, images may be organised around a Centre and Margin principle, with Centre the nucleus of information on which marginal elements depend. These axes are schematised in Figure 9.12.

A further textual dimension is the relative salience of elements in an image or page layout that draws readers’ attention to one element before another. Salience may be indicated by a number of factors, including size, colour intensity or the strength of vectors, as well as centre-margin, left-right, top-down positions.

Blogger Comments:

[1] To be clear, for Halliday, Given and New are elements of the information unit, not the clause.

[2] To be clear, 'ideal' and 'real' are ideational categories, not textual categories. That is, Martin & Rose follow Kress & van Leeuwen (1996) in mistaking a claim about ideational organisation for textual organisation.

[3] To be clear, the functionality of this problematic model can be tested by applying it to the following image:

The claim then is that this pictographic image construes the following textual distinctions:

• the 'New World' (western hemisphere) is Given, whereas the 'Old World' (eastern hemisphere) is New;
• the northern hemisphere as Ideal, whereas the southern hemisphere is Real; and
• the Mediterranean region is Centre, whereas all else is Margin.

Or more specifically:

• North America is Given and Ideal;
• South America is Given and Real;
• Eurasia is New and Ideal;
• Australasia is New and Real;
• Africa is Marginal and neither Given nor New.

[4] To be clear, Martin & Rose have just previously interpreted these factors as resources for interpersonal graduation but here reinterpret them as resources for textual salience.

Problems With The System Of Interpersonal Meanings In Images

 Martin & Rose (2007: 326, 327):

In sum, the two photos illustrate options in attitude, engagement and graduation, set out in Figure 9.11.

Blogger Comments:

[1] To be clear, as demonstrated in previous posts, none of these applications of ATTITUDE features survives close scrutiny.

• The authors' application of affect misrepresented ideational construals of emotion as interpersonal assessments using emotion (affect). 
• The authors' application of appreciation confused ideational construals in a photograph with the appreciation of these by a viewer (the authors). 
• The authors' application of judgement involved reconstruing the boy in a photograph as a metaphor for Nelson Mandela, whom the authors judged as tenacious, and then incongruously transferring that judgement to the boy.

[2] To be clear, as observed in previous posts, the authors here misrepresent 'engagement' in the sense of Kress & van Leeuwen (1996) as 'engagement' in the appraisal sense.

[3] To be clear, Martin & Rose have not demonstrated three levels of graduation in images. As demonstrated in previous posts

• in exemplifying the graduation of appreciation, the authors confused scalable ideational qualities with the degree of their appreciation of them, and mistook textual prominence for interpersonal graduation, and
• in exemplifying the graduation of judgement, the authors interpreted the hand shape of the boy in a photograph as intensifying the tenacity they attributed to Mandela; see [1] above.

Applying Appreciation, Graduation And Engagement To The Inauguration-Flag Photograph

Martin & Rose (2007: 326, 327):

On the other hand the inauguration-flag photo invokes positive appreciation, including aspects of reaction, composition and valuation. With respect to terms exemplified in Table 2.10, the inauguration crowd appears imposing, exciting and dramatic, as does the huge flag, whose composition is both complex and unified, and which carries values that are at once profound, innovative and enduring. These values are amplified by the size and centrality of the flag, and the intensity of its colours. With respect to engagement, the people are facing directly away from the viewer, so we are obliquely invited to enter the scene in the direction they are facing.

Blogger Comments:

[1] To be clear, Martin & Rose are not identifying the meaning created by either the photographer or the boy in the photograph, but by specific viewers of the photograph (themselves). In doing so, they confuse ideational construals made by the photographer with interpersonal assessments made by the viewer.

[2] To be clear, here Martin & Rose confuse scalable aspects of the depicted flag with the degree (graduation) of their own appreciation. Moreover, to the extent that the foregrounding and centrality of the flag focuses attention on the flag, this is the "graduation" of textual meaning, not interpersonal meaning.

[3] To be clear, as previously observed, this is 'engagement' in the sense of Kress & van Leeuwen (1996), but misrepresented by Martin & Rose as 'engagement' in the appraisal sense.

Misapplying A Misrepresentation Of Engagement To A Photograph

 Martin & Rose (2007: 326):

As the boy directly faces the viewer, his defiance/celebration engages us directly, but at the same time his oblique gaze averts a potentially confronting challenge to the viewer. The message is not that I’m defying you, but is rather an invitation to join us in the victory over injustice.

Blogger Comments:

[1] This is misleading, because the boy does not directly face the viewer. His face is oriented to the left and down, relative to the viewer, his eyes further to the left, and his body tilts down and faces to the right of the viewer.

[2] To be clear, the authors' claim that the boy is expressing defiance is inconsistent both with the occasion, the inauguration of Mandela as President, and with other photographs of children purported to depict defiance, such as:

[3] To be clear, as previously observed, this is 'engagement' in the sense of Kress & van Leeuwen (1996), but misrepresented by Martin & Rose as 'engagement' in the appraisal sense.

Applying Judgement And Graduation To A Photograph

Martin & Rose (2007: 326):

In APPRAISAL terms, the photo of the boy invokes a positive judgement of tenacity that must be read in relation to the texts that surround him.  The protest against the regime construed by his raised fist reflects the tenacious resistance of Mandela and his comrades as recounted in the adjacent Freedom text.  The fist can then be read as amplifying his tenacity to the level of defiance (more so than if he had waved or saluted with an open hand).  This is a retrospective reading of his tenacity as defiance against the old regime;  on the other hand his tenacity can also be read prospectively as youthful determination in the nation’s hopes for the future. These are complementary readings as protest against the regime vs celebration of its overthrow, that are expanded by the texts above and below the photo — image-text relations that are discussed in the following section.

Blogger Comments:

[1] To be clear, Martin & Rose are not identifying the meaning created by either the photographer or the boy in the photograph, but by specific viewers of the photograph (themselves); see further below.

[2] To be clear, if the meaning of the image depends on the accompanying text, then the meaning is made by both semiotic systems, not by the image alone.

[3] To be clear, here Martin & Rose construe three levels of meaning within the image:

• the tenacious resistance of Mandela and his comrades, realised by
• the protest against the regime, realised by
• his raised fist.

More specifically, the middle level of meaning, the protest against the regime, is metaphorically encoded by reference to the lowest level, his raised fist, and the highest level, the tenacious resistance of Mandela and his comrades, is metaphorically decoded by reference to the middle level, the protest against the regime.

That is, the judgement of tenacity in this image of a boy is made on Mandela and his comrades by Martin & Rose.

[4] To be clear, here Martin & Rose have become confused by their levels of abstraction and incongruously transferred their judgement of the tenacity from Mandela and his comrades to the boy ('his tenacity') in the photograph. This is analogous to transferring a judgement of a movie character to the actor playing the rôle. With this confusion, they claim, without supporting argument, that the shape of the boy's hand is an amplification of his tenacity to the level of defiance, despite the fact that 'tenacity' means persistence, whereas 'defiance' means resistance.

[5] To be clear, here again Martin & Rose misattribute the tenacity they have ascribed to Mandela and his comrades to the boy ('his tenacity') in the photograph, and decode his tenacity by reference to youthful determination in the nation's hope for the future.

[6] To be clear, Halliday (1985: xvii) comments on discourse analysis seem apposite here:

A discourse analysis that is not based on grammar is not an analysis at all, but simply a running commentary on a text … the exercise remains a private one in which one explanation is as good or as bad as another.

[7] To be clear, as can be seen below, the texts above and below the photograph of the boy (p324) do not expand the meanings of the photograph, as interpreted by Martin & Rose:

'regimes' above photo of young boy

On the day of the inauguration I was overwhelmed with a sense of history. In the first decade of the twentieth century, a few years after the bitter Anglo-Boer war and before my own birth, the white-skinned peoples of South Africa patched up their differences and erected a system of racial domination against the dark-skinned peoples of their own land.

'effects' below photo of young boy

The structure they created formed the basis of one of the harshest, most inhumane, societies the world has ever known. Now, in the last decade of the twentieth century, and my own eighth decade as a man, that system has been overturned forever and replaced by one that recognised the rights and freedoms of all peoples regardless of the colour of their skin. (Mandela 1996: 202)

Applying Appraisal Theory To Images

Martin & Rose (2007: 325):

In terms of APPRAISAL developed in Chapter 2, images can inscribe feelings, for example with an image of a person crying or smiling, or invoke them with images that we respond to emotionally; they can invoke appreciation of things by the relative attractiveness of the object or scene presented; and they can invoke judgements of people, by means such as their activity, stance or facial expression. Engagement with the viewer can also be varied in images, for example by the gaze of depicted people looking directly at the viewer, obliquely to one side, or directly away from the viewer into the image. And of course feelings, appreciation and judgement can also be amplified and diminished.

Blogger Comments:

[1] To be clear, this is a basic misunderstanding that pervades work on APPRAISAL. The AFFECT system of APPRAISAL is concerned with interpersonal assessment through emotion, not with the ideational construal of emotion. The depiction of a person crying or smiling is an ideational construal of behavioural processes that manifest states of consciousness (Halliday & Matthiessen 2014: 302).

[2] To be clear, creating an image that induces an emotional response in the viewer is not a variation of creating an image that depicts an interpersonal assessment through emotion.

[3] To be clear, this blurs the distinction between an appreciative assessment made by the creator of an image and feelings of appreciation induced in a viewer by an image. This is analogous to blurring the distinction between what a speaker says and the reaction of the addressee to what is said.

[4] Again, this blurs the distinction between a judgemental assessment made by the creator of an image and judgements induced in a viewer by an image. Again, this is analogous to blurring the distinction between what a speaker says and the reaction of the addressee to what is said.

[5] To be clear, this misunderstands the APPRAISAL system of ENGAGEMENT, which is concerned with whether or not other points of view on propositions are acknowledged, how they are acknowledged, and to what rhetorical ends.  As Martin & White (2005: 92) explain:

[Engagement] is concerned with the linguistic resources by which speakers/writers adopt a stance towards to the value positions being referenced by the text and with respect to those they address … the different possibilities for this stance-taking which are made available by the language, … the rhetorical effects associated with these various positionings, and … what is at stake when one stance is chosen over another.

Martin & Rose, on the other hand, here misinterpret engagement as (metaphorically) realised by the behavioural stance of an entity of an image (metaphenomenon) relative to a viewer of the image (phenomenon). That is, this is 'engagement' in the sense of Kress & van Leeuwen (1996), but misrepresented by the authors as 'engagement' in the sense of Appraisal Theory.

Problems With The System Network Of Ideational Meanings In Images

 Martin & Rose (2007: 325):

A very general outline of options for ideational meanings in images is given in Figure 9.10.

Blogger Comments:

As previously demonstrated, these systems model textual meaning, not ideational meaning. On the one hand, it is the textual metafunction that focuses on phenomena, and on the other hand, it is the textual metafunction at the level of context, mode, that is distinguishes the channel of 'construal'. That is, the authors' network is theoretically inconsistent in terms of both metafunction and plane of symbolic abstraction.

With regard to the wiring of the network, the claim is that entity-focused images either classify entities (e.g. 'boy') or present them as composed of parts (eg. 'head', 'fist' etc.), but not both.

With regard to the argument on which the network is based, as previously demonstrated, Martin & Rose have not provided an instance of a complex activity, and have not provided a consistent theoretical argument for their inclusion of the term 'indexical'.

The Authors' Inconsistent Use Of Peirce's 'Iconic', 'Indexical' And 'Symbolic'

Martin & Rose (2007: 325, 333n):

Beyond this is the manner in which they are construed. Photos and realistic drawings can depict entities and activities iconically; there is a direct visual relation between the image and the category it construes. In contrast, images such as flags or diagrams construe their categories symbolically; the viewer must know the symbol to recognise its meaning. 

In between are images that are neither iconic nor symbolic, but indicate categories by one or more criteria; an example is the relation between the crowd, the dignitaries on the stage, and the flag, which indicate the categories of the people, their leaders and the nation by their relative positions — bottom, top and middle. In Peirce’s 1955 terms, this kind of visual construal is indexical.⁷ 

⁷ Previous efforts to interpret ideational and interpersonal meanings in visual images have been based on analogies with grammatical categories of process types, mood and modality (e.g. Kress and van Leeuwen 1996, O’Toole 1994, Unsworth 2001) rather than discourse semantics. In keeping with the discourse oriented approach here, and to keep labels manageable, we have used the same terms as for verbal texts wherever possible. For example, where Kress and van Leeuwen use the cryptic terms ‘overt/covert’, we use ‘explicit/implicit’; and where they use polysemous terms ‘concrete/abstract’, we have found the semiotic terms ‘iconic/indexical/symbolic’ less ambiguous.

Blogger Comments:

[1] To be clear, in SFL Theory, "the manner in which they are construed" is modelled as channel, a system within MODE, the textual dimension of the context (culture); see Halliday & Matthiessen (2014: 34). It will be seen in the following post that Martin & Rose misconstrue this as an ideational system at the level of discourse semantics.

[2] To put this in terms of SFL Theory, for such images, whose graphic channels might be subcategorised as photographic and pictographic, there is a natural (non-arbitrary) relation between their content and expression.

[3] To put this in terms of SFL Theory, for such images, whose graphic channel might be subcategorised as ideographic, there is a non-natural (conventional) relation between their content and expression.

[4] To put this in terms of SFL Theory, unlike the preceding characterisations of 'iconic' and 'symbolic' images, this characterisation of 'indexical' images is not concerned with a relation between content and expression, but with a relation between levels of symbolic abstraction within the content of the image:

That is, Martin & Rose reconstrue the meanings that are 'iconically' realised in the image as metaphorical symbols of a higher level, more congruent meaning.

To be clear, in order to be theoretically consistent with their characterisation of 'iconic' and 'symbolic' images, the authors need to demonstrate an indexical relation between the content of the image (its meanings) and the expression of the image (its ink patterns).

[5] To be clear, all the terms — iconic, symbolic and indexical — derive from the semiotics of Peirce, a model that is epistemologically inconsistent with SFL Theory. Peirce (1955: 102-3):

According to the second trichotomy, a Sign may be termed an Icon, an Index, or a Symbol. 

An Icon is a sign which refers to the Object that it denotes merely by virtue of characters of its own, and which it possesses, just the same, whether any such Object actually exists or not. It is true that unless there really is such an Object, the Icon does not act as a sign ; but this has nothing to do with its character as a sign. Anything whatever, be it quality, existent individual, or law, is an Icon of anything, in so far as it is like that thing and used as a sign of it. 

An Index is a sign which refers to the Object that it denotes by virtue of being really affected by that Object. It cannot, therefore, be a Qualisign, because qualities are whatever they are independently of anything else. In so far as the Index is affected by the Object, it necessarily has some Quality in common with the Object, and it is in respect to these that it refers to the Object. It does, therefore, involve a sort of Icon, although an Icon of a peculiar kind ; and it is not the mere resemblance of its Object, even in these respects which makes it a sign, but it is the actual modification of it by the Object. 

A Symbol is a sign which refers to the Object that it denotes by virtue of a law, usually an association of general ideas, which operates to cause the Symbol to be interpreted as referring to that Object. It is thus itself a general type or law, that is, is a Legisign. As such it acts through a Replica. Not only is it general itself, but the Object to which it refers is of a general nature. Now that which is general has its being in the instances which it will determine. There must, therefore, be existent instances of what the Symbol denotes, although we must here understand by " existent," existent in the possibly imaginary universe to which the Symbol refers. The Symbol will indirectly, through the association or other law, be affected by those instances ; and thus the Symbol will involve a sort of Index, although an Index of a peculiar kind. It will not, however, be by any means true that the slight effect upon the Symbol of those instances accounts for the significant character of the Symbol.

[6] Firstly, the content plane of images is not stratified into semantics and grammar, so the discourse semantic vs grammatical distinction does not apply. Crucially, if images did have a grammar, it would be possible to read them aloud — to verbally project locutions — as is possible for written texts.

Secondly, neither of the pretexts for relabelling Kress & van Leeuwen's original terms withstands close scrutiny. On the one hand, the original distinction 'overt/covert' is simpler, not more "cryptic", than 'explicit/implicit'. On the other hand, the original distinction 'concrete/abstract' is not ambiguous in this context, but, more importantly, it is consistent with SFL Theory, whereas Peirce's 'iconic/indexical/symbolic' distinctions are not.

Moreover, rebranding other people's work to get credit for their ideas is Martin's modus operandi, as demonstrated on this blog, as well as on other blogs here and here. For example, Martin (1992) rebrands Halliday's speech function as his negotiation, rebrands Halliday & Hasan's (1976) cohesion as his discourse semantics, rebranding their cohesive reference as his identification, their lexical cohesion as his ideation, and their cohesive conjunction as his conjunction (now 'connexion').

More recently, Martin and his colleagues have rebranded Cléirigh's model of gestural and postural semiosis as their model of paralanguage, incongruously rebranding linguistic body language as "sonovergent" paralanguage, and epilinguistic body language as "semovergent" paralanguage, on the pretext that the (wrongly conceived) meaning of these invented words is more transparent. Evidence here.

The Authors' Analysis Of An 'Activity-Focused' Image

 Martin & Rose (2007: 323, 325):

The inauguration photo construes a simple activity, in which the crowd is looking up to the stage and across to the left, underneath a huge flag. Within this activity however, the image could also be interpreted as implicitly classifying the ordinary people in the lower foreground, separate from the dignitaries above them on the stage. The central flag can then be interpreted as mediating these categories, representing the superordinate category of the nation. The flag itself is a compositional image, in which the categories of South African peoples and histories that it symbolises are implicit. That is, the red, white and blue refer to the pre-apartheid era British flag, and black, green and yellow to the flag of the African National Congress, all converging from the past towards the future.

In sum, these photos illustrate the four ideational categories we have suggested for images: classifying or compositional entities, and simple or complex activities.

 

Blogger Comments:

[1] To be clear, the claim here is that this photograph is 'activity-focused' (on 'looking') rather than 'entity-focused'. However, Martin & Rose provide no rationâle, either in terms of content or expression, for this claim that it is the activity of looking that is under focus.

It might be alternatively argued that the photograph is 'entity-focused', since its most salient element is the new South African flag which is highly relevant to the represented occasion. Moreover, the authors themselves gloss the image (p323) as an entity, not an activity: 'new South African flag, in the crowd, at the inauguration'.

[2] To be clear, this demonstrates the arbitrariness of the authors' framework. On the one hand, having classified the photograph as 'activity-focused', Martin & Rose demonstrate that it can be just as easily classified as 'entity-focused'. On the other hand, their interpretation of the image as classifying 'ordinary people' as lower and separate from the dignitaries 'above' runs counter to the new social equality that the occasion celebrated. 

Alternatively, the photograph can be analysed, in terms of the textual metafunction, as foregrounding (highlighting) the crowd and backgrounding the dignitaries, which is more in keeping with the celebration of the liberation of the powerless from the powerful.

[3] Trivially, this mistakes meronymy for hyponymy. To be clear, 'nation' is not a superordinate ('hypernym') of 'ordinary people' or 'dignitaries' because these latter are not subtypes (elaboration: hyponymy) of 'nation'. On the other hand, 'nation' can be interpreted as comprising (extension: meronymy) both 'ordinary people' and 'dignitaries'.

[4] To be clear, unknown to Martin & Rose, in pointing out the relevance of the flag to the occasion, they have provided a cogent argument for interpreting the flag as the element under focus, and for interpreting the photograph as 'entity-focused', rather than 'activity-focused', in their framework.

[5] To be clear, Martin & Rose have not illustrated their notion of complex activities, and their simple activities were those of saluting ('entity-focused' image) and looking ('activity-focused' image). Moreover, their 'classifying' is merely the identification of a depicted entity, and 'compositional' merely acknowledges the fact that entities have parts.

The Authors' Analysis Of An 'Entity-Focused' Image

 Martin & Rose (2007: 323, 324-5):

In this framework, the photo of the young boy is a classifying image. In nuclear terms, we have a young black boy (central), with his hand raised in a fist (nuclear), in front of a crowd (peripheral). But from the perspective of field, each of these elements has at least two possible referents. The boy at once represents the past roles of black youth in the anti-apartheid resistance, and their future lives in a free South Africa. His ‘black power’ salute evokes both the schoolchildren’s historical protests against the regime, and the celebration of Inauguration Day. In his 1995 edition, Mandela refers to this gesture as the Afrika salute (in photos between pp402-3), underlining its function in indigenous solidarity. And the boy can be construed both as a member of the crowd, and as its embodiment — a leader in other words, implying a connection with Mandela. 

The potential ambiguity of visual images is part of their power: their interpretation is left relatively open to the viewer, widening their appeal, and their multiple interpretations can map onto each other in the manner of metaphors, to evoke more general or abstract categories than the simple images they depict (cf. the discussion of grammatical metaphor and mode, in section 9.1 above).

Blogger Comments:

[1] To be clear, the claim here is that this photograph is a classifying image because it is 'entity-focused' rather than 'activity-focused'. However, Martin & Rose provide no rationâle, either in terms of content or expression, for this claim. It might be alternatively argued that the photograph is 'activity-focused', since the 'Afrika salute' is highly relevant to the represented event, as well as being what chiefly distinguishes this image of a boy from other images of boys.

[2] To be clear, this is not an argument for the claim that the photograph is 'entity-focused' because it analyses its content in terms of ideational meaning (nuclearity) rather than textual meaning (focus); see previous post. Moreover, the analysis, which interprets the entity as if it were a nominal group, is inconsistent with the authors' own model (p98):

If this model is applied to the authors' verbal description of the photograph, then black boy is the centre (Classifier Thing), young is nuclear (Epithet), and both with his hand raised in a fist and in front of a crowd are peripheral (Qualifier).

[3] To be clear, in SFL Theory, such "referents" are Values that are realised symbolically by Tokens.

[4] To be clear, these reinterpretations of the meanings of the photograph, the boy and his raised fist, are reconstruals of them as metaphorical realisations of a higher level, more congruent meaning; see Halliday & Matthiessen 1999: 272, 289).

[5] This is a non-sequitur. If the boy "embodies" the crowd, then he "embodies" the opposite of a leader. As such, the photograph does not imply a connection with Mandela in this sense.

[6] Trivially, this ambiguity could also narrow the appeal of an image to viewers, as when one possible meaning of an image is interpreted as offensive to some members of a community.

Mistaking Textual Meaning For Ideational Meaning

Martin & Rose (2007: 323-4):

In ideational terms developed in Chapter 3 above, the primary focus of a visual image is either on entities or on activities. Entity-focused images either classify them or compose their parts; activity-focused images construe either a single activity (simple) or an activity sequence (complex). As we discussed for genres in Chapter 8, images may also have secondary foci realised by their elements.

Blogger Comments:

[1] Here Martin & Rose mistake textual meaning for ideational meaning. To be clear, it is the textual metafunction, not the ideational metafunction, that is concerned with the focus of information, the highlighting of ideational (or interpersonal) meaning.

[2] To be clear, these are all bare assertions, unsupported by argument, and their validity will be assessed in the text analyses that follow. For the moment it can be noted that the authors make no acknowledgement of the distinction between content and expression, nor of how the textual focus on one type of ideational meaning, rather than another, is expressed.

[3] To be clear, in terms of SFL Theory, this misconstrues the relation between textual meaning ('foci') and ideational meaning ('elements') as realisation, despite the fact that both are of the same level of symbolic abstraction (meaning). (Realisation is the relation between different levels of symbolic abstraction.)

Confusing Material And Semiotic Phenomena In Misunderstanding Context

Martin & Rose (2007: 321-2):

In our discussion of mode above we talked about the way in which the exophoric references in Vincent Lingiari’s speech made it ‘context dependent’ — dependent on our being there or on reading images of what was going on. Another way of putting this would be to say that more than one modality was involved, using the term modality here in the sense of a modality of communication such as language, music, image or action. To understand Lingiari, in other words, we need to process language in relation to image, or language in relation to action. There are two modalities co-articulating what is going on. In register terms what this suggests is that we need to expand our conception of mode to embrace multimodal discourse analysis (hereafter MDA). This entails moving beyond linguistics into social semiotics and taking into account as many modalities of communication as we can systematically describe.

Blogger Comments:

[1] To be clear, Martin & Rose model context as register and genre (varieties of language), but here contradict their own model by using 'context' to mean both the first-order material setting ("being there") and the second-order co-text (images).

[2] To be clear, by 'action' here, Martin & Rose mean the material setting. That is, they misconstrue first-order material experience (phenomena) as second-order semiotic experience (metaphenomena). Meanings projected from the material setting are another matter.

[3] To be clear, despite conventional opinion, music itself — unlike musical theory, notation and lyrics — is not a semiotic system. If it were, it would have long been possible to construct system networks of the meaning contrasts realised by sound contrasts. Instead, as a perceptual phenomenon, music potentially induces mental processes of emotion, desire and cognition.

[4] To be clear, inconsistent with the opening sentence, here Martin & Rose return to their own model of context, which misconstrues culture as functional varieties of language (registers).

[5] Trivially, SFL Theory models language as a social semiotic system. That is, the linguistics deployed by Martin & Rose is already within social semiotics.