Asian Journal of the Scholarship of Teaching and Learning

The Asian Journal of the Scholarship of Teaching and Learning (AJSoTL) is an international, peer-reviewed, open-access, online journal. AJSoTL seeks to create and nurture a global network of academics and educators who will discuss ongoing changes and future trends in tertiary education.

Learning-Oriented Assessment and the Scholarship of Teaching and Learning: A Review of Excellence in University Assessment by David Carless

Book Review

Johan GEERTSEMA

 

Address for Correspondence: Assoc Prof Johan GEERTSEMA. Centre for the Development of Teaching & Learning, 15 Kent Ridge Road, Singapore 119225, Singapore.



Email:  johan.geertsemea@nus.edu.sg

Recommended citation:

Geertsema, J. (2017). Learning-oriented assessment and the scholarship of teaching and learning: A review of Excellence in University Assessment by David Carless. Asian Journal of the Scholarship of Teaching and Learning, 7(1), 83-90.

This book by David Carless, Professor of Education at the University of Hong Kong (HKU), will be invaluable for at least three groups of people. First, it is essential reading for all teachers who work in higher education, given that assessment drives learning: assessment tasks are “key drivers of [students’] efforts and learning approaches”, as Carless puts it elsewhere (2015b, p. 964). Second, those interested in the scholarly study of assessment and feedback will find a lot of thought-provoking analysis of relevant practices; moreover, the model of learning-oriented assessment that underpins the book is a powerful vehicle for thinking about this complex and important topic. And finally, because of the emphasis on making practice public and teaching visible, the book is valuable for thinking about ways in which to share and disseminate teaching practice so as to scale up quality enhancement.

Teaching is still too often considered to be the private affair of the teacher and her or his students, and the case studies that Carless presents and analyses—based on the practice of award-winning teachers at HKU—are illuminating in making the theoretical principles that underpin his book public, visible, and concrete. Moreover, it is evident that Carless does not impose his theoretical views on the teachers he studies, but instead draws inspiration from their practice. The book thus assists in not only providing a critical analysis of good and interesting assessment and feedback practices, but in disseminating these practices to a wider audience from within the disciplinary—Architecture, Law, History, Geology, Business—and also cultural contexts in which they are located: the disciplines of the practitioners, and also the wider contexts of Hong Kong and Asia.

In the Introduction, Carless sets the scene and presents his framework of learning-oriented assessment (Figure 1), which consists of:

  • Learning-oriented assessment tasks
  • Developing evaluative expertise
  • Student engagement with feedback


As Carless puts it, “the idea of learning-oriented assessment is that
all assessment should support the advancement of student learning” (2015a, p. 6). The book then develops the argument that “it is the interplay of these three elements which impacts significantly on the kind of learning which students derive from assessment processes” (p. 6), and that “student learning is deeply influenced by the three elements of learning-oriented assessment: the assessment tasks which they undertake; their development of evaluative expertise; and their engagement with feedback processes”. Assessment, feedback, and expertise in evaluation are—and need to be seen as—closely intertwined. Though the framework assumes a criterion-referenced approach, there is evidence that suggests Carless may not view it as entirely dichotomous to norm referencing, also known as grading on the curve (2015a, pp. 109, 165, 172).

One of the key points here is that two interconnected elements of the framework form the basis of, and as such provide support for, learning-oriented assessments tasks: developing evaluative expertise in the case of both teachers and students, which entails developing self-evaluative capabilities; and engagement by students with feedback. These two elements are essential for what Carless terms assessment literacy.

 

 

Figure 1. The learning-oriented assessment framework (Carless, 2015a, p6).


The book consists of four parts, which systematically elaborate on this framework in relation to assessment and feedback practices. At the end of each chapter Carless provides a summary of the key points, as well as main implications for practice—a hands-on strategy that helps to render the book very useful. Part I, “Learning and assessment”, introduces Carless’ notion of learning-oriented assessment, highlights the competing priorities of assessment encapsulated in the idea of ‘double duty’, and provides an overview of influential research on assessment and feedback. Part II, “Designing and implementing assessment tasks”, focuses on specific cases in History, Law, Geology and Business, while Part III, “Engaging with quality criteria”, considers the larger issue of how judgment works and how ‘connoisseurship’ develops: i.e. expertise in making judgments of value, with specific reference to the place of concrete exemplars and worked examples to illustrate quality. Finally, Part IV, “Reconceptualising feedback and ways forward”, identifies signature feedback strategies in particular disciplines and draws an analytical distinction between two feedback paradigms. The first is the traditional, conventional approach that sees feedback as monologic information transfer to students, which Carless contrasts with a more recent course of action that aims at greater sustainability in viewing feedback as dialogic interaction: more than merely providing comments on students’ work, but also engaging them in discussion of these comments in order to support learning and growth.

Two notable features of the analysis relate to the focus on actual academic teachers, and as importantly, the prominence of the student voice in the data. For example, Carless shows himself finely attuned to the Confucian-heritage context of the students when he considers group projects (2015a, pp. 50-55), and throughout the case studies that he presents he considers data collected from interviews with both students and their teachers. The book is therefore, as already intimated above, oriented towards authentic contexts and practices.

However, Carless is somewhat skeptical about the term authentic assessment: while he found

plenty of evidence from the cases that the teachers were promoting ways of thinking and practising which mirrored real-life uses of the discipline [and although]the concept of authentic assessment is useful, I remain unconvinced about the appropriateness of the term itself in that few forms of assessment are genuinely authentic because students would rarely do them in the same format if they were not being assessed for certification purposes. Accordingly, I prefer to use the term ‘assessment mirroring real-life uses of the discipline’ (2015a, p. 230-231).

Such authentic assessment may be more relevant to “soft-applied” and professional disciplines, as Carless notes (2015a, p. 64). Be that as it may, a key aspect of the featured practices—regardless of discipline, though there are of course disciplinary variations—is that they seek to engage students with the research literature and experimental data so as to connect what they learn not only with disciplinary protocols and conventions, but with real-world issues: by and large, these are instances of “contextualised assessment” (Carless, 2015a, p. 64). This is captured by the important concept of WTP, which Carless usefully highlights in his discussion of authentic assessment: “the phrase ‘ways of thinking and practising’ (WTP) in a subject area, [is used] to describe the richness, depth and breadth of what students might learn through engagement with a given subject area in a specific context” (McCune & Hounsell, 2005, p. 257). A key point that Carless makes is that WTP in the subject can be facilitated “by assessment tasks focused on real-life problems and issues contextualised within specific disciplinary situations” (Carless, 2015a, p. 63). This is because WTP emphasises not only engagement with literature and data, but crucially also students’ “growing mastery of the conventions of written and oral scientific discourse” (McCune & Hounsell, 2005, p. 256) that are relevant to the discipline they are studying. Students can see the point of the assessments tasks since such tasks support them in developing as participants in a disciplinary community. Fostering WTP in this way can thereby serve to result in increased student motivation and engagement with their own learning and that of others.

Fostering this kind of engagement is essential, given that assessment has a very strong impact on student learning, as noted at the start of this review, and thus we need to find ways of positively optimising that impact: this is what drives the idea of learning-oriented assessment, and what will help one deal with the competing priorities of assessment encapsulated in the classic notion of ‘double duty’. This notion holds that assessment always and inescapably has “functions other than the ones teachers and examiners normally think about and take account of” (Boud, 2000, p. 160), and it is one of the key challenges in developing good assessment practices that Carless considers in chapter 1. His discussion of the key challenges is framed by the recognition of, first, the centrality of assessment in education, and second its complexity given that it is about several things at once, with a number of competing priorities being required of it. Some of the ways in which assessment activities have to do double duty are that they have to encompass (formative) assessment for learning and (summative) assessment for certification; have to focus on the immediate task and on implications for equipping students for lifelong learning in an unknown future; and have to attend to both the learning process and the substantive content domain. But if, as Carless proposes, we see all assessment as always being learning oriented and thus having the duty of fostering learning, then this point of departure will make ‘summative’ assessment more ‘formative’: examinations and tests will then become opportunities for providing feedback, and their design a means of encouraging deep approaches to learning. Rather than asking students only to reproduce knowledge, a learning-oriented summative assessment design will ask students to identify and engage with real-life problems in the discipline (2015, p. 13), which goes back to the importance of WTP and contextualised assessment.

Aside from the obviously important discussion of learning-oriented assessment, a second notable feature of the book is how practice-based it is. After setting the stage in Part 1, the procedure in subsequent chapters is to focus on particular assessment and feedback designs as practised by actual academics. In each case, Carless provides context to the teachers concerned: information on their disciplines, their various approaches, and the classes they teach as well as their assessment and feedback practices. He highlights the rationale for the assessment task designs, and then analyses them in depth with reference to the literature on assessment discussed earlier (in chapters 1-3) as well as other relevant studies, focusing on how these practices exemplify learning-oriented assessment. In the discussion of each practice, he considers student perspectives on the design and sometimes, where appropriate, provides judiciously phrased suggestions for revising the design.

Carless provides meticulous descriptions and then thoughtful, generous analyses of practices in History (chapter 4), Law (chapter 5), Geology and Business (chapter 6), as well as the key issue of quality criteria and students’ engagement with it. In terms of the learning-oriented assessment framework, such engagement with quality is crucial for developing evaluative expertise. This is why assessment literacy is crucial for quality, not only in the case of the students being assessed (or engaging in assessment and feedback to peers or, relatedly, to teachers) but also, of course, in the case of the teachers themselves: what are the principles of good assessment and feedback, and how are they iteratively articulated in practice? Consequently, after providing a thought-provoking description of how important specific, discipline-oriented criteria are—i.e., ones that move beyond indicators of quality in relation to such generic attributes as ‘critical thinking’—Carless in the remainder of Part III discusses the value of peer review and the use of exemplars in clarifying criteria and thus quality to students (pp. 137-146), before examining the ways in which they engage with criteria in especially the History, Geology, and Architecture cases (chapters 8 and 9).

Not only does this approach make the book eminently practical and useful for academic teachers, but crucially, it moves teaching and learning out of the private domain of the classroom. The teachers whose work Carless discusses are thoughtful and reflective, and they are willing to share their practice and subject themselves, in essence, to peer review and feedback by the author and his research team. In this respect, the book exemplifies what the scholarship of teaching and learning is all about: a reflective approach by the teacher towards their teaching and towards student as well as faculty learning, an openness to peer feedback and critique, and a willingness to share practice by ‘going public’. To end this review, two points are worth noting in this respect.

First, with the exception of the teacher from Law (and of course, Carless himself, who is an expert in Education), none of the academics studied “can be said to specialise in learning-oriented assessment” (Carless, 2015a, p. 88). The implication of this is important: while it is true that, to improve and enhance practice, teachers’ awareness of the scholarly literature is crucially important—since reflecting on it will help to underpin practice with evidence-informed rigour to complement the ‘wisdom of practice’—there may be no need for these teachers to become specialists in particular areas of teaching and learning (such as assessment). This is to say that we need quite carefully to consider the appropriate level of engagement with learning theory expected of academics (Roxå et. al, 2008; Mårtensson et al., 2011), given that these academics are experts in their own disciplines, not in Education. Unrealistic expectations of the highest levels of expertise in learning and teaching may be counterproductive in threatening the identities, and indeed the good practices, of those who engage in high-quality teaching.

Second, in his discussion of the teacher from Law, Carless highlights a key challenge for SoTL. This teacher is diligent and innovative, a senior colleague who is respected and well-regarded. Yet while there “appears to be no overt resistance to his ideas from his colleagues … there seems to be limited interest or commitment to expand his ideas in their own classes” (Carless, 2015a, p. 106). As Carless points out, this is a major barrier to enhancement, as it militates against scaling up this teacher’s good practices beyond his own courses. What this point underscores, is the importance of culture change: sharing practice, as the teachers do whose practice Carless studies, is a necessary but not sufficient condition for change. The question that arises is what the ways and means may be of facilitating the spread of good practice: how best can SoTL foster the kind of widespread change needed for institutional enhancement beyond a small number of teaching award winners? One way may be to study good practice, as Carless has done, and then share it in the form of a thought-provoking, rigorous, yet practical book. But for such practice to spread, of course the book will need to be read and discussed. Ultimately, it is through fostering scholarly conversations about teaching and learning that we might move towards change across the board.



REFERENCES

Boud, D. (2000). Sustainable assessment: Rethinking assessment for the learning society. Studies in Continuing Education, 22(2): 151-167. http://dx.doi.org/10.1080/713695728

Carless, D. (2015a). Excellence in university assessment: Learning from award-winning practice. London: Routledge.

Carless, D. (2015b). Exploring learning-oriented assessment processes. Higher Education 69(6): 963–976. http://dx.doi.org/10.1007/s10734-014-9816-z

McCune, V. & Hounsell, D. (2005). The development of students’ ways of thinking and practising in three final-year biology courses. Higher Education 49(3): 255-289. http://dx.doi.org/10.1007/s10734-004-6666-0

Mårtensson, K., Roxå T., & Olsson, T. (2011). Developing a quality culture through the scholarship of teaching and learning. Higher Education Research & Development, 30(1), pp. 51–62. http://dx.doi.org/10.1080/07294360.2011.536972

Roxå, T, Olsson, T., & Mårtensson, K. (2008). Appropriate use of theory in the scholarship of teaching and learning as a strategy for institutional development. Arts & Humanities in Higher Education, 7(3), 276-294. http://dx.doi.org/10.1177/1474022208094412

 



About the Author

Johan Geertsema directs the Centre for Development of Teaching and Learning (CDTL).

0 Reply

Reply to this Discussion

Do you have a view you'd like to share?

Name*

Email*

Comments*

Verify*

capcha