Asian Journal of the Scholarship of Teaching and Learning

The Asian Journal of the Scholarship of Teaching and Learning (AJSoTL) is an international, peer-reviewed, open-access, online journal. AJSoTL seeks to create and nurture a global network of academics and educators who will discuss ongoing changes and future trends in tertiary education.

Victor TAN - Department of Mathematics, Faculty of Science, National University of Singapore

YAP Von Bing - Department of Statistics and Applied Probability, Faculty of Science, National University of Singapore

CHNG Huang Hoon - Department of English Language and Literature, Faculty of Arts and Social Sciences, National University of Singapore

Address for Correspondence: Associate Professor Victor Tan, Department of Mathematics, National University of Singapore, Block S17, 10 Lower Kent Ridge Road, Singapore 119076

Email: mattanv@nus.edu.sg

TAN, V., YAP, V. B. & CHNG H.H. (2015) SoTL Practices in Mathematics Classrooms: Promoting Student Interest and Learning Outcomes in Mathematics. *Asian Journal of the Scholarship of Teaching and Learning, 5*(2), 124-132.

A review of *Doing the Scholarship of Teaching and Learning in Mathematics^{1}* (2015) Jacqueline M. Dewar and Curtis D. Bennett (eds.). USA: The Mathematical Association of America (Incorporated).

The book, *Doing the Scholarship of Teaching and Learning in Mathematics*^{2}, is a wide-ranging resource book. This 20-chapter book is a useful resource for different types of scholars, particularly for reflective teachers, or scholarly practitioners, or those who are newcomers to the scholarship of teaching and learning (SoTL) and who may wish to know "What is SoTL and why do SoTL?"; "What are the benefits (and challenges) of conducting SoTL research?"; "How does one get started on a SoTL investigation?" Doing SoTL in Math is also an important work for mathematicians, more specifically, mathematicians who are deeply invested in educating and engaging undergraduate students about the ‘exciting’ world of Mathematics, what Mathematics is, and what Mathematics can do.

More generally, this work also foregrounds critical issues and challenges encountered in undertaking any research process – the issues relating to methods and methodologies, particularly with random sampling and control groups; institutional review boards’ oversight and clearance on involving human subjects in research; and the replicability and generalizability of one’s research findings, particularly when the research undertaken relates to classrooms, students, and education.

In this short review, it is impossible to do justice to all these complex issues. We shall therefore not try to be all-encompassing in our discussion here, but will instead focus on some very interesting lessons learnt from reading about the 15 classroom experiments conducted by colleagues teaching Mathematics to undergraduate students in the US. We will draw on a number of examples from these 15 experiments to highlight specific points of interest. We will also attempt to offer a critique of some of these experiments, not so much as to belittle the tremendous effort put in by these scholars, but more in the spirit of advancing our pedagogical techniques in active student engagement and learning outcomes in a potentially difficult fundamental subject like Mathematics.

Many useful lessons and strategies may be gleaned from the 15 experiments assembled in this book as "Illustrations of SoTL Work in Mathematics" (p.51):

We begin with Chapter 7, "A quantitative and qualitative comparison of homework structures in a multivariable Calculus class" by Lynn Gieger, John Nardo, Karen Schmeichel, and Leah Zinner (pp.67-76), where we note the following:

Most Mathematics instructors are concerned about students not having enough practice to master the relevant skills for their Mathematics courses. These instructors tend to address this concern by setting homework assignments. However, there are some issues regarding such assignments. For example, if the homework is not graded, students may not have sufficient motivation to complete them. If it is graded, students also have the tendency to work on their homework only when submission dates are approaching. If students find mathematical problems challenging, they may end up copying friends’ homework to beat the deadline. There are also problems with the grading of homework. If the class size is small, the instructor probably can handle the grading him-/herself or easily find a competent TA to do the job. However, for large classes, instructors have to rely on TAs to help with grading, and often face quality control issues. It is essential that graders go through homework carefully and provide sufficient relevant feedback to the students. But often, graders would just mark out ticks, crosses and question marks with minimal written comments. Even when comments are given, the quality of the comments may not be good.

In a project carried out by an interdisciplinary team consisting of four faculty members from different departments (mathematics, education, biology and psychology), the effectiveness of an online homework system in a calculus class was assessed. Their investigation yielded two main findings. With an online homework system, students were working on homework more frequently and practicing problems on a more regular basis. They also preferred online homework over pen-and-paper assignments as the former could provide them with more timely feedback. However, students were concerned that use of the online system could "affect their personal connection to the instructors" (p.72). Such findings provide insights for Mathematics instructors who may like to explore alternative ways of motivating their students to do more practice exercises.

We found this project to be an excellent illustration of SoTL work as there is a clear demonstration and articulation of the various aspects of this type of research. First, there was a discussion of how the research questions were formulated followed by a literature review. Then, the research methodology was described in great detail. What is noteworthy was the employment of both quantitative (surveys) and qualitative (focus group interviews) approaches in data collection. As mentioned by the authors, "due to [their] small sample size, [their] statistical power to detect significant differences was limited. [They hence] turned to qualitative analysis to gain further insight" (p.71). They further asserted that "the SoTL approach performed in [their] study demonstrate[d] the successful and rapid implementation of a study whose data, although limited in scope, can inform an instructor about instructional changes aimed at bolstering student success" (p.72). There was a discussion of how the team analyzed the qualitative data, which is something new to most STEM (Science, Technology, Engineering, Mathematics) researchers. The informative summary of the coding process in the "constant comparative analysis" should be useful to readers interested in applying a similar research design. Using this study as an example, interested instructors may embark on their own research by applying similar approaches to higher level Mathematics classes on other topics with different contexts.

Gieger et al. also identified the limitations in carrying out the project, including the small class size and infeasibility of having a control group. One of the authors who was also the course instructor found it difficult to "give up control and knowledge of the data for the duration of data collection [as he] did not see any of the surveys or focus group data until after the students’ course grades were submitted" (p.72). This underscores the importance of ethical considerations when carrying out SoTL work. The authors also gave useful advice to instructors, in particular faculty members on teaching tracks, on how "they can utilize basic skills of experimental design, implementation, and analysis to pursue research into effective teaching methodologies" (p.73) i.e. by engaging in SoTL. Finally, the authors also shared with the readers the benefits of working as a multidisciplinary team. One aspect is the increased awareness of the requirements of different disciplines, which "can inform future syllabi designs and promote effective articulation of anticipated goals and skills" (p.73).

Besides individual chapters, some sets of studies organised around particular themes also proved to be instructive. One set of chapters that we found very useful is the one loosely organised around a common pedagogical goal, namely to engage students in thinking deeply about Mathematics and Statistics through experiencing real world problems, i.e. Chapters 10, 11, 12. The authors tried hard to find evidence supporting the efficacy of their interventions, and in the process grappled with methodological issues which commonly arise in SoTL. In our view, their strategies seem to show promise for the accomplishment of their pedagogical goals. Implementing these approaches in a variety of situations can help further validate their findings, and may eventually result in improved pedagogy.

The first of these chapters, Chapter 10 on "Using SoTL to assess the outcomes of teaching Statistics through civic engagement" by Cindy Kaus (pp.99-105), was partly spurred by the project, Science Education for New Civic Engagements and Responsibilities, in which civic engagement was incorporated into a general education course on statistics. In Kaus’ study, the treatment group got to do civic engagement projects that were presented to the community, while the control group had ample but indirect exposure to civic engagement, driven by ethical considerations. Briefly, students in the treatment group took the course in 2007 and 2008, and they brainstormed topics, formed groups, gathered and analysed data, wrote a paper, developed a poster, and wrote an action letter to a person or organization that could benefit from the project. An example was the action letter sent to a senior citizen group questioning whether "Does where you live affect the price you pay for prescription drugs?" In contrast, the control group is comprised of a group of students who took the 2010 course, which featured only classroom discussion of civic, social and environmental issues in all course activities.

Kaus had two questions: did the innovation increase students’ confidence in doing and communicating statistics, and did it increase retention. To answer them, she used two surveys tools: the Student Assessment of Learning Gains (SALG) both before and after the course, and the Institutional Improvement Questionnaire (IIQ) of her university. It was found that students engaged in community-based projects showed larger gains (difference between responses before and after the course) across 13 questions of SALG, compared to students who did not enage in such projects. On the IIQ, 48% of students responded that the course activity which "helped [them] learn the most" was "group projects". It seems clear then that active engagement in real world issues improves retention and can increase students’ confidence in learning statistics.

In Chapter 11, "A pedagogical odyssey" by Michael C. Burke (pp.107-116) started with trying to help students gain a deeper understanding of the mathematical object "function". This chapter describes some insightful pedagogical challenges in teaching the connection between theory and reality, to an extent not often seen, even in statistical contexts. (Although Burke taught calculus, many of his ideas are readily portable to the teaching of statistics.) When he posed the following question which showed that the Irish population was increasing exponentially from 1700, 1750, 1801, to 1841, and invited students to find a formula (a mathematical model), and to predict the numbers in 1851, 1901, etc., he found that the students became confused when they were told that the potato famine caused a catastrophic crash in the Irish population. Their projections did not reflect what really happened historically and they did not how to deal with the breakdown of their self-constructed mathematical model. He observed that although "[they] could ...use their models to make projections... my students had a great deal of difficulty when they tried to reach a conclusion ... in some cases, they did not even believe the work that they had done" (p.109-110).

Nonetheless, despite their initial bewilderment, he found that these students’ scores on the final examinations were quite similar to students who were taught the conventional way, and concluded that "it was possible to augment the standard mathematics curriculum with writing assignments of this type without compromising mathematics instruction and student learning" (p.110). The whole experience led him to conclude that besides "embed[ding] the mathematics I was teaching into compelling, important and authentic contexts" (p. 114), he needed to "fully embrac[e] the complex teaching challenges inherent in teaching through these contexts" (ibid.). He testified that to "make significant changes ... to understand what is happening in [the] classroom, [a teacher] must begin with observation."

As John Holcomb’s "Presenting evidence for the field that invented the randomized clinical trial" (Chapter 12, pp.117-123) noted, it is not uncommon for students to do one or two projects involving data analysis and report writing in a Statistics course. Holcomb set the bar even higher by assigning six group projects based on a North Carolina health survey data (instead of the usual one or two projects) in an introductory course. After publishing this approach, he was encouraged to think more deeply about how to assess whether students have learnt skills such as data summary and the entire process of hypothesis testing. Realising that these skills are hard to assess without a computer, which is often absent in the examination hall, he added take-home components in the mid-term and final examinations as another iteration of the introductory course. Examination scores were used to gauge learning outcomes, although the details were not reported in this chapter. Surveys on student attitudes towards statistics and teamwork were also administered. It was found that 66% of students reported having a positive experience working with their project mates.

Holcomb states, "As a statistician, I was trained to think that the only way to prove that an outcome is caused by a treatment is to implement a randomized experiment," (p.118) a view that appeared to be shared by reviewers of journals which refused to publish his work. But this view is too strict. While it is true that non-randomized studies are prone to confounding results, many important causal relations have been established using a large number of studies with concordant results in various contexts. One well-known example is the health effects of smoking: there was not a single randomized experiment on human beings.

Both Holcomb and Kaus deliberated carefully on ethical and other issues related to a control group. In Chapter 12, on page 119, John Holcomb states "… giving students an assignment of this magnitude and of such stakes without providing them with sufficient practice, in the form of homework, would be unfair and unethical". He also acknowledged that "as the instructor of both sections, I could, albeit unconsciously, sway the results in the direction I wanted" (p.119). As a truly randomized experiment is often impossible in SoTL studies, we face the likely possibility that the treatment group differs from the control group in important ways, i.e. there can be unknown confounders. While formal statistical tests of hypotheses can be performed, the results, including the P values, can be hard to intepret^{3}. More resources should be directed to understanding confounders and controlling for them in studies in a wide variety of situations.

Yet another interesting set of examples is presented in the next theme which looked into the use of pre-class assigned readings and reading questions in Statistics in Chapter 13 (by Derek Bruff) and in Mathematics in Chapter 14 (by Mike Axtell & William Turner). Sharing the same concern that Gieger et al. had on student preparedness for class, Bruff and Axtell & Turner documented their experiments using pre-class assigned readings and reading questions. Both sets of authors asked if pre-class questions were doing anything to help students understand the subjects, and if yes, what kinds of questions (definitional, conceptual or computational) are beneficial for facilitation of student learning.

A key lesson that we took away from these chapters is that pre-class computational questions that require students to work through a problem are the most effective in enabling learning, compared to both pre-class definitional questions that only require regurgitation of textbook materials and conceptual questions that asked students to mimic a solution from the text. This was because solving computational problems not only provided hands-on experience that promoted knowledge retention, it also aligned well with the preferred inductive learning style of the engineering students in this class. To put it simply, students could not "fake [their] way through a computational problem" in the way that they could easily and accurately reproduce solutions to other kinds of problems, without learning a conceptual idea from a text (p.133). This need to move to move away from definitional or conceptual levels towards hands-on problem solving or application of knowledge is an important lesson that applies not just to teaching Mathematics or Statistics but is most likely true of other subject learners as well.

The second key lesson was that these chapters epitomised a key SoTL principle, which is to "go public" (Shulman 1993) with one’s work so that, as the editors Dewar and Bennett have noted, "others can build on it" (Dewar & Bennett, 2015, p.127). These two chapters not only share the use of readings and reading questions, one of them – the Bruff experiment – in fact came about because of the Axtell-Turner work. The authors clearly underscore the point that conducting relevant literature survey in doing SoTL is beneficial as such a survey of the field allows us to learn from others who have worked in similar domains. Teaching and learning practices should not be locked up within the privacy of classroom walls. Instructors ought to heed the SoTL exhortation to document SoTL research into teaching practices and disseminate such work in the public domain, in the same way that disciplinary research has accepted external peer review as a validation practice.

This is a most interesting resource book comprising of a generous sharing of the trials and tribulations of committed teachers who put effort into improving student learning outcomes through active, deep engagement. Although we have only highlighted 6 chapters in this review to serve as exemplifications of the usefulness of doing SoTL in a subject like Mathematics (and Statistics), what we felt about this work also applies to the other chapters that we did not comment on directly. We admire this book for its collegial spirit in sharing the what, how, and why of doing SoTL in Mathematics.

The lessons learnt – in working through methods and methodologies, handling data and human subjects, and in articulating the SoTL questions in each undertaking – all these and more will prove to be a great resource for anyone who may be thinking of starting out on their own SoTL journey, not just in Mathematics or Statistics, but in any subject for that matter. Perhaps Derek Bruff puts it best when he concluded that: "Every course I teach presents similar opportunities to look across student work and identify patterns in their learning that can inform my teaching choices. The purpose of teaching is to promote student learning. Asking questions about student learning and attempting to answer them by collecting and systematically analyzing evidence of student learning, particularly qualitative evidence, equips me to fulfill that purpose more effectively" (p.135). This is why we should conduct SoTL research in our classrooms, and this book functions as a potentially useful contribution to the literature review of any budding SoTL researcher.

Axtell, Mike and William Turner. (2015). "An investigation into the effectiveness of pre-class reading questions". In Jacqueline M. Dewar and Curtis D. Bennett (eds.), *Doing the Scholarship of Teaching and Learning in Mathematics* pp.137-142. USA:The Mathematical Association of America (Incorporated).

Bruff, Derek. (2015). "Conceptual or computational? Making sense of reading questions in an inverted Statistics course". In Jacqueline M. Dewar and Curtis D. Bennett (eds.), *Doing the Scholarship of Teaching and Learning in Mathematics* pp.127-135. USA:The Mathematical Association of America (Incorporated).

Burke, Michael C. (2015). "A pedagogical odyssey". In Jacqueline M. Dewar and Curtis D. Bennett (eds.), *Doing the Scholarship of Teaching and Learning in Mathematics* pp.107-116. USA:The Mathematical Association of America (Incorporated).

Dewar, Jacqueline M. and Curtis D. Bennett (eds.). (2015) *Doing the Scholarship of Teaching and Learning in Mathematics*. USA: The Mathematical Association of America (Incorporated).

Gieger, Lynn, John Nardo, Karen Schmeichel, and Leah Zinner. (2015). "A quantitative and qualitative comparison of homework structures in a multivariable Calculus class". In Jacqueline M. Dewar and Curtis D. Bennett (eds.), *Doing the Scholarship of Teaching and Learning in Mathematics* pp.67-76. USA: The Mathematical Association of America (Incorporated).

Holcomb, John (2015), "Presenting evidence for the field that invented the randomized clinical trial". In Jacqueline M. Dewar and Curtis D. Bennett (eds.), *Doing the Scholarship of Teaching and Learning in Mathematics* pp.117-123.USA: The Mathematical Association of America (Incorporated).

Kaus, Cindy (2015), "Using SoTL to assess the outcomes of teaching Statistics through civic engagement". In Jacqueline M. Dewar and Curtis D. Bennett (eds.), *Doing the Scholarship of Teaching and Learning in Mathematics* pp.99-105. USA: The Mathematical Association of America (Incorporated).

Shulman, Lee (1993), "Teaching as community property: Putting an end to pedagogical solitude" in, 25, 6; November 1993, *Alt-Press Watch*, p.6. Accessed on 14 May 2015 at http://www.iub.edu/~tchsotl/part4/shulman%20community%20property.pdf

____________________________________________________________________

1 A number of chapters in this book are focused on Statistics, not Mathematics, although the majority of the experiments gathered here come from Mathematics.

2 Henceforth abbreviated to "*Doing SoTL in Math*".

3 The P value is the chance that the test statistic is more extreme than the observed test statistic calculated from data, assuming the null hypothesis is true. A smaller P raises more suspicion against the null hypothesis.

Do you have a view you'd like to share?