The great debate about the use of student experience surveys in higher education has again reared its ugly head. Merlin Crossley from UNSW recently pointed out, in a News Corp editorial, that although flawed, student experience surveys still provide the best indicator of teaching quality in higher education. While there is undoubtedly value in surveying students to determine their level of satisfaction, this argument misses the point when it comes to quality teaching.
The trouble with student experience
Much of the focus of learning and teaching in higher education in the last decade has been on the student experience. Improving student experiences at university is undeniably a worthwhile cause. Students should absolutely expect to be satisfied with their experiences at university and feel supported by their teachers and the institution.
The reasons why students might be satisfied or dissatisfied, however, often have nothing to do with either the quality of their learning or the quality of the teaching. Asking students to self-report on their perceptions of their learning is fraught for well-known reasons and for some less well known.
As has been widely researched and reported, there are fundamental biases that influence the ways in which students respond to surveys about their learning experiences, no matter how targeted the questions. These biases tend to disadvantage teachers who are women, are less experienced or are perceived as less attractive.
What has received less attention is that students also tend to be poor at judging their own learning progress. There is a lot of evidence suggesting that students are relatively accurate at predicting how well they will perform on assessment. This accuracy does not translate to progress. Learning is about ongoing development, not just about a capacity to perform. Therefore, the inability of students to accurately judge how well they are progressing is a problem for them, for teachers and for leaders in higher education.
If teaching is aimed at helping students to learn, rather than just perform on an assessment task, there are serious questions to be asked about the capacity of student evaluations to assess teaching quality. Even if the instrument provides a sense of how well students think their experiences have prepared them for an exam, this does not mean they will be able to use the knowledge meaningfully in the future. It is just as likely that the instrument is evaluating how confident students are in their knowledge. Confidence may not correlate at all with how much they have actually developed as emerging professionals, scholars or scientists.
Excellence vs expertise in teaching
From the perspective of leaders and teachers, there is a clear issue with relying on student surveys to determine teaching quality. What is perhaps more troubling is that, even if the instrument does really assess how confident students are about their performance on an exam, this is a poor and potentially misleading indicator of teaching quality.
Research is increasingly demonstrating the importance of failure and struggle in the learning process. Students commonly interpret struggling as a negative experience. Many graduates, when asked, can recall a subject they struggled through that, in hindsight, was one of the most transformative experiences of their life. At the time though, it was probably interpreted negatively. Was it a quality learning experience or not?
If teaching excellence is determined by student experience, there remains a possibility of excellence being defined by popularity, gender or attractiveness. It could also be influenced by how easy students find the learning experience. None of these factors are necessarily good indicators of quality teaching, with the latter now being seen as a possible indicator that a student has not engaged with the material deeply enough.
One caveat here, of course, is that it is patently counterproductive for students to struggle with understanding the curriculum, with the requirements of learning and assessment tasks, or with the reasons why they are being asked to do what they do. These should be as straightforward as possible and made perfectly clear to students. The acquisition and updating of deeper conceptual understanding, on the other hand, does often require grappling with new ways of thinking and synthesising knowledge. This is hard cognitive work for most people.
As an alternative to questionable instruments like student evaluations as the key indicator of quality, other indicators can be more systematically employed. Teaching is a profession, a domain of expertise, and should instead be determined, as is the case in other professions, by what the evidence says about quality. Progress in the learning sciences, in educational research and through scholarly evaluations is consistently providing meaningful advances in this evidence base. Excellent teaching needs to be determined and evaluated through expert review and recognition of world-leading practices that demonstrably enhance student thinking and understanding. Student perceptions of these practices are grossly ineffective for this purpose.
The tendency to recognise excellence as perceived by students over expertise and scholarship of learning and teaching creates some risks for the sector. When perceived excellence is the goal, practices are driven by the need to improve the student perceptions over student learning. While supporting student experiences is an important outcome in and of itself, that does not mean it should be intricately bound to quality teaching. Doing so risks veering towards a dangerous market-driven approach where naive perceptions of quality trump actual, evidence-based practices.
An analogous situation would be if you were to build a gym based on surveys of gym-goers’ satisfaction alone. You would end up with a coffee shop full of people wearing lycra. As with a gym, the aim is not just to make people happy but to get results. Unfortunately, increases in learning are not as obvious as increases in fitness, hence the difficulty in determining quality teaching.
Quality teaching leads to quality learning
Supporting students and facilitating effective learning are required of every teacher in higher education. However, if teachers are not also adequately helping students to learn how to cope with ideas and concepts that are difficult to grapple with, then it is a disservice to students. The problems that graduates face in the 21st century are complex and difficult. The most transformative learning experiences are also often the most difficult.
The student experience is important but learning is equally, if not more, important. Expertise in learning and teaching need to both inform and evaluate the efficacy of practices, not just students and not just survey instruments that are poor indicators of quality learning and teaching. It is time to treat the student experience, quality student learning and quality teaching in higher education as the separate constructs that they are. The evaluation system currently relies predominantly on one instrument to try to evaluate satisfaction with the student experience, quality student learning and effective teaching all at once. The result is that student experience surveys don’t generally assess any of them well.
There is absolutely a place for determining whether students have a positive experience at university, and there must continue to be so. It is also critical that there is a mechanism for determining how well university teachers support students in their courses. However, it is worth considering the implications of continuing to rely on these surveys as indicators of quality teaching. While there may be a relationship between student satisfaction and effective facilitation of their learning, they are not the same thing. There are better options available for determining teaching quality based on evidence and expertise about practices that demonstrably lead to quality student learning.
Dr Jason Lodge is a senior lecturer at UniMelb’s Melbourne Centre for the Study of Higher Education.Do you have an idea for a story?
Email [email protected]