Gender Bias in Student Evaluations of Teaching

by Denise Wilson, November 10, 2021

Student evaluations of teaching (SET) were originally designed to be formative by providing  valuable input to instructors in higher education. When used as a tool for improving teaching, student responses to close-ended and short-answer answer questions on SETs can provide helpful feedback to instructors as well as those who mentor or otherwise work with instructors on professional development. 

Since their initial introduction, however, SET  have continued to shift from formative to summative.  In many institutions, the ratings provide by students on a just a small subset of items far overshadow the short answers and other items that in total, provide a more comprehensive picture of college teaching.  The numbers within this subset of ratings are often used in key decisions for promotion , tenure, hiring, and firing. 

Using this shorter list of SET numbers is convenient and quick. Unfortunately, there is plenty of evidence that these ratings are biased and are not consistent or adequate measures of student learning (a short review of the literature can be found here). Gender bias, where female instructors consistently receive lower ratings than their male peers for the same courses or for different sections within the same course, is especially well documented. Such bias generates concern about how SET ratings are used by higher education institutions to evaluate women instructors and how these ratings impact the future morale and effectiveness of the women who read them and take them to heart.     

Recently, our research team compared student perceptions of how well faculty supported them in their courses with how those same students rated those faculty on student evaluations of teaching in engineering courses at one large public university. We compared a pair of median scores:  Instructor Effectiveness in the Course (SET) and Students' Sense of Faculty Support (our survey).

Instructor Effectiveness was measured using the median score from one item on the university SET form:  "The instructor's effectiveness in teaching the subject matter was:" Students had the option of selecting Excellent (5), Very Good (4), Good (3), Fair (2), Poor (1).  

Students' Sense of Faculty Support was measured in a research-based survey that was not affiliated with the university's educational assessment office.   Faculty support contained eleven items that had been validated in multiple previous studies in higher education and had a high internal consistency (reliability) of 0.92:

• The professor in this class is willing to spend time outside of class to discuss issues that are of interest and importance to me.
• The professor in this class is available when I need help.
• The professor in this class is interested in helping me learn.
• The professor in this class cares about how much I learn.
• The professor in this class treats me with respect.
• The professor has clearly explained course goals and requirements.
• The professor teaches in an organized way
• The professor often uses real-world examples or illustrations to explain difficult points.
• The professor often stays after class to answer questions.
• The professor often stops to ask questions during class.
• The professor is often funny or interesting.

All of the above items were rated on a scale from Strongly Agree (5) to Strongly Disagree (1).  

When we compared SET Instructor Effectiveness to Students' Sense of Faculty Support, we found that when asked a general question about instructor effectiveness, students exhibited negative bias toward women relative to more specific (and objective) questions about faculty support behaviors.  Students completed both the instructional support surveys (that contained the Faculty Support items described previously) and SET in the last 2-3 weeks of the term associated with the course being evaluated.   Four of the five female instructors (80%) in the study received higher Faculty Support ratings than their SET ratings would suggest while only three of the nine (33%) of the male instructors did so (female instructors are shown in red; male instructors in blue):

These results also show that students rated women lower in instructor effectiveness than men while still reporting that these same women offered them levels of support that were above the trendline for the entire dataset.    

What does this data really tell us?  First, it tells us that the correlation between general impressions of instructor effectiveness and more specific reports of faculty support is not particularly good.   Our data also reinforce the idea that women often receive lower SET scores than men for similar levels of instructional support and teaching quality.   Alongside other related research that demonstrates this type of negative bias against women, these results reinforce the call to rethink how we evaluate teaching among engineering faculty and instructors.  In a time when equity is central to the radar of many colleges of engineering around the country, gender bias in SET underscores the need to transform the conversation we have about student evaluations of teaching -- both those held in meetings with other faculty and administrators and those held inside one's head when SET reports deliver negative and deflating messages after a semester of long hours and hard work.  

As a country, we need excellence in teaching too much to compromise it by demoralizing good teachers with faulty rating systems. We can do better.

Have you had a similar experience to Our Words do Matter or would you like to share a story, concern, or experience that relates to what you have just read?  Click here to share (all responses are private and kept confidential).

Denise Wilson is a professor of electrical and computer engineering at the University of Washington in Seattle, Washington. Her research interests in engineering education focus on belonging, engagement, and instructional support in the undergraduate engineering classroom.   

Sexual Harassment Starts Early in Engineering

Sexual harassment doesn't always look like the obvious and egregious cases we read about in the the newspaper. But even "milder" forms of harassment can have lasting and serious effects. The first step to improving the culture is recognizing that nobody should have to "grow a thicker skin" or accept that "boys will be boys" in order to participate in engineering. Both women and men need to stop excusing the behavior.

When my daughter was in 9th grade, she wanted to take her high school’s Introduction to Technology course. I was of course supportive of her interest and encouraged her to sign up, thinking nothing more of it. A few weeks into the fall, she mentioned to me that she was the only girl in the class. I expressed surprise that this was still the case in this day and age. I recalled how thirty years earlier, I had been the only girl in an accelerated middle school math class, but that was a generation ago! Of course, I know that there is still a significant gender imbalance in engineering and technology, but surely not to such a degree, and not so early in the pipeline as 9th grade! I told her I was proud of her for being there and that I knew she was self-confident enough to manage that strange situation just fine. Which she would have been if it had only been a slightly awkward classroom context.

A few weeks later, she dropped some oblique comments about how a couple of the guys at her table grouping were being obnoxious – making some crude comments; teasing her; tugging on her hair. She didn’t seem particularly upset by it; just annoyed. So, I basically dismissed it. I agreed with her that it was obnoxious, then I told her to just ignore them. Told her that some high school boys can be real jerks before they manage to “grow up.”  I didn’t think much more of it, until the day she broke out in tears after school. This was out of character, and I suddenly realized something was really wrong.

It turned out that my daughter had just been following my lead. I had signaled to her that “boys will be boys,” and we just have to ignore it and get on with our own work. But I had failed to really listen to my daughter when she spoke up before. Perhaps she sensed from me that I have personally have more of a tendency to put up with bad behavior with only annoyance and dismissal. Or perhaps she was really embarrassed to relate the details. Whatever her thinking, she had been downplaying to me what had really been happening, until she just couldn’t anymore. 

I was horrified as she finally was able to share more of the details with me. It turned out that she was a target of really serious sexual harassment. There were not only dirty jokes intended to embarrass her – she found obscene images left on her desktop. The boys had progressed from touching her hair to bumping into her “accidentally.” There were two boys in the seating group of the four of them who were the active offenders, but the third boy knew what was going on and never made any effort to intervene.

Once I understood the scope and seriousness of the problem, I went immediately to her school. The response sounded right – the administrator promised swift discipline. The teacher expressed shock that this had been happening in his classroom; I confess I was a bit dubious about this and wondered how much deliberate obliviousness was involved to avoid dealing with a difficult classroom management issue. He offered to assign another student in the class to essentially be a chaperone – to be another set of eyes to watch out for my daughter and intervene when needed. Of course, this only served to set her apart even further, and reinforced her “victim” image with the class, as someone who needed extra support to handle the class. And I confess that in my ignorance, I overlooked subsequent situations in which I should have intervened on her behalf. I failed to anticipate all the ways the school would continue to fail her, including not protecting her from being scheduled in future semesters for class sections that included the offenders.   

I am proud that my daughter was mature and emotionally strong enough to be able to channel this devastating experience into positive outcomes. She educated herself by volunteering at the local shelter for victims of domestic abuse and she relentlessly advocated throughout the next four years to try to convince the administration to implement sexual harassment training for all students and teachers. And she has gone on to study engineering in college. But I am still devastated that the profession to which I have given my life had already failed her so miserably, and I am still devastated that I, who certainly should have known better, acted too slowly to be part of the solution. I had never personally suffered under behaviors this extreme. But it wasn’t just a case of not really recognizing what was happening – because I should have been stopping this even if things were a lot less severe. I think after my decades being “one of the guys” in so many settings, I had learned to go along to get along, and just ignore a lot. But it shouldn’t be that way – and if I am not speaking up, then I am part of the problem.

Now, a few years later, I have some perspective on the situation. I wish sexual harassment would have gone the way of the dinosaurs by now. But it hasn’t. And it occurs more often in engineering and technology settings than it does in general. So, regardless of my own experiences or observations of harassment, I hope that I now better recognize what I need to be doing. I need to recognize that just going along with it – the locker room behavior, the “just joking” moments – is unacceptable in any school or work setting. And my personal threshold for tolerance is not necessarily a good marker for what crosses the line. It is not too much to ask for civil and professional behavior from everyone in the engineering sphere: the managers, the engineering colleagues (men and women), the technicians and operators and tradespeople, the clients, the instructors (university and K-12), and the other students. In fact, it is the law. And we each have an obligation to protect, and even demand, that. For the sake of my colleagues. For the sake of the next generation of engineers. For the sake of the profession of engineering. We can all do better.

Have you had a similar experience to Sexual Harassment in Engineering, or would you like to share a story, concern, or experience that relates to what you have just read?  Click here to share (all responses are private and kept confidential).