Gender Bias in Student Evaluations of Teaching

by Denise Wilson, November 10, 2021

Student evaluations of teaching (SET) were originally designed to be formative by providing  valuable input to instructors in higher education. When used as a tool for improving teaching, student responses to close-ended and short-answer answer questions on SETs can provide helpful feedback to instructors as well as those who mentor or otherwise work with instructors on professional development. 

Since their initial introduction, however, SET  have continued to shift from formative to summative.  In many institutions, the ratings provide by students on a just a small subset of items far overshadow the short answers and other items that in total, provide a more comprehensive picture of college teaching.  The numbers within this subset of ratings are often used in key decisions for promotion , tenure, hiring, and firing. 

Using this shorter list of SET numbers is convenient and quick. Unfortunately, there is plenty of evidence that these ratings are biased and are not consistent or adequate measures of student learning (a short review of the literature can be found here). Gender bias, where female instructors consistently receive lower ratings than their male peers for the same courses or for different sections within the same course, is especially well documented. Such bias generates concern about how SET ratings are used by higher education institutions to evaluate women instructors and how these ratings impact the future morale and effectiveness of the women who read them and take them to heart.     

Recently, our research team compared student perceptions of how well faculty supported them in their courses with how those same students rated those faculty on student evaluations of teaching in engineering courses at one large public university. We compared a pair of median scores:  Instructor Effectiveness in the Course (SET) and Students' Sense of Faculty Support (our survey).

Instructor Effectiveness was measured using the median score from one item on the university SET form:  "The instructor's effectiveness in teaching the subject matter was:" Students had the option of selecting Excellent (5), Very Good (4), Good (3), Fair (2), Poor (1).  

Students' Sense of Faculty Support was measured in a research-based survey that was not affiliated with the university's educational assessment office.   Faculty support contained eleven items that had been validated in multiple previous studies in higher education and had a high internal consistency (reliability) of 0.92:

• The professor in this class is willing to spend time outside of class to discuss issues that are of interest and importance to me.
• The professor in this class is available when I need help.
• The professor in this class is interested in helping me learn.
• The professor in this class cares about how much I learn.
• The professor in this class treats me with respect.
• The professor has clearly explained course goals and requirements.
• The professor teaches in an organized way
• The professor often uses real-world examples or illustrations to explain difficult points.
• The professor often stays after class to answer questions.
• The professor often stops to ask questions during class.
• The professor is often funny or interesting.

All of the above items were rated on a scale from Strongly Agree (5) to Strongly Disagree (1).  

When we compared SET Instructor Effectiveness to Students' Sense of Faculty Support, we found that when asked a general question about instructor effectiveness, students exhibited negative bias toward women relative to more specific (and objective) questions about faculty support behaviors.  Students completed both the instructional support surveys (that contained the Faculty Support items described previously) and SET in the last 2-3 weeks of the term associated with the course being evaluated.   Four of the five female instructors (80%) in the study received higher Faculty Support ratings than their SET ratings would suggest while only three of the nine (33%) of the male instructors did so (female instructors are shown in red; male instructors in blue):

These results also show that students rated women lower in instructor effectiveness than men while still reporting that these same women offered them levels of support that were above the trendline for the entire dataset.    

What does this data really tell us?  First, it tells us that the correlation between general impressions of instructor effectiveness and more specific reports of faculty support is not particularly good.   Our data also reinforce the idea that women often receive lower SET scores than men for similar levels of instructional support and teaching quality.   Alongside other related research that demonstrates this type of negative bias against women, these results reinforce the call to rethink how we evaluate teaching among engineering faculty and instructors.  In a time when equity is central to the radar of many colleges of engineering around the country, gender bias in SET underscores the need to transform the conversation we have about student evaluations of teaching -- both those held in meetings with other faculty and administrators and those held inside one's head when SET reports deliver negative and deflating messages after a semester of long hours and hard work.  

As a country, we need excellence in teaching too much to compromise it by demoralizing good teachers with faulty rating systems. We can do better.

Have you had a similar experience to Our Words do Matter or would you like to share a story, concern, or experience that relates to what you have just read?  Click here to share (all responses are private and kept confidential).

Denise Wilson is a professor of electrical and computer engineering at the University of Washington in Seattle, Washington. Her research interests in engineering education focus on belonging, engagement, and instructional support in the undergraduate engineering classroom.   

Three Beers and a Rape Myth

 A Story from the Engineering Research World

Some topics should remain off limits at professional events.  But when no one is looking and alcohol is flowing, conversations can easily spin out of control. The following is a typical story of how quickly a conversation can shift from professional business talk to topics that are not only inappropriate but trigger fear and anxiety for the women at the table.  

As a graduate student, when I had my first paper accepted to a technical conference, I was both excited and nervous.  Would I pull off the presentation smoothly?  Or would I choke and forget any number of technical details regarding the circuit I was presenting?  Or worse, would I become suddenly unable to utter a single articulate sentence and go down in flames of embarrassment? 

I pondered all these possibilities while over-preparing for the three-day, regional conference where I would be presenting my research paper. In between making slides and re-rehearsing, I would think about the upcoming trip with some excitement. I was looking forward to entering into the world of technical conferences, presenting my research, and learning far more from others in a short few days than I could reading on my own or working solely with my lab mates.  And, while I was keenly aware that my intellectual credibility was on the line with a tough audience at the upcoming conference, it never occurred to me that my physical safety might be called into question. 

On the first day of the conference, several of my fellow graduate students (all male) and I drove the two hundred some odd miles to the conference venue. Funding was in short supply and driving was the least expensive of our options.  After arriving safely and checking in to our hotel, we headed out to dinner. It was hot and humid, so I dispensed with business casual clothing and wore the coolest options I could find in my suitcase. Consistent with graduate student budgets, we all headed to an inexpensive bar and grill. I slid into the booth first and perused the menu with great interest, looking for the most calories for the fewest number of dollars. The food was what you expect at a bar and grill.  Filling, not particularly healthy, but well priced. I was still working on my first beer when the rest of the guys were on their second or third. They spent most of the meal talking technical. I listened to their conversation at first but after a while, my mind wandered, and I drifted toward observing the other diners for a change of pace.

When my mind came back to planet Earth, the topic of conversation had shifted drastically.   One of the guys who was on his third beer had started talking about rape (exactly how anyone could go from talking about circuits to talking about rape in a matter of minutes was beyond me).  In the next few minutes, he rambled on about the many things women did that invited rape.  Short skirts. Tight dresses.  Cleavage on display. Big smiles.  The list was pretty long. I wondered what kind of women did not qualify for rape according to Mr. Three Beers' ongoing description of the many things that women did to invite and deserve it. 

At first, I felt like a detached observer of the conversation. Then, I wrote it off as a function of the three beers I'd watch this guy drink. I expected the conversation to shift. I expected someone to step in and redirect it. But, as the guy went on and on, I started to squirm in my seat. I was trapped in the booth and would have had to draw attention to myself to get out of the situation and leave (never mind that I had no way back to the hotel but with these guys and the one car we shared). I became more and more aware of the short summer skirt and tank top I'd worn to deal with the heat and humidity. I'd left my phone in the room and pondered my options for getting back to the hotel some other way. I tried to keep my facial expressions calm and unperturbed. I tried to keep my anger in check. I stayed quiet. It seemed like the safest option.  

And when the dinner finally and mercifully ended and I was back at the hotel and out of the car, it took all of my self-control not to run back to my room. But of course, then there was the other issue: my faculty PhD advisor had required us to double up in rooms "for budgetary reasons." I was the only female in the group and deserved no special considerations so there I was... in the same room as one of the guys who listened to what Mr. Three Beers had to say without saying a single word. Great.  

Surprise, surprise: I didn't sleep much that night. The dinner conversation rolled around in my head as I tossed and turned. I had never heard of normalizing rape or rape mythos or rape culture before. It was my first exposure to a man who seemed to genuinely believe that women often brought rape upon themselves. While I had no doubt that Mr. Three Beers deserved some damage to his private parts for his position on the subject, I certainly wasn't going to do anything or say anything considering the precarious position that limited research funds had left me in.  So, on guard, I tried to rest and didn't sleep. Tried to focus on technical matters, but thought about Mr. Three Beers instead.  

The next day was my paper presentation.  You'll never guess how it went.

Have you had a similar experience to this blog (Three Beers and a Rape Myth) or would you like to share a story, concern, or experience that relates to what you have just read?  Click here to share (all responses are private and kept confidential).