Harvard's Heather Hill and Jon Star reflect on how COVID impacted math teaching and learning, and where do we go next.
The latest National Assessment of Educational Progressed showed big declines in students' math performance -- in some cases as low as 20 years ago. The results showcased the effects of the pandemic and in particular how hard it was to teach math, say Harvard experts Heather Hill and Jon Star. In this episode of the EdCast, they share why the scores dropped significantly, how challenging it can be to teach math, and ideas on how to move forward from this moment.
Share this episode with your networks! Use #HarvardEdCast and tag HGSE on any social media channel.
Twitter: @HGSE
Instagram: @harvardeducation
Facebook: @HarvardEducation
LinkedIn: Harvard Graduate School of Education
TikTok: @harvardeducation
Jill Anderson:
I'm Jill Anderson. This is The Harvard EdCast.
The recent result of the National Assessment of Educational Progress showed huge drops in students' math performance, leaving many educators to ponder what happens next. Heather Hill and Jon Star say math struggles aren't a new issue for students. They are Harvard experts on math instruction and curriculum. They say teaching math during and after the pandemic has been uniquely challenging. Jon knows firsthand considering he returned to classroom teaching during this time. I wondered what makes it so hard to teach and learn math, and what can be done to change it. First, I asked them what they thought about the NAEP scores showing such big declines in math.
Heather Hill:
This was not shocking to anybody who's been watching what the scores have been looking at, like from other assessments, like state assessments, like private companies that do assessments. We knew that things were going to look bad. The longitudinal NAEP also looked bad. So, this was not surprising.
I think what was surprising to people was how much more the math scores dipped than the ELA scores. One thing that we know from the research literature is that math scores have always been more sensitive to students' opportunities to learn. When I teach a class on the impacts of policies on ELA and math scores, it's not uncommon to find that math scores are actually moved by policy, they are affected by policy, and ELA scores simply aren't. So, this is a canonical example of that. There was not a policy but a national emergency and it moved those math scores a lot more than it moved the ELA scores.
What's interesting is why this happens. The thinking among most people is that math learning primarily happens in schools. Kids are exposed to ELA in many places in their daily lives. They talk with their parents at the dinner table. They read texts from friends. They read books. They listen to music lyrics. They interpret those music lyrics. They make arguments with their parents about how late they should stay out at night, or whether they should be able to get the extra popsicle after dinner. So, a lot of those ELA skills are getting built even in the absence of kids being in school. School is the only place that kids, for the most part, learn math, and that's probably what's driving some of this.
A second reason is that math is cumulative. So, if you miss fractions, you're going to have a hard time when you get to high school and you start learning algebra, because fractions are really the foundation for a lot of what's happening in algebra.
It's also possible, a third explanation is that it could be that math was just taught much less efficiently in the pandemic. Something about the move to hybrid or the move to instruction being largely online. It could be that teachers were able to keep the features of ELA instruction that kept that high quality, but they couldn't do that in math. They were resorting to worksheets or they were resorting to videos from YouTube that were not very good and not very aligned to the kinds of things that kids were supposed to be learning that year.
Jon Star:
One way I think about trying to explain what's going on with the math scores would be thinking about the learning that's going on, but another would be about the instruction that students have received.
With respect to the learning, I think it's really critical to think about the particular age that this drop was most significant, eighth grade, and the type of mathematics that kids are learning around that time. Every year is not the same in kids' trajectory in math. It's not sort of growing linearly. There are some years that are really more critical than others, I might argue. So, in these years leading up to eighth grade, and where the pandemic hit for these students, this is when they were transitioning from arithmetic into algebra. Into the all important realm of symbolic mathematics, which is so critical to their future in whatever else they're doing mathematically.
This is where these students experience their most challenging years during the pandemic in terms of math learning. They really may have suffered in their ... not only their learning of fractions, which happened kind of in the late elementary age, but in their proportional reasoning skills, in their pre-algebra, in their transition into algebra, which is fundamentally what's assessed on the eighth grade NAEP. So, there's really a lot that's been going on, or that we would've hoped has been going on mathematically for these students over the past years, and it just hasn't happened in the quantity or the quality that we hoped. So, in that sense, it's no surprise that they're really struggling. Those struggles are not going to be easy to make go away. They're really lacking some fundamental knowledge about algebra that they're going to need for other future courses. So, there's a content based explanation for thinking about this in terms of what they learn.
I think we can make the same point about instructionally what might be going on. What I might wonder is that the instruction that the teachers have been providing during the pandemic, whether it's online or whether it's in other settings, I worry a little bit that what affordances teachers have available to them in that instructional realm, they, in some way, emphasize what might be the least desirable aspects of math instruction that we would want to see. So, there's going to be more use of worksheets, there's going to be more teacher lecture, there's going to be less student interaction. The ways that teachers have had to teach. It's not necessarily the teacher's fault, it's just the way that they've been forced to teach during the pandemic. It isn't what we know to be the most effective way to teach math, but that really is all that the teachers had at their disposal now. So, we're seeing the consequences of that.
It's this terrible interaction between the ways the teachers were forced to teach, which may not be the most effective way to teach math, and the content that students, as a result, are missing out on that is so critical to them in eighth grade, but even more so moving forward.
Jill Anderson:
Wow, there's a lot there to think about and a lot to unpack. Is math just far more difficult to teach than other subjects? Is math far more difficult to learn than other subjects? What's kind of going on here that math is always a challenge?
Heather Hill:
Yeah, that's not a small question. The best evidence is that the average teacher who ... Average K5 teacher is probably better at teaching ELA than at teaching mathematics. There's a lot of feeling among teachers at that grade level, that they're not math people, they don't love math, they don't feel confident in the way that they learned it. And then it makes it hard for them to teach it feeling confident, and teach it with the conceptual knowledge that we would hope that they would use to teach the content.
Teachers go through teacher education programs. They take a couple courses. Typically, a math methods course or a math content course, but it's not quite enough to relearn six years or eight years of math in relatively more sophisticated ways. So, that's one piece. It explains why math is typically not quite taught as well across the board, as you're going to see in ELA.
Jon Star:
Teachers may hold certain knowledge, but also, certain beliefs about math that may challenge their ability to teach it well. Again, just to emphasize, it's not just the knowledge, which might be true, but also, there's a lot of beliefs about math, what math is, what it means to learn math that teachers may have developed over their own years of schooling, that may not be that productive for the ways that we hope they would teach math. Our teacher education programs have gotten a lot better at trying to not only increase that knowledge, but change those beliefs as well, but it's very difficult.
Heather Hill:
Since the common core, there has been a kind of free-for-all. It's changing maybe in the last five years or so. But for a long time there was a kind of free-for-all about the materials teachers would use to teach math. It was kind of like a mark of pride to write your own things or write them with colleagues or find them on the internet. What ends up happening, teachers aren't meant to be curriculum designers. That is a full-time job, design a curriculum for kids. There's things that you just need to worry about, like, "Is the definition of fraction that gets used in sixth grade built on in seventh grade? Are kids getting exposed to multiple definitions of fractions that will then be later used in algebra?" I think because there was this long period of time when teachers were asked to kind of create their own program of study for their kids, I think the overall quality was lost a little bit.
Jon Star:
Our system is structured in a particular way, in that our elementary school teachers teach all subjects. There's a lot of reasons why that's what we do. Historically, there's a lot of pieces to our system that make that the way that we need to do things. But in other countries, that may not be the case. In other countries, people who teach math may only teach math, even at very young ages. They may have much greater mathematics training, both content and in terms of teaching math, than our elementary school teachers have about math. So, it's an interesting thing to wonder is whether that has something to do with the challenges that we're facing, and whether there's a interest in thinking more about that possibility of having specialists who only teach math, or only teach math in science at particular grades, and what that might have afforded us if we did that.
Jill Anderson:
I'd love to know what you hear or what kind of feedback you get from teachers when these types of NAEP results come out and they're kind of depressing.
Heather Hill:
I think teachers are much more kind of concerned when their own kids aren't doing well. I mean, we think that there's a sort of, "Well, that's a national problem," but if they see their own kids falling behind or suffering, or the kids walk into the classroom having a lot of unfinished learning, I think that that's pretty devastating for teachers. It's just hard to watch kids struggle in that way, especially watch your own kids struggle in that way, for teachers.
It also just makes it a lot harder for teachers to do their job. I mean, they're supposed to be teaching on grade level mathematical material. If they have kids coming in the door in all different places, they have to then help some kids catch up and keep some other kids occupied. It multiplies the difficulty of teaching mathematics.
Jon Star:
I'm not sure teachers would've been following the NAEP scores in the same way that we might or the press might. But I think teachers have had such a hard time over these years teaching math. They have been doing their very best, but it's been so difficult. I think Heather touched on what I see as a really central challenge that teachers are facing, which is that they're dealing with a class full of students who are coming from so many different places in terms of what they know and what they don't know. On the one level, that's always a challenge of teaching, is differentiating for the class that you're faced with.
But I think COVID has made that even more challenging. That you have some students coming in who have had nothing over the past year, they've have nothing of substance, and you have others that maybe didn't lose but a little bit. You have this enormous diversity of prior knowledge that you're trying to wrestle with. You're trying to figure out as a teacher whether you remediate for all those students that really need some serious remediation, but how do you do that at the same time that you're supposed to be continuing to move forward with grade level content? You just can't stop. You can't say, "Oh, you're in the sixth grade, but you don't know the fifth grade material, we're just going to do fifth grade this year." You can't do that. You have to continue moving forward. But there's some students that really did not get any of the fifth grade material.
So, how do you do that? It's an enormous challenge. We don't really have curricula that are designed for that particular challenge. Especially in its most extreme version, which is what we're faced with right now. Pedagogically, instructionally, it's just a really hard thing to do. It's kind of like extreme differentiation, when just the basic differentiation is hard enough as it is. So, this is just tough. So, I don't think teachers are surprised so much that this is the direction that we're headed, because they've really been living with this on a day-to-day basis.
Heather Hill:
Can I ask a question? The listeners should know that Jon teaches eighth grade. How much longer does it take you to plan a lesson when you have that extreme differentiation?
Jon Star:
Well, it seems like it's a completely different planning thing. Like, planning for those lessons is completely different than if I didn't have to deal or minimally was differentiating. That it forces me to look at each problem that I'm asking the students to engage with me on and think about what everyone in the room is bringing to it, and how I might need to modify that task or that problem, or the discussion around that problem to account for the different places that the kids are. It's tough. I think for teachers who have had minimal experience doing that, who do they go to ask questions about how to do this? I'm not sure who helps them out with this. The curriculum doesn't do a great job with this. This is not what most curriculum are designed to do. And even experienced teachers are really challenged by this. It's a core task of teaching that's there always, but it's a particularly challenging extreme version of that task now.
Jill Anderson:
I was wondering, Jon, because you've done a lot of work looking at math interventions and things of that nature, are there practices that can be adapted to this moment in time?
Jon Star:
Well, I'm not sure that there's an easy fix to this. This is hard. I always am a little reluctant to claim the sort of uncharted territory argument because there's been phases in the past, some that I'm aware of, some that maybe I'm not aware of, where we've dealt with things that have been similarly challenging. But I do think this is something that we need to foreground a little more than we have in the past. The ways in which teachers are needing to differentiate in a different kind of way, and how supports for that can exist, either curricularly or in professional development.
People like us on the research side, maybe we need to dive more into that directly and think about it. Because we haven't had to be thinking about that directly so much either. Again, it's a core part of teaching, but we haven't had to deal with it in the same way in the past. I think there's room for us to reengage with that, as well as researchers, just to help out. To design curriculum, to design interventions, to think about what's most effective. Other than something like tutoring, which we know to be effective. But what can the teacher do in a whole class setting to really help this situation?
Jill Anderson:
What are some things and resources that we can tap into?
Heather Hill:
Well, Jon was talking about tutoring, which, if done well, can yield really big gains for kids in terms of catching up. It's targeted to where the kids are. You can even put two or three kids in a group if the kids are needing to travel that same path back to kind of mathematical knowledge or up to mathematical proficiency. As long as the tutoring is of decent quality, you're going to also be able to figure out, "Okay, that kid is getting it." A good tutor will be like, "Okay, you got it now. You can move on to something else." Whereas the whole class, a teacher doesn't necessarily have that kind of knowledge of each individual child, or the capacity to check in with each individual child. So, those are just some of the reasons that tutoring seems to work.
There's also been some research on double dose. This would be for the older grades. So, making sure that kids can just make up the instructional minutes that they lost during the pandemic by having an algebra class, but then having a pre-algebra class that supports them, aligned with what they're learning in algebra. So, not completely like, "Let's just redo eighth grade math again while you're learning ninth grade math." They need to be synced up together so that, as kids are learning skills in algebra, they're getting the support for those in the other class that they're in.
Jon Star:
Yeah. I only had a couple things. One, that though we're in tension a little bit between these remediation goals and these advancing in grade level content goals, I think we should resist the inclination to do too much remediation in these situations. That I'm not sure that ultimately helps us. It just kind of kicks the can down the road, if you will. That I think teachers, though it's extremely challenging, need to figure out ways to continue to have kids move forward with grade level content despite the fact that they're bringing some serious prerequisite knowledge gaps. That's hard, but I think we need to do that.
And then maybe the second thing I'd say, and this is maybe more toward people like us researchers in the field, we need to think about partnerships between researchers and folks in schools to try to solve these really difficult problems. This is a really challenging, contextualized, embedded problem in the schools. It's really nuanced. And I think it could benefit from the kind of partnerships, the research practice partnerships that some people in the field are doing, but others are not. I think that would be a way that we could all try to contribute towards solving this really tough problem.
Jill Anderson:
Not everybody needs to be panicking, but in some cases we do need to panic. But I think that there is a big piece to these results in that it shows the disparities among people of color.
Heather Hill:
Yeah. I don't worry as much about the absolute numbers in this particular case, what really worries me is the gaps that have opened up. In particular between communities where kids were already disadvantaged, potentially, didn't have schools open as much as communities where schools were open. Those tended to be whiter and more affluent. What happens when you see those achievement differences widen is that opportunities will also widen. Because that turns into, who makes it through the first semester of college math in order to major in engineering, in order to major in computer science?
There was a pretty serious gating ... We have a kid in college and those are pretty serious gating moments. When you start to see the kinds of differences that have opened up on the NAEP, that's going to be reflected in the distribution of kids and the opportunities that kids all across the board have to participate in college level math, college level STEM fields, and ultimately, STEM careers. So, that's devastating. It's also devastating just because we don't have enough STEM trained individuals in this country. So, to lose kids in large number because of this I think is pretty ... it's pretty devastating.
It's like, if you told me kids can answer three fewer fractions problems in eighth grade, I'd be not that bothered by that. It's more that we have introduced even more inequity into the system because of the way the pandemic played out in schools.
Jill Anderson:
What do you think is the best path forward from here?
Heather Hill:
First, kids are pretty resilient. So, it is all about opportunities to learn. You give kids opportunities to learn and they learn stuff. In some ways, it's just counterbalancing those lost opportunities to learn, and making sure that kids make up the time in math class. I mean, in some ways it just boils down to that. It helps if we target the resources to the kids in the communities that saw the worst of it in the pandemic, so that we're making up the differences that grew and we're able to make outcomes more equitable. But we know that there were differences in who got to go to school during the pandemic, so we need to target resources to the communities that lost the most time there.
I mean, we can talk about improving math instruction across the board, but I also just think this isn't the time to bring in brand new initiatives when teachers, like Jon was saying, are dealing with so many other things. This is a hard job we're asking teachers to do, and schools to do. Right now, I think focusing on the basics can help kids catch up, because expose the kids to the content and they will learn the content. This is one thing that we know after 150 years of research in educational psychology.
Jon Star:
This is not terribly concrete, but keep the content specific nature of this challenge in mind. That we're talking about kids' learning of mathematics. As I said earlier, there's things that we were hoping they would learn in this particular time period that are related to algebra. This is also about kids' geometry knowledge, I'll mention. Because what's happened over the past few years is that something had to be cut in the curriculum as teachers were triaging. And too often, from my interactions with teachers, it's geometry that's been cut. I haven't looked closely enough at the NAEP results to know to what extent the score drop is related to particular question types, like geometry, that they didn't know much about. But kids are going to be moving into high school where they're expected to take geometry courses and they haven't seen any geometry, at least the geometry that they're expected to take.
Anyway, the larger point is that there's a content specific nature of these challenges, both in terms of the teaching and the learning, that I hope that we'll keep in mind. This isn't just any subject where the scores went down, what can we do to improve it? It's actually mathematics, and that should factor in to any answers that we put forward.
Jill Anderson:
Jon Star is an educational psychologist and professor at the Harvard Graduate School of Education. He is also a math teacher. Heather Hill is a professor at the Harvard Graduate School of Education, where she is also a faculty co-chair of the Teaching and Teacher Leadership Program. I'm Jill Anderson. This is the Harvard EdCast, produced by the Harvard Graduate School of Education. Thanks for listening.