Episode 31
Episode summary
Revisions to economic data are crucial so users of that data have the most accurate and current information possible. Still, recent revisions to the Bureau of Labor Statistics’s monthly job report have been larger than some observers would like. Caleb and Liz discuss the trend and how we can think about them in the broader business climate.
Subscribe
Transcript
Liz Wilke (00:00:31) – Hi, I’m Liz Wilke.
Caleb Newquist (00:00:35) – I’m Caleb Newquist.
Liz Wilke (00:00:37) – And this is the Gustonomics Podcast. In each episode, we bring you a little bit of economics knowledge so you can be more informed, use the information in your business or work, or impress your unimpressionable dad.
Caleb Newquist (00:00:53) – Please remember to rate, review, and subscribe to the show or share it with an economics curious friend. Anything you can do to spread the word about the podcast is greatly appreciated. Hello, Liz.
Liz Wilke (00:01:07) – Hello, Caleb.
Caleb Newquist (00:01:09) – Is your dad impressionable?
Liz Wilke (00:01:11) – Given that my father is a regular listener to this podcast, I would very much like to decline to answer this question. Oh, OK. I will I will say my father is a very sensitive, intelligent person and not not easily impressed.
Caleb Newquist (00:01:33) – OK, he sounds like a good dad. Sounds like a good dad. OK, great. Good.
Liz Wilke (00:01:41) – What about your father? Is your father unimpressionable?
Caleb Newquist (00:01:44) – No, my dad is my dad is also sensitive and kind and and fun to be around. He’s very gregarious and and no, if I tell him something even like, you know, low grade, you know, interesting, he’s he’s he’s he gets excited. So like, yeah, he’s a good sport, too. Liz, I’m going to ask you about something that is common in the economics world, and that is data revisions. This is a very common thing. You and I have talked about it off the mics a little bit, and we thought we’d do a podcast about it because it’s a useful topic. But, Liz, why why are data revisions? Why are we talking about data revisions today?
Liz Wilke (00:02:33) – Well, Caleb, we have talked about this, and actually we’ve talked about it in light of all of these jobs reports that have come out, because every time a jobs report comes out, you know, especially in the last, I don’t know, year to two years or so, we’ve been really, really, really watching the jobs report. Right. What’s the jobs report going to do? What’s it going to mean for inflation? So every time there’s like a good or better than expected jobs report, we we’re really trying to read into it to find out what’s going to happen. That’s not necessarily a bad thing to do. But the thing is that those jobs reports, when they come out, get revised. And the problem with using them now is that there have been some really big revisions and revisions of large size are becoming a lot more common in the last year or so. And so we’re maybe putting a little too much confidence in the jobs numbers when they come out and not thinking about the revision. So we’re going to talk about revisions today.
Caleb Newquist (00:03:30) – OK, so before we get into specifics of what brings this topic up, can you just, you know, it may be it’s obvious, but why do revision, why, why, why do we have revisions? What, what, what role, what are they and what role do they play in kind of economic data reporting?
Liz Wilke (00:03:52) – Yeah, so every month the Bureau of Labor Statistics, the Bureau of Labor Statistics uses its big data generating machine, which is armies of people and surveyors and data analysts, and they do what’s called a establishment survey. So they go to businesses and they say, do you have an open job? Did you hire anybody? If you hired somebody, how many people did you hire, et cetera, et cetera. So they take the results of this survey. They aggregate it all the way up to get this big number and they apply some weights for industry and business size and what have you, based on what they think they know about how many businesses there are of each size. And then they come out with this number. So there’s a lot of extraction, right, from what people say in a survey to what the actual big number is. And so, you know, they get the results in. There’s only four weeks. So they do the survey early in the month and there’s only so much time between now and when the numbers come out.
Liz Wilke (00:04:48) – And so they’re really crunching the numbers, but there might be some statistical adjustments in this survey that maybe we didn’t think about. So it’s really just like a top line preliminary number. And they say it’s preliminary. That’s just not the part that we focus on. And then, you know, after they sort of really churn through, right, all of those numbers across all of those people, then they come out with the final number for the month. So that’s a monthly revision. They actually also have an annual revision because they do this census of businesses. So how they reweight the numbers depends on how many businesses of each kind and each size they think there are. And they get that number annually. So every year they actually do an annual revision of those numbers, too, against the new, like, data that they have about how they should be weighting each of these businesses.
Caleb Newquist (00:05:34) – OK, so a couple of questions. Number one, are the revisions, what are the revisions usually big or they usually small?
Liz Wilke (00:05:50) – Great question that I don’t have a very clean answer to so a typical revision It’s let me set it this way. It’s not Uncommon for a revision to be up or down maybe 20 or 30 percent typically sort of and I’m saying typically like Before they started getting bigger, which is what we’re talking about now. Yeah. Yep So you think like 20 or 30 percent? I think I know what you’re gonna say Caleb, which is like that’s a really big revision seems great like 20, you know if I give an estimate 20% up or down you wouldn’t have a lot of confidence in that estimate, right? but the thing is as a percentage of the whole labor market It’s incredibly small. It’s incredibly precise number Like I I want to press home how kind of magical it is that we can get a number that’s even this precise Even though 20 or 30 percent feels big So to to give you like a Sense of proportion here. Let’s say you have a jobs number. That’s two That’s three hundred thousand in a month and you’re off by a hundred thousand That hundred thousand as a percentage of the hundred and seventy million people That are in the labor market right and working for themselves or for businesses. It is like 0.05% Yep of you know The total mass right that they’re trying to measure so even a pretty big error is actually a pretty small error If you think about it that way, right?
Caleb Newquist (00:07:27) – Okay. So and the reason that these revisions happen is number one. It sounds like this is just Part of the process like they get more information and so they get they better understand the number but also When from a historical perspective, they want to use the best best information possible when they’re doing like Historical trend analysis, for example they just want to make sure that they’re using the most accurate most up-to-date number possible and those numbers will Change slightly over time. And so that’s that’s The reason that these numbers change over on a monthly basis But then also like you said the annual adjustment they make
Liz Wilke (00:08:09) – Yeah, you could think of it like you take a photo and It’s maybe a little grainy and then you run a program over it that sharpens it up and cleans up the lines , right. Your first photo is basically of the thing you took a photo of But the image is sharper and clearer and more precise right the more times you sort of sharpen here or adjust the image there Right to bring out like aspects of that photo So I would say the first the first print is a grainier photo and then you do some subsequent Revisions so that you get a really precise picture right of the photo that you took
Caleb Newquist (00:08:43) – Right, and this isn’t unheard of like in economics. So for example, if you go back wait, one of our early episodes was on The the recession that never came right and one of the things that we know about recessions If you listen to this podcast, is that sometimes a recession occurs, but we don’t know until after the fact, right?
Liz Wilke (00:09:05) – Yeah, in fact usually a recession occurs and we don’t know until after the fact.
Caleb Newquist (00:09:10) – Right and then there’s times when people think “Oh, we’re definitely in a recession,” and then the data comes out later and it’s like no it wasn’t great, but it wasn’t technically a recession. And so that’s this is kind of just this is a feature I think of economics not really a bug right the revision or or or looking at the past with with with More clarity than when we’re looking at it in the present.
Liz Wilke (00:09:37) – Yeah, I think that’s fair I would say I don’t think that’s unique to the economics profession, right? I mean, I think it’s a I think it’s a pretty human II human thing to do To not have full clarity about what exactly is happening to you in the moment And then when you look on it with hindsight, right you can tell a much cleaner clearer story for yourself, right? I mean economics is doing that too It’s just you know, we’ve really I think the real difference is we’re putting so much stock into this number every single month right that is like inherently and sort of like We’re putting so much stock into this number every single month, right? That’s kind of like inherently fuzzy
Caleb Newquist (00:10:17) – That’s the perfect segue into why we’re talking about this because policymakers don’t live in the future. They live in the present and they’re making policy decisions based on the data that they have in the present and so that’s part of the challenge that they’ve been experiencing recently is that they’re getting the data, they’re making decisions and then six nine twelve months down the road They have these revised numbers and they think those numbers are quite a bit different And I don’t know if I would have made the same choice Had I Known that this was what was actually going on. And so maybe that sounds a little bit convoluted. So Liz I guess my question is Liz I guess my question We’re talking about this because policymakers have kind of been grumbling about these big revisions to data numbers recently, right?
Liz Wilke (00:11:11) – Yeah. Well, we’re talking about it because policymakers care about it. The Federal Reserve definitely watches these numbers, right, and the unemployment rate to think about, are we sort of losing some steam in the labor market? Should we adjust the interest rate? Are we losing some economic strength, right? But we keep getting these pretty good numbers. And so that is creating an argument, right, to sort of stay higher for longer. That’s just one part of the data, the bundle of data that they’re looking at. But it’s a big part. And honestly, markets, right, investors, right, put money into short or long-term assets depending on how they think the economy is going and what they think the Fed will do. And a lot of people care about these numbers. But because there are such large fluctuations, it’s almost like we’re getting less and less a sense of sort of security and confidence about these numbers, right? So we are really attached to them because we make a lot of decisions that affects the sort of world that we live in based on these numbers. But these numbers are becoming less and less reliable month on month. So for example, in 2023, the smallest revision was for January. It was just additional 2%. The original estimate came out that we could gain 472,000 jobs. And we had actually gained 482,000 jobs, right? 10K on 500K, it’s not such a big jump. But you look at June of 2023, the original print said that we had only gotten 105,000 net jobs. But the final revision came in at nearly a quarter of a million net jobs, 240,000, which is about 130% off the mark, right?
Liz Wilke (00:13:02) – So it’s really big, right? And so every month in 2023 had some revision. And some of them were really, really quite big.
Caleb Newquist (00:13:10) – And do people who look at this stuff for a living, do they have a sense of why the variations were so large in 2023?
Liz Wilke (00:13:23) – So nobody is really entirely sure. They’re just entirely sure that it’s happening. So one of the things is that the establishment survey, the survey of businesses that underlines all this data, is aligning less and less with other measures of employment that we have out there. Nobody is quite sure why, to be honest. I don’t think we can say with certainty about why that is. But people do know that the underlying survey isn’t quite as good a measure as it was before the pandemic. And that’s true of lots of data points, right? We’re still sort of figuring out seasonality is different. The way households are forming is a little different. The way businesses are hiring is a little bit different. It could be some combination of all of those things. Plus, people honestly aren’t responding to these surveys at the rates that they were. And that’s having an impact, too, over the long term. So there are probably not just one reason. There’s probably a lot of reasons.
Caleb Newquist (00:14:19) – OK. And then so does the Bureau of Labor Statistics, is this something that they’ve acknowledged? Are they studying it? Are they looking to make revisions to the survey? Like, what are their, if any, actions are they taking right now?
Liz Wilke (00:14:36) – Without having talked to anybody at the Bureau of Labor Statistics specifically about this, I can still say with pretty high confidence that they’ve heard about this problem. I have full faith that a number of people, especially thanks to the beautiful modern technology of the internet, have alerted them to this phenomenon. I have no doubt that they are working on it. But when they will have their heads fully wrapped around this and be able to make some corrections, I actually do not know. At whatever time they make those adjustments, it will be too late for policymakers to act anyway, or financial markets to act. So the question isn’t, when will they make the adjustments? Because that’s going to come too late for everybody living their daily life. It’s, how should we think about the imperfect numbers that we have now? And what I would say is two things. One, use the jobs numbers directionally, not precisely. So if we have positive job number, positive job number, positive job number, don’t think we gained 450,000 jobs, all told. Think the job market’s still pretty good as sort of a directional indicator. So pay attention to the sort of big differences between numbers. Pay attention to the difference between 100,000 jobs and 600,000 jobs. And then pay attention to if it’s positive or negative. And I would say any jobs number that is probably plus or minus 100,000 is probably not that much different from zero that you should really worry about it, to be honest.
Caleb Newquist (00:16:27) – Yeah, because again, we’re talking about, on a monthly report, it may seem like a big number. But in the grand scheme of things, it’s a very small number. And so it’s more about, if I understand you right, it’s more about the general feeling. Like what does three or four months of the economy adding jobs, you can feel generally pretty good about that without getting too hung up on revisions of 1% or 2% here and there.
Liz Wilke (00:17:06) – Yeah, I think that’s right. And I think that policymakers and people who invest money and people who think about what the future holds, it’s going to be a red herring to put too much stock into a single month of jobs reports. These things take time to change. And those changes take time to accumulate. So you’re going to want to think, not just what does this jobs report say,… reports say? And what does that say about where we’re trending directionally?
Caleb Newquist (00:17:35) – Yeah, which is tough to do, right? Because the media tends to, when these job numbers, when these reports come out, they kind of hang on every word. Like everybody’s anticipating the first Friday of the month because that’s when the number comes out. And so it can kind of be tough to step back from it all and be like, OK, well, this number isn’t that great. But in a month’s time, they’re going to tell us a slightly more accurate number. And last number was pretty good. And the month before that was pretty good. So I’m actually not going to sweat this too much. It’s hard to like, cooler heads don’t always prevail in those situations. And what I think you’re saying is, try to keep your head on any given jobs number. Just try to keep your head because it really matters. The context of all of it matters.
Liz Wilke (00:18:31) – Yeah, I mean, I think most people would agree that the news cycle runs on our adrenaline. And so to sort of level that out, right, at a jobs report to say, OK, here’s the jobs report, but here’s what it looks like over the long term, kind of nothing to see here, or generally a good story or not such a good story, that’s not as fun to read in the news. I’m going to go out on a limb. I’m not a journalist, but I’m going to go out on a limb and say I don’t think that’s as fun to read. And so I understand why we don’t get that report every time. But if you want to step away from that and sort of think big picture, you got to look month over month. And then you also have to think a revision even of 100K, which is a pretty large revision, is like well within what you might think of like the tolerance, right, for the economy to handle. And it might even be within the margin of error of the survey to measure.
Caleb Newquist (00:19:28) – Right, OK. Any final thoughts on revisions, Liz, and how we should all think about them when we’re making our economic soup?
Liz Wilke (00:19:38) – What I first would say is that when we start doing the employment situation report, we should probably go back and tell our listeners what the revisions have been for the last few months.
Caleb Newquist (00:19:50) – I will take that note. You are on.
Liz Wilke (00:19:52) – I think that we should do that now that we’ve done this episode. But for people who are just seeing the headlines, just honestly take a beat. This number is actually really hard to come up with because you have to do the survey and a bunch of waiting and all that stuff. But despite the high level of precision of this number, right, 100,000 jobs in 170 million people is just a really, really high level of precision. It is still a fallible number. And it is inherently a little fuzzy. And by focusing too much on this one tree, you’re going to lose the view of the forest. So take it directionally. Take the magnitude. And then take it over several months. And that’s going to give you a much better signal of where the economy is really headed or where the labor market really is than focusing on this one specific number that’s getting lots and lots of revisions made to it lately.
Caleb Newquist (00:20:44) – All right. That’s it for this episode. Hope you learned something new and useful for yourself or your business. Please let us know what you think of this podcast in a review or sharing it with a friend or colleague who might enjoy it. I’m Caleb Newquist.
Liz Wilke (00:20:56) – And I’m Liz Wilke. Thanks for listening.
