How can probabilities be used to improve economic projections, especially for considering multiple alternatives? Probability is the likelihood of a particular event, measured between zero and one; zero means no chance, one is certainty. On a coin flip, the probability of heads is 50%. Tetlock is a political scientist, combining politics with psychology and decision-making. Tetlock was an originator with The Good Judgment Project (GJP) on improving the accuracy of probability judgments of real-world events. The early work on the GJP was summarized in Expert Political Judgment. Forecast accuracy was not great, overall about 50/50 (described as equal to a dart throw by a chimp). The only significant difference for forecast accuracy was hedgehogs versus foxes. Hedgehogs have a specific view of the world (say supply-side economics) and base all judgments through this perspective. Foxes have no specific view, but rely on multiple cues and sources of information; foxes also tend to change their probabilities as new information became available (good or bad, from whatever source is relevant). On average, hedgehog accuracy was worse than the infamous dart-throwing chimp. Foxes were significantly better.
Consider how probabilities can be used in local real estate, applying the 10 commandments. The greater Austin area is our real estate market. A specific real estate team has a unique (inside) perspective and experience. The outside view contains economic and democratic information (as recent as is available). The perspectives of experts (like a real estate economist) can be crucial, plus additional local perspectives. Look at what is particularly important. Find the “goldilocks zone,” important points that can be reasonably analyzed. Generating revenue from selling real estate may be a primary focus.
Given the current pandemic, there is considerable uncertainty and potentially a wide range of alternatives from mild effect (it will be over soon, there are local positives like lower interest rates, real estate sales should be stable or improving) to severe (it will be long-term and devastating, with a potentially crippling effect even on the local real estate market; it will be next to impossible to show houses in person). Super-doom is not out of the question. With considerable uncertainty, probability statements should be cautious. As more information becomes available, probabilities can be more confident (moving away from, say, 50-50). Look for errors and biases as probabilities are updated. Keep at it; practice improves prediction.
“What if” questions can be important. What if federal lending expanded during the crisis. Should you file for Small Business Administration help and for what purpose (e.g., just staying afloat and paying people, expanding operations)? What happens if local rules get tightened?
Bayesian Analysis and Brier Scores
Probabilities can be informal or formal and quantitative. Bayesian analysis is the standard type of formal quantitative analysis for this probability perspective. Combining a quantitative approach with accumulated Brier scores (see below) indicates relative effectiveness of the process. [Note that Brier scores can still be used for informal (subjective) probabilities.] Here is the basic model:
P(H given E) = [P(E given H) x P(H)] / P(E), where
H is the hypothesis
P(H) is the prior probability
E is the new data (evidence)
P(H given E) posterior probability
P(E given H) is the likelihood
P(E) is the marginal likelihood (model evidence)
A simple example gives a rough idea how it can be used. You want to predict next month’s revenue. Last month your revenue was $100. You start by assuming that there is a 50/50 chance that revenue will be $100 or more (that’s the prior probability). You want to determine the probability of this month’s revenue is $105. After further analysis you estimate that there is a 50% change that revenue will be $105 and a 50% chance it will be $90. Plugging into the model:
P(105) = (105 x .5) / [(90 x .5) + (105 x .5)] = 52.5 / (45 + 52.5) = 53.6%
According to Bayes, there is a 53.8% chance that revenue will be $105 and a 46.2% chance it will be $90. This is highly over-simplified, but a starting point for further analysis.
As new information becomes available, these numbers will be adjusted. Assume the Federal Reserve increases liquidity and Congress passes further stimulus, good news that increase the probability of rising revenues of $105 to 70%. Then recalculate:
P(105) = (105 x.7) / [(90 x .3) + (105 x .7)] = 73.5 / (27 + 73.5) = 73.1%
The adjusted Bayes shows a 73.1% change of revenue of $105 and a 26.9% change of $90. The adjustments would continue as new information becomes available.
The accumulated accuracy of the forecaster is tabulated using a Brier score, developed by Glenn Brier in 1950. It measures the mean squared difference between the predicted outcome versus the actual outcome. The lower the score the better (kind of like golf). If the actual revenue was $105 (or higher) and the Bayes score was 1, the Brier score is zero. Say actual revenue was $105 and the Bayes score was 73.1%, then the Brier score was (.731 – 1)2 = -.2692 = .072. If the actual revenue was $90, then the Brier score was (.269-1)2 = .534. The worst score (if you are totally wrong) in 2. The point is the closer your score is to the actual result, the lower your Brier score and the lower the better. [Note: if your prediction always remains at 50%, your Brier score will always be .25. Brier scores can be used for any probability predictions and don’t require Bayes.]
The focus with real estate started because of a given tele-conference. The techniques are generalizable to many circumstances. When are the uses of probabilities particularly helpful? Just thinking in probabilities can be a useful exercise. The above Bayesian example is a quantitative probability. Adding in subjective probabilities can be beneficial. A subjective probability is based on personal judgment and experience about whether a particular outcome is likely to happen. No formal calculations are necessary (bad news for quant guys, great news for everyone else).
Covid19 predictions are being made daily and updated with new information. Great. The additions I’d like to see are probability statements. For example, projected number of total cases and total deaths are common. I would like to know more about the specific models, including any information about past accuracy (there have been previous epidemics). Quantitative models usually project a range, say 30,000 to 150,000, with 60,000 the most likely. Often, the only number presented is 60,000. Probability statements would be useful. What is the probability of 50,000 or fewer deaths? Over 100,000. Then, update these on a daily basis as new information comes in. Brier scores can be calculated as the probabilities are updated. The Good Judgment Project does exactly that for Covid19, which will be a subject of a future blog.
In Part 3 we return to the local real estate market for analysis.