Run an OLS regression of the dummy (1, 0) variable on the probability forecasts to test if these forecasts are unbiased.
https://www.philadelphiafed.org/research-and-data/real-time-center/survey-of-professional-forecasters/data-files/rgdp
The actual real GNP growth in the targeted quarter (real time preliminary 1st month announcements) and the corresponding (1, 0) dummy variable indicating 1 if the quarter experienced a negative real GNP decline are also provided. Since you will need this binary outcome variable in your calculations and since there hasn’t been any negative GDP growth since after 2015Q3, you can safely use 0 in the target variable for the rest of the sample period. Also, due to Government shutdown, real time GDP growth has a missing value for 1995Q4. Since this quarter didn’t have negative growth, use “0” for this quarter also.
Based on these two series, do the following:
1) Compute the QPS.
2) Run an OLS regression of the dummy (1, 0) variable on the probability forecasts to test if these forecasts are unbiased.
3) Draw the two conditional distributions of the forecasts given the actual is 1 or 0, on the same diagram and note the probability over which these two lines intersect. You can smooth the curves. Do these conditional densities look good and useful? Which one looks better? What does this mean in terms of the effectiveness of these forecasts to guide people regarding an impending negative growth quarter?
4) Draw the Receiver Operating Characteristic (ROC) curve with values of the threshold ranging from 0 to 1 at an increment of 0.1. Below 0.5, you can search for optimal value for the threshold at increments of .05.
5) One popular criterion to choose the optimal threshold to convert probability forecasts into binary forecasts is Kuiper’s or Peirce Skill (PS) score. How is it defined? What threshold does PS suggest for these forecasts? Is it close to the probability over which the two conditional densities intersect (see #3 above)? What are the associated hit rate and false alarm rates for this optimal threshold? Are these values reasonable to you given your personal risk appetite for type I and type II errors in forecasting recessions?
