Writing in the Globe and Mail, Macdonald-Laurier Institute Senior Fellow Philip Cross says the error in the Statistics Canada jobs report is not symptomatic of larger problems at the agency responsible for collecting federal government data.
Cross, a former chief economic analyst with Statistics Canada, says this week’s controversy over the misstated July jobs numbers masks the fact that data quality at the agency is higher than it’s ever been.
“They messed up one data point in one series. Big deal. Anyone who lets one data point affect their view of the economy should not be doing analysis”, writes Cross.
On Friday Statistics Canada revised its earlier figure of 200 jobs added for the month to 42,000 jobs.
Cross also wrote about the role that human error can play in data processing for the National Post earlier this week.
By Philip Cross, Aug. 15, 2014
Statistics Canada has undergone some significant changes in recent years. The long-form Census was replaced by the voluntary National Household Survey. Munir Sheikh, its chief statistician, resigned over the Census controversy. Its budget was cut, leading to the revocation of its ‘no-layoff’ policy. These represent significant changes, and have led analysts outside Statistics Canada to speculate they contributed to this week’s unusual retraction of erroneous July employment numbers, corrected Friday to show an increase of 42,000.
However, it would be a mistake to link the error in the July employment data to the broader changes Statistics Canada has undergone in recent years. The replacement of the long-form Census has no impact on the overall sample design for the Labour Force Survey, which uses the population estimates from the still mandatory short-form Census (besides the update for the 2011 results will not occur until next year). Nor did budget cuts affect the key surveys at Statscan, like the Labour Force Survey or the CPI (the latter is actually undergoing a major expansion). Elsewhere, the reduction in Statcan’s staffing was mostly met by a wave of retirements or, as in my case, a welcome shift to the private sector.
Should we be surprised that Statistics Canada can undergo transformative change and keep its eye on the ball of producing good data? This is the third time in recent decades Statistics Canada has gone through a major upheaval of its existing practices. In the late 1970s, widespread questions about both its data quality and its management led to thorough external reviews and a major overhaul of how the agency was managed. In 1986, the Census was cancelled, and then reinstated on the proviso that Statscan fund it out of its existing budget. This led to extensive program cuts as well as large price increases for its products.
In all these instances, Statistics Canada showed a surprising (to outsiders) ability to adapt to rapidly changing circumstances. What emerged, like today, was a better, leaner, more focused organization whose management capabilities have not been questioned since the early 1980s.
The most unfortunate by-product of the Census controversy is that people now scrutinize every action of the agency. To outsiders, Mr. Sheikh’s legacy is his resignation over the Census. Within Statscan, his major legacy was a measurable improvement in data quality that was largely maintained after his departure. So while some people look at this week’s jobs data error as symptomatic of an organization’s standards wilting under the pressure of too much change, the truth is that data quality is much better than before 2008. It will never be perfect, because both the underlying data and the statisticians who handle it are imperfect. So not every mistake should be taken as a sign of organizational failure. Sometimes mistakes happen, and all that they reveal is that people make mistakes, no matter how hard one tries to avoid them.
People should get agitated about Statscan over substantive issues. Wring your hands that the CPI over-states price changes over long periods. Write your MP complaining that the Labour Force Survey doesn’t follow the U.S. practice and exclude 15 year olds. Take to the barricades that for an energy superpower like Canada, measuring energy exports has become a monthly adventure, routinely revised by $1-billion a month. But don’t use the July employment incident to evaluate how the statistical system is functioning overall. They messed up one data point in one series. Big deal. Anyone who lets one data point affect their view of the economy should not be doing analysis. Move along folks, nothing to see here.
Philip Cross is the former chief economic analyst at Statistics Canada.
