NYSE President Lynn Martin and other exchange officials confirmed to CNBC that the root cause of the Big Board’s trading glitch at the Tuesday open was due to a manual error involving the Exchange’s Disaster Recovery configuration.
After the 9/11 disaster, the NYSE was obligated to maintain a primary trading site (at the NYSE) and a back-up site (which is in Chicago).
On Monday evening, routine maintenance was being performed on the software for the Chicago back-up site.
On Tuesday morning, the back-up system (Chicago) was mistakenly still running when the primary system (NYSE) came online.
Because the back-up was still running, when the primary site started up some stocks behaved as if trading had already started.
As a result, Designated Market Makers (DMMs) who would normally publish an opening auction print for each stock were prevented from doing so because the system operated as if an opening had already occurred. This caused significant price dislocations and trading halts.
Martin said the NYSE was looking into implementing stronger testing protocols.