User login

LFH News Letters



Stay informed on our latest news!

Newsletters

Notable Operational Failures in Post Implementation

The most critical and expensive failure is after the implementation maybe days, weeks, or months into operation. These failures are unpredictable, unexpected and by far the most costly, because of impact on customers. The following list of failures, collected over the decade, happened post-project, after the implementation:

  •        In March 2007, US Airways struggled with a faulty reservation-and-ticketing system, and kept lines down by adding workers and asking travelers to use the Internet for check-ins.
  •        In December 2006, a computer systems outage made it difficult for air traffic controllers in Florida to identify and track more than 200 flights in the air, allowing some planes to come too close together, according to officials.
  •        In April 2006 Microsoft 's MSN search engine, the third most popular in the U.S., suffered an hours-long outage as queries returned an error message instead of Web page results.
  •        In January 2006 Tokyo Stock Exchange Inc. was forced to halt trading 20 minutes earlier than normal because its system was close to capacity. In December software was questioned after an erroneous order to sell 610,000 shares of J-Com Co.
  •        In July 2005 HSBC admitted hardware failure caused a major systems crash that hit thousands of customers for ATM, credit/debit, online services and internet, and it was the worst in its history.
  •        In December 2004, Comair airlines had to face a cancellation of over 1,000 flights on Christmas Day after its computer systems for reservations crashed.
  •        In November 2004, a computer failure at the Department for Work and Pensions (DWP) stopped 80,000 staff from processing new pensions and benefits claims for several days.
  •        In October 2004, a computer failure at Waikato Hospital (NZ) left thousands of health workers out of pocket and forced the manual processing of patient records.
  •        In October 2004, Avis Europe took a €45m hit due to problems with a new ERP system. Development halted with delays & higher costs due to implementation and design problems.
  •        In September 2004, hundreds of flights were grounded for 3 hours at Western US airports. A computer failure knocked out radio contact between pilots and air-traffic controllers. In five instances airplanes passed very close to each other.
  •        In August 2004, a computer crash prevented thousands of UK pensioners collecting benefits payments on the busiest day of year after the £500m Benefits Transfer system went down.
  •        In June 2004, RBC fell behind processing salary deposits thousands of Canadian workers as millions of transactions were affected by a computer glitch that caused payroll delays.
  •        In June 2004, an air traffic control computer failure saw massive air disruption across the UK. All flights from UK airports were grounded after a problem at the National Air Traffic Service.
  •        In March 2003, 4,700 Kaiser Permanente patients received wrong medications because a computer glitch that caused labeling errors.
  •        In July 2003, Orbitz LLC, an airline-owned travel site, suffered a 24 hours outage with database problem.
  •        In March 2003, 4,700 Kaiser Permanente patients received wrong medications because a computer glitch that caused labeling errors.
  •        In February 2003, thousands of BlackBerry users experienced service outages amid interruption on the company's network.
  •        In February 2003, a glitch knocked Microsoft's bCentral services, leaving 15,000 small-business Web sites inaccessible for 8 hours.
  •        In January 2003, MSN Messenger outage lasted five hours and affected 75 million customers.
  •        Web site downtime cost UK businesses a total of £565m in 2001, and is expected to rise to £715m in 2002 according to Yankee Group. The report cites two main culprits: physical failure and human error.
  •        In October 2001, a 12-hour outage put out most of Toronto-Dominion Bank’s elec-tronic banking channels. Millions of customers were unable to access their accounts.
  •        In September 2001, a program glitch at Rogers Cable opened a back door into email accounts of 423,000 subscribers for 12 hours, allowing anyone to hijack them.
  •        In September 2001, customers trying to use Citibank’s 2,000 ATMs in New York and other parts of the country continued to be thwarted by system problems.
  •        In June 2001, the NYSE halted trading for more than an hour due to a failed software upgrade that caused half of its listings to be disabled. The exchange suspended trad-ing in all stocks. The NYSE halted trade in October 1998 with similar problems.
  •        In March 2001, a computer malfunction temporarily shut down the flight dispatch system used by a Delta Airlines subsidiary, causing cancellations and delays affecting flights all day across the entire travel network in the eastern U. S. and Canada.
  •        In January 2001, hardware problems shut down eBay for 11 hours. The outage was blamed on a failed data-backup system.
  •        In December 2000, for the third time in less than two weeks, Amazon crashed for about 40 minutes after what the on-line retailer said was an internal software mix-up.
  •        In December 2000, Walmart.com collapsed several times over the weekend as the company refined the newly designed Web site. Upgrades and site improvements trig-gered three blackouts, ranging from an hour to 90 minutes.
  •        In November 2000, AOL suffered two outages that locked out 12,000 subscribers. A separate hardware problem also blanked email for thousands of users for 90 minutes.
  •        In April 2000, a six-hour telecommunications outage stopped hundreds of trains on the tracks and disrupted rail traffic throughout CSX’s transportation system.
  •        In March 2000, the Toronto Stock Exchange suffered a third blackout. The outage shut Canada’s main trading venue twice during the day, for more than 2.5 hours. These examples are just the tip of the iceberg. Most outages are unreported, as organizations will avoid the harmful effects of publicity and the likely loss in customer confidence.