On Thursday, March 29, there was the biggest failure in the work of the Telegram messenger at least for the last year: the service was unavailable for about four hours for users from Russia and other CIS countries, Europe, the Middle East and Africa. The company does not disclose how many of its users could face problems.
As explained on his page on Twitter, the founder and head of Telegram Pavel Durov, the cause of the malfunction was the disconnection of electricity, which is fed to the company's servers in Europe. "To make sure that such disruptions do not happen again, we established a more direct and continuous connection between our engineers and data center electricians, and also began to modernize our power-related equipment to make it more efficient," Durov clarified and brought Apologies for the inconvenience caused to tens of millions of users.
The founder of the company in his explanation also pointed out that "every fast-growing service has to face growth diseases".
On the eve of the malfunction, on Wednesday, March 28, Pavel Durov wrote in his Twitter, that since the beginning of this week, 700,000 new users have joined the messenger daily. And last week Telegram reported on overcoming the bar in 200 million active users a month.
Disconnected from the outlet
Telegram in recent years does not disclose information on how the infrastructure to support the work of the messenger. Only in January 2014 in the official Twitter account of the company it was noted that the servers supporting its work are located in five different locations around the world. In particular, servers serving Europe operated in London. However, now Pavel Durov told RBC that the company does not have servers in London, refusing to comment further.
There is no information on the design of Telegram systems and in documents for investors prepared in the framework of the ICO (initial coin offering - the initial placement of coins of crypto currency or tokens, special "tokens" giving their owners certain rights) of the Telegram Open Network (TON) platform. The company's presentation (a copy of RBC's) only notes that "the physical infrastructure of the company reflects the conviction of its founders in the need for decentralization" and Telegram has a lot of independent server clusters located on different continents and in different jurisdictions.
The managing director of IXcellerate data center operator Dmitry Fokin noted that the data center backup system can be organized according to various schemes and requirements depending on the required levels of object reliability. Reserves are built by installing diesel generators and uninterruptible power supplies with batteries that can be connected with the loss of industrial (urban) electricity. The data center must have a fuel stock and contracts for its supply to ensure autonomous work for a long time (usually not less than 12-24 hours). In the data center of the highest - the fourth level of reliability - it is usually guaranteed that due to possible failures or stopping of equipment, the center may be unavailable for no longer than 26 minutes a year in total. But there are data centers with a possible downtime of no more than 5 minutes and 16 seconds per year. "For companies that work online (365x24x7), it is very important to reserve not only the power supply of server clusters, but also the servers themselves," Fokin said. "It's expensive, but necessary." The cost of the backup power supply system averages 20-30% of the cost of the computer complex. "
The representative of another popular messenger, Viber, told RBC that the company's servers are located in a variety of data centers, so that the problem in one data center will not affect the messenger. "Viber servers are also located in several regions on almost all continents of the planet in order to increase availability and reduce the response time of the service," RBC's interlocutor noted. Representative WhatsApp did not answer the questions of RBC.
When did not work
At the end of February 2018, Telegram users complained about delays in sending and receiving messages. Representatives of the company confirmed the problems in the messenger's work with users in Europe, Middle East and North Africa. The source of the newspaper "Vedomosti" then noted that in Russia with a problem encountered a maximum of 10% of users, and a malfunction in the service for them lasted about a quarter of an hour.
A major malfunction in the messenger's work occurred in mid-December 2017. The complaints of users were received for the delay and the inability to send messages, as well as problems with authorization. Problems affected users from Russia, Ukraine, Italy, Germany, Iran, Egypt, Spain, Holland and several other countries. Pavel Durov linked the failure to the "incredible" growth in traffic.
In September 2017, in several countries in Southeast Asia, users could not access the application. The company explained the failure of the power outage and malfunctions in the emergency power supply system in the data center in Singapore.
A week before, Telegram users in Europe had complained about delays in sending messages. In addition, sometimes the history of correspondence was not loaded. Pavel Durov explained the problem by the failure of one of the server messenger clusters due to a surge in user activity. As a result, about 15% of users encountered the problem, which at the time of the crash were on the network.
In early February 2017, Telegram was not available to some users from Europe and the Middle East due to server problems. The problem was eliminated an hour after the first failure messages.