It is a Sunday afternoon, a few days after the largest hacker’s attack against web community I have ever seen. I’m siting in front of my notebook reading the Web Hosting Talk’s explanation about the recent problems they had. And the more I read, the stronger is my conviction that this is a classic failure story. I’m sure that leadership of the world’s largest web hosting forum will disagree with this and will interpret this article the wrong way. I’m curious whether the WHT leadership would admit their own faults which led to the current situation. WHT has a tradition of intolerance of opinions that put its rules, guidelines and policies on public discussion. But let’s see the story.

The world’s premier web hosting forum Web Hosting Talk (WHT) has recently been hit. The attack sparked a lot of buzz within the hosting industry. I have already spoken to some of my good friends who are members of WHT members and they told me that a large part of their posting history has completely gone. According to WHT’s forum leaders admid that and said, the attack on the community they manage was very well revolved in attacker’s mind. The attackers overcame the forum’s security measures and accessed it “via an arcane backdoor”, which according to Denis Johnson (WHT nickname SoftWareRevue) was protected by additional firewall. The team of the world’s biggest web hosting forum admit that the hacker has deleted WHT backups and 3 databases – users, posts, threads.

The forum’s maintenance team said that they still can not say whether the attackers accessed the private message data of their members. The WHT claims that “Absolutely no credit card or PayPal data was exposed”? So what happened and what would be the consequences for WHT and its members.

Who’s Behind the Attack?

“Do we know the motivation behind the attack?”, this is a subtitle of the WHT’s explanation of what happened published in a thread titled “Couldn’t access WHT recently? Get the full story why…“.

“We don’t know enough at this time, so any insight would be purely speculative in nature. WHT is a platform where positive and negative information is shared and exposed about business and individuals. Under TOS policy, we cannot edit or remove user-generated content at the request of an unsatisfied third party. Therefore, WHT tends to become the target for disgruntled individuals and businesses”, this is the explanation of the forum’s leadership.

The only think I would add here is that WHT’s leadership would clarify what do they mean by saying “disgruntled”. I would call the attack against the WHT a criminal act and I call for an investigation that would bring someone in court. But I wouldn’t call anyone who carried out “sophisticated and calculated” attack disgruntled. If we have to think of the attackers of people disgruntled of something we definitely have to discuss what is the thing (Policies, Rules, Guidelines, etc.) they would be discontented with.

Lost Members Data And History!

The big issue for WHT is not that the forums was attacked. This is something that would happen to anyone and I certainly believe that we must be tolleant and to give WHT’s system administrators some time to restore the members’ data, posts, threads and etc. Unfortunately they will be a lot of voices who will not miss to ask the questions like “Why does the WHT team isn’t able to restore the member’s posting history?” and also “How WHT member can assure members that their personal data (IP addresses and etc.) were not exposed or stolen?”. Take a look at WHT’s explanation about forum’s restore.

The offsite backup, the onsite backup and the operational data were destroyed by the attacker, so we’ve resorted to a physical back-up of last resort. Unfortunately, we are experiencing difficulty restoring from our most recent physical backup. At this point, October is the most recent backup that we were able to restore. We continue to work to extract data from a more recent set of DVDs.

I will only add that if this message was released by a web hosting provider that faced similar problems, it would force it to exit hosting business.

The forum leaders explained that the have locked down the forum’s infrastructure to avoid any further damages and of course to restore the website. “We also had to block the potential for a repeat attack. Now we are working on investigating how much prior data is restorable, reinstating premium memberships, contacting business partners, and communicating with the community members. We are also doing everything possible to identify the attacker and bring them to justice”, wrote Denis Johnson and added that WHT’s team is “working hard to restore trust among community members”.

He also explains that the forum has 3 protected data back-up units with 1 offsite behind a firewall and a 4th physical data back-up layer. However the hackers, according to WHT, have “deliberately targeted our data back-up systems”. The forum leaders admit that their disaster recovery plan didn’t anticipate such a scenario.

An Advice to WHT Members

Denis Johnson explained that password encryption technology which WHT uses is “strong for securing non-financial data” and suggests forum members to change their passwords frequently and not use the same user name and password for the forum as they may use for online banking.

The forum leadership admit that as a result of the attack to their system, now WHT members e-mails are now vulnerable to SPAM. “The attacker posted stolen email addresses on file sharing sites”, says DEnis Johnson in his thread.

What Happened To WHT Members Accounts?

The forum is now running a backup version from October 2008, because its administrators were not able to recover a recent copy of the website. The official statement says that the website is “temporarily using a version of the database from October 2008”. But all new members after this date have lsotheir accounts. I’m very curious what would happen to the account of someone who registered since October 2008 if the WHT succeeds to restore its registration, but at the same time the member has registered again using the same data (Username, IP address, etc.)?

Usually WHT closes accounts like these. It is now important WHT members to be assured that they will not be banned because the forum’s troubles. “We may still be able to recover your account, but we don’t know yet. Please register with the same username you used before”, says Denis Johnson.

By Out Of Excuse

After all that happened and after being unagle to restore a recent copy of the website WHT says “We take the protection of user-contributed data very seriously…”, and also “yet even institutions that spend millions of dollars a year on Internet security are exploited. Anyone recall NASA being hacked some years back?”…

What Experts Say?

I have opened a discussion about WHT’s attack in Linked In so I would able to here what do web hosting professionals think about the attack. Here is the first comment. It has been posted from Andre van Vliet (Administrator of Webhostingtalk.nl and CEO of SolidHost)

“From what I gathered, WHT.com is taking the position of victim of a crime. Things are being said such as “even nasa got hacked a few years ago – this hacking could’ve happened to anyone”, says Andre.  He adds that he is particularly sceptical about the position which WHT.com is taking here. “Instead of saying that it could’ve happened to anyone, they should be taking their responsibility by simply stating they screwed up”, wrote Andre. He also added:

Don’t get me wrong here, I’m not at all saying that the attack can be justified in any way – it is definitely a crime. However if you leave your car running while you’re at a shop and it gets stolen – of course you’ll be pissed off, but you basically allowed it to happen yourself.

What people still don’t seem to realize is that the majority of the internet is insecure – and nobody cares until it actually goes wrong. In a hacking attack like this – where more than likely things could’ve been done to prevent this from happening, the wht.com administrators should take the responsibility for the fact that this happened in the first place.

If this somehow happened due to a screwup at the end of a third party (from WHT.com’s perspective), then the wht.com admins are obviously not to blame – but in my opinion in a case like that, that should be publicly stated. The way they put it right now doesn’t indicate anything like that.

I have asked Andre a question – “You are administrator in Dutch WHT, so you are probably the right person to ask. Do you find acceptable that WHT wasn’t able to restore a recent back up…? They now run one from October 2008.” – and he answered:

Considering that the majority of WHT is based on a free service, I cannot really comment as to whether or not it is acceptable. However if I were a paid member or advertiser, I certainly wouldn’t find that acceptable at all. The fact that it is even possible to remotely delete all files (all backups and the normal files) is quite ridiculous. At least one copy of the backups should be stored on a server that is not remotely accessible.

Another member of LinkedIn group “Hosting Industry” Peter Zendzian, Managing Partner at ZZ Servers said:

I am quite certain WHT has more security in place than the average web based company and that still did not prevent this. From reading how the attack happened, it sounds as if the attacker found a method of entry that was by no means a front door. Social engineering, upset employee/contractor, unsecure laptop. There are so many ways they could have gained entry. This no matter how it appears is a crime and until all the details are available, it is impossible to lay blame anywhere. And, yes this can happen to anyone. That is one reason why security specialists are so expensive.

Jeffrey J. Hardy added that it has been justly said that the price of liberty is eternal vigilance.

“The same may be said of security. I have no inside information regarding the particulars of this attack, but I know that if a professional thief wants your car, no alarm, Lojack, or other security device will prevent its unlawful acquisition. The only responsibility–in my opinion–that WHT has in this is not having an isolated backup program in place. If you have a sound, off-site, unconnected restore program in place, you should not loose more that a week’s data. If you are NASA – only a day. I empathize and wish them well.”

“The leader let us all down and now we hear excuses” added Tariq Hyder.

The Future For Web Hosting Talk

I think that it will take some time for the forum to rebuild its image. It is not possible a failure like this not to affect on WHT’s business. I would expect a decreased interest of hosting businesses to advertise in Web Hosting Talk (of course I hope this not to happen).

Another thing is that some of the most active members of the community may get discouraged and it would take some time before they get motivated to again actively participate to the discussions. I would also add that WHT leadership should spend a lot of time and energy in assuring members that nothing happened with their personal data.

While WHT is in trouble, other web hostign forums have a chance to attract attention and to increase influence within the web hosting industry. Of course it will be difficult for them to take advantage from the WHT’s misery, the same way as it will be hard for WHT to recover from the attack. I can not project what would happen and whether WHT will stay on top as a hosting community. But if anyone wants to sit on the throne they definitely have a a lot of work to do. They also should hope that those who have WHT accounts will be as disappointed of the forum’s business as they are during the first days after the attack.

Of course Web Hosting Talk has 2 big advantages over its contenders – the dedicated team and the INET’s support. These two would be enough for the WHT to stay on top.