EZBOARD ATTACKED? That's what the CEO says.

[r0ss]

p079.ezboard.com/fezboard...=504.topic (http://p079.ezboard.com/fezboardimportantannouncements.showMessage?topicID =504.topic)

On Tuesday May 31 ezboard's systems were the victim of a very precise and malicious internet attack. This attack resulted in the loss of a significant amount of current and historical board postings and interrupted services across all systems.

[r0ss]

complete data restoration may not be possible

attacker deleted current board and back-up data

48 hours to get data back to approximately 9,000 boards

Regrettably we may not be able to restore data to all boards that lost data.

Second to restoring board data we are gathering information on how the attacker went about his or her business and what evidence was left behind.

We we keep you updated as the restoration progresses.

[bladepusher]

Guesses anyone? :chip1

[3parrots]

It wasn't me.:gamecock2

[Toy Ranch]

I think it was Rodney.

[r0ss]

Prolly it was JustJG

[r0ss]

Fucking ezboard. Half my fucking archives wiped clean, fucking 2/3 of my community chest missing. Fucking bastards, some fucking shit hole ezdude, the **** is that?

Damn. It just hit me, this is depressing. Fucking just close down TRB and fucking be done with the whole fucking mess.

[flawedplan]

I feel the same way. And though all the scholarship is gone from my board to support this contention, I firmly believe that allowing for ugly, defeated, vengeful, self-pitying and hopeless feelings is the way to get passed them, so, yeah, let's go with it.

[r0ss]

Just what I needed, Robin. I got a little smile from reading that.

I'm so glad you're here. Thanks.

[flawedplan]

MY WORK WAS THE MOST IMPORTANT SCHOLARSHIP ONLINE IN THE HISTORY OF THE INTERNET!

[sjot]

Hang in there, Ross. We need you and we need TRB! I peeked in the archives - I am so, so sorry. :(

FP, I'm so sorry about the losses at your board too. :(

[Indigowhite]

My condolences to anyone who owned stock in EZBoard... This has to be one of the top 20 fubars in recent web history.

Sorry about everyones loss of content :|

[Indigowhite]

Prolly it was Bard

Prolly it was JustJG

Hrumph ..only if they got in peeing range of the server farm...

Actually was probbaly one of their min wage server monkies who after a twelve hour shift fell asleep and knocked his 7/11 coffee into the premiere 486 dx 66 server :insane

Oh 486?

My mistake..was prolly a Wang .... :insane

[Rawbunzel]

On the bright side none of us will ever have to own our old words again! We can all start fresh and new.:spin1

I'm going to miss all those old words though...:cryhard

Newts lost all its old posts too.Depressing. Some of those posts were all that was left of long gone online friends.

[z Irene z]

I checked the Waybackmachine:

www.archive.org/ (http://www.archive.org/)

Unfortunately, ezboard apparently blocked the ability of that site to archive ezboard pages.

I also did some searches on google. The only cached ezboard pages appear to be pages which list threads, not the posts on any specific thread.

[happy is here]

<blockquote><strong><em>Quote:</em></strong><hr>Fucking ezboard. Half my fucking archives wiped clean, fucking 2/3 of my community chest missing. Fucking bastards, some fucking shit hole ezdude, the **** is that?

Damn. It just hit me, this is depressing. Fucking just close down TRB and fucking be done with the whole fucking mess. <hr></blockquote>

OMG, Ross, don't be so angry and calm down.
Don't even think about closing TRB!!!

We need the TRB and we need you and I love this board. Someone might criticize me again for saying that, but to whom it may concern know that I don't give a ****!

So, we're sorry for Ross' and everyone's loss of posts, threads, archives, etc.
The beat goes on, we are still here!
We pick up the pieces and rebuild!

<img border=0 src="http://ross.adnetsol.com/graphics/art/1th_drum.gif" />

<img border=0 src="http://ross.adnetsol.com/graphics/art/1neko6.gif" />

[r0ss]

We pick up the pieces and rebuild!

Thanks, Happy. I just need to keep occupied until ezboard does everything they can do.

I'll do a bunch of music business work. Throw myself into my work for awhile.

[happy is here]

<blockquote><strong><em>Quote:</em></strong><hr>I'll do a bunch of music business work. Throw myself into my work for awhile. <hr></blockquote>

That's good, Ross!

Here is a hug for you! <img src="http://img.photobucket.com/albums/v104/Happy14/cuddle2.gif" alt="Image hosted by Photobucket.com">


A hug for everyone who feels bad about this mess.

<img src="http://img.photobucket.com/albums/v104/Happy14/cuddle2.gif" alt="Image hosted by Photobucket.com"> <img src="http://img.photobucket.com/albums/v104/Happy14/cuddle2.gif" alt="Image hosted by Photobucket.com"> <img src="http://img.photobucket.com/albums/v104/Happy14/cuddle2.gif" alt="Image hosted by Photobucket.com"> <img src="http://img.photobucket.com/albums/v104/Happy14/cuddle2.gif" alt="Image hosted by Photobucket.com"> <img src="http://img.photobucket.com/albums/v104/Happy14/cuddle2.gif" alt="Image hosted by Photobucket.com"> <img src="http://img.photobucket.com/albums/v104/Happy14/cuddle2.gif" alt="Image hosted by Photobucket.com"> <img src="http://img.photobucket.com/albums/v104/Happy14/cuddle2.gif" alt="Image hosted by Photobucket.com"> <img src="http://img.photobucket.com/albums/v104/Happy14/cuddle2.gif" alt="Image hosted by Photobucket.com"> <img src="http://img.photobucket.com/albums/v104/Happy14/cuddle2.gif" alt="Image hosted by Photobucket.com"> <img src="http://img.photobucket.com/albums/v104/Happy14/cuddle2.gif" alt="Image hosted by Photobucket.com"> <img src="http://img.photobucket.com/albums/v104/Happy14/cuddle2.gif" alt="Image hosted by Photobucket.com"> <img src="http://img.photobucket.com/albums/v104/Happy14/cuddle2.gif" alt="Image hosted by Photobucket.com">

[GammaWaif]

This is very hard. I don't even have the years of history you all do, and it's hurting. The Icehole is completely wiped, nothing left. My home board.

Of course we can rebuild, and move ahead. But we will also grieve, in the meantime. So, I'm grieving.

They say we are are memories. . .the boards lose some of who they are as entities, when their archives are lost, I think.

"Preserve your memories, they're all that's left you." - Simon & Garfunkle

[TerrorEd]

I've been trying to post, can't seem to get through the process. :wall2


"Damn. It just hit me, this is depressing. Fucking just close down TRB and fucking be done with the whole fucking mess."

No way! :eek That ain't you, Ross. You're no quitter. If anyone can get to the bottom of this shit, you can. You know the in's and out's of EZ better than most folks. It would appear you've got membership support.... ;)

I hope this post goes through..... :insane2

Ed. :hat

[george]

iam pretty sure they will give credit back for the community chest thingie hummmmm i may be mistaken but wasnt there 456,987.98 in there??? shifts eyes

[browneyedk]

Well, I hope they find the jerk that did this. Very sad indeed. I am a member of several other boards and seems as if I too lost some recent monetary contributions I made.

Really sucks!

[19microbes55]

The Icehole is completely wiped, nothing left.

Yeah, yesterday it was gutted but the carcass was still there. Some time in the wee hours of the morning the wolves got the carcass. Nothing left this morning. Not a single word.

Fat lot of good the damage control I tryed yesterday did.

This thing is STILL happening. It's still cascading thru their system. If they can't restore the system until they can stablize it.

[mimiOh]

I want to cry..gone..poof..all of it..

I knew it was an attack, it had to be the way things were being eaten up.

[Meerkat70]

I still can't bloody well post anywhere. If this reply actually goes ontot he board, I may throw a party.

[blondieia]

My Gawd, the people that did this must have stolen a fortune in community chest money. I'm sure that was the main motivation; the rest of it is just a smokescreen.

[Trouble1957]

If that money is gone bye-bye, then I have three little words that may apply.

CLASS ACTION SUIT

[giiglehoot]

I'm kind of wondering around lost myself. sure glad I could still get to this place. any other damage reports coming in?

:mad

[blondieia]

I've been trying a lot of boards this morning. It's pretty hit-and-miss. On some boards I can't post at all, on others I can post in certain forums.

[19microbes55]

How does a hacker even get at the backups unless it's an inside job? Backups shouldn't even accessable to the internet.

p079.ezboard.com/fezboardfrm87.showMessage?topicID=7236.topic (http://p079.ezboard.com/fezboardfrm87.showMessage?topicID=7236.topic)

[z Irene z]

Could a hacker have created a "virus" which didn't activate immediately? That's the only way I can see the backups being affected.

[19microbes55]

p079.ezboard.com/fezboardfrm87.showMessage?topicID=7256.topic (http://p079.ezboard.com/fezboardfrm87.showMessage?topicID=7256.topic)

Are they now saying they have not even "got to the root of the problem"?

[blondieia]

19microbes..yours is one of the boards I haven't been able to post to at all.

[19microbes55]

yours is one of the boards I haven't been able to post to at all.

It was wiped totally clean last night.

I did set up a temporary forum a while ago that seems to work normally.

You can't reply to any threads in the rest of the forums.

[chococake]

I'm so sorry Ross, and every other board owner, and posters who lost their home boards. This seems to be the only board that my posts are going though.

Has there been anything on the news about this? This is a major hack, yet I haven't seen anything any where.

[mivona]

It is reported in the news here: www.halifaxlive.com/artma...7762.shtml (http://www.halifaxlive.com/artman/publish/ezboard_010605_7762.shtml)

The only report on Google that I could find.

[chococake]

Thanks Miv. Strange that there isn't more.

[KatieP]

It doesn't involve a missing woman or a celebrity in a sex scandal. Of course the US press isn't covering it.

[3parrots]

Interesting to me:
the founder is Chinese/Vietnamese, I think,... and the company is in SF.

I haven't known many (foreign born ?) Asians connected to the social aspect of communications, ie. message boards. Bravo!


"Ezboards was founded in 1999 by Vanchau Nguyen and is based in San Francisco."

www.halifaxlive.com/artma...7762.shtml (http://www.halifaxlive.com/artman/publish/ezboard_010605_7762.shtml)

[r0ss]

Important ezAnnouncement:
ezboard Attack Update - 12:00 PST 06/02/05 We are continuing to restore posts and are working on additional means to restore further data. At this time we do not have an ETA for the completion of the restore process. Indications are that the restoration will be completed in the next 24 to 48 hours. We appreciate your patience and recommend that you look for updates in the Help forums. Please help us to get you answers more quickly by reading the sticky topics in the Help forums before posting your question. Many times your questions will have already been answered.

[Toy Ranch]

They should get in touch with Ray Romeo and ask him what to do... he has experience with these things.

[19microbes55]

p097.ezboard.com/bthegreatezboarddisasterof2005 (http://p097.ezboard.com/bthegreatezboarddisasterof2005)

[ramonaleona]

posting posting one two three...

[DELMER DELMER]

roger that dusa

[ramonaleona]

that was the first i was able to post on ezboard today - er - i think today - the days are all a blur to me anyway - but - you know that...

[DELMER DELMER]

dusa , if ezboard is working then we all can get our heads back on straight and not be confused anymore :lol

[ramonaleona]

hey you - yoo hoo - delmer - i'm talking to you over in another place... :lol

[DELMER DELMER]

from the looks of it...ezboard is all ours right now dusa :lol

[ramonaleona]

yours and mine - mine and yours - ours - ah my hero - delmer...

hey - let's get hitched...

the moment is ours for the taking...

tomorrow is another day...

it will all - (most likely) - be wiped clean again anyway - and - noone will ever know...

yes - let's do it...

- smooch...!

:b

[zeldas]

and the latest

osted: 6/3/05 13:46
New Post ezboard Attack Update - 06/03/05 Dear ezboard Family,

Good morning and thank you for your patience as we continue to restore your data and return our systems to normal operation.

This will be a long update, as I want to be as thorough as possible and incorporate threads from the many phone calls and e-mail discussions I have had with ezOps, admins and users in the last few days. I hope that by being open and explicit we can give you a better understanding of what has happened, where we are in the restoration process, and when we can all expect to get back to normal.

I've broken all this information into topic areas to make it easier for you to find the topic that you personally care about the most.

When will my board get its data back?
The most difficult and painful thing for me to report is that it will be impossible to restore full data on all boards. We will send out a Global Announcement once all possible restoration procedures have been completed. During this process, you can expect to see some slowness on the boards, but we’ll return to normal speeds as soon as the restoration is complete.

I realize this creates great uncertainty for many of you still waiting for data restoration, and so I want to give you the following, much more detailed description of the data restoration process we are undertaking in our efforts to get the most back we can, on the fastest time frame possible:

Data restoration began the day of the attack with the restoration of eight servers that we could salvage immediately. Sadly, no other intervention was successful at that time.

Since that time, we have been assembling data from three sources: the remaining historical backups of some 8,000 Gold and trial boards from early May, the current data recently written to hard drives since the attack, and data in RAM memory since the attack.

The actual restorations using this assembled data have been underway since Thursday afternoon, and are being done on a server-by-server basis. The process is automated and each server takes approximately 6 hours to restore. We can restore between 8 and 10 servers in parallel. To some of you this may seem like a long time to restore data on a single server. And it is. However, we are using an innovative process to maximize the data recovery and testing it thoroughly to make sure that (a) we don't erase the posts that have been made since the attack and (b) the data is being properly restored.

When this server-by-server process is complete, we have one final source of data that we may be able to use for restoration. With quick thinking and fast action on the part of your technical team we were able to do an emergency backup of active data. To be clear, this is by no means all the board post data, only the active data at that time. We will be restoring this data as a last step in the process.

We are working as fast as possible on the restoration process - believe me, the faster this is completed the happier we all will be.

What happened in the attack?
What we know is that late in the evening on the holiday Monday, May 30 here in the US, a person with significant and specific information attacked our systems. The attacker was successful in erasing historical board posts and the backup systems, which were password protected and on a separate network. They did not erase the posts of boards that were active on the system at the time.


What more can you tell me about the back-ups?
The issue was not the back-up systems themselves. Our back-up systems have been working well and are separate from our production environment. They are not accessible without inside technical information and passwords. The issue was that either through luck or (more likely) specific information, the attacker was able to delete both the historical files and the back-ups.

What are you doing so this doesn’t happen again?
We have put additional safeguards in place to reduce the opportunity for this to happen in the future. We will also be undertaking a highly detailed security and back-up audit in the coming weeks.

You should also know that we have been Alpha testing the new community software for three weeks and in addition to increased security, redundancy and back-up systems, it will have the ability to allow ezOps to back-up their communities (in addition to daily and weekly incremental backups) to their home computers in an automated fashion. For some ezOps this should provide an additional level of safety and peace of mind.

Can't you give us more frequent updates?
We have been updating you as soon as there is new information. Since the attack we have put out five global messages and have had two conference calls with ezOps and admins. In addition, the Customer Service help desk and Support Forums have been very active. On a more personal basis, Steve DeMello and I have been responding to hundreds of e-mails from users, ezOps and admins. As of 7AM PST on Friday we have responded to every e-mail that has been sent directly to the two of us.

If you are looking for ways to get up to date information fast I recommend reading p079.ezboard.com/fezboard...ouncements first. Please read the materials already available before contacting CS -- the response queues are pretty full, as you might imagine.

What can you tell us about the attacker?
I can tell you that we have suspects, we have evidence and we are working with the FBI. I can also tell you that we will be working to have the attacker punished to the full extent of the law.

While we do have evidence, we can always use more, good, factual evidence. We are offering a $5,000 reward to the person or persons that provide us with factual information that leads to the direct arrest and conviction of the attacker. We are not interested in conspiracy theories, finger pointing or childish pranks. We are interested in information such as community postings or blog entries where the attacker is bragging about the attack and other factual, real information. Please e-mail information to legal@ezboard.com. When you e-mail please provide a link to the original material AND a screen capture of the original material or at the very least, copy and paste the original material into your e-mail. Please be smart, ethical, law abiding and honest about the way you find the information your provide to us.

Now the fine print: A reward will be paid to the person or persons that deliver to legal@ezboard.com information that leads to the direct arrest and conviction of the person or persons that attacked ezboard.com earlier this week. Should more than one person provide identical or substantially similar information that is used for the arrest and conviction of the attacker, the reward will be split equitably amongst the providers of that information.

Will you be providing Credits for Gold Boards?
Yes we plan to provide credits. We need to understand the scope of the data restoration before we decide on the amount of the credit. I would anticipate further news on credits to be available in the next week.

What happened to my Community Chest Balance?
The Community Chest balances were deleted as well, but we do have a back up of that data from a few days before the attack. We need to recalculate the actual Community Chest balances by taking the balance as of the date of the backup, adding Community Chest contributions and subtracting payments from Community Chest. We hope you agree that restoring Community Chest data is a second priority to getting the board posts restored

Please don't worry about your Community Chest balance, it will be restored and updated.

So, where do we go from here?
First let me say that we at ezboard are deeply sorry about the attack this week. The person that did this to us probably had a great chuckle when they pressed the send key to initiate the attack. Little did they know that they would affect so many lives in so many ways. I have heard from cancer patients, historians, mothers and schoolteachers about the way that their boards were damaged and were of vital importance to them. What the attacker did was take a great big piece of many peoples lives including the folks here at ezboard. For that, I hope we catch the attacker and return the favor by taking a bit of their time - behind bars.

We are working to restore the boards as fast as we can. We know your trust in ezboard has been deeply shaken. Now we are trying to rebuild that trust, and are realistic that this will be difficult and take time.

We plan to keep messaging you on our progress. Some of you may have already seen data restored as we started the process on Thursday. Please check p079.ezboard.com/fezboard...ouncements for updates.

In closing, we are working to get systems back to normal and believe that they will be back soon. We are fixing the security and backup processes now and will be performing a security and back-up audit in the coming weeks. Thank you for being an ezboard user, and for reading this, and we hope that you will continue to use ezboard. Let’s put our communities and our network back together, and bring the attacker to justice.

Sincerely,

Rob

Robert Labatt
CEO
ezboard, Inc.

robertlabatt@ezboard.com

reply to this message

[zeldas]

* Yawn Ross please delete one

[DELMER DELMER]

ultra totem! :lol

[z Irene z]

I just checked and the threads on the Gaffan memorial forum at the Caboose are intact.

[Toy Ranch]

Joe's Forum had been wiped clean... but it seems to have been restored. :)

[chynna]

:)

Thanks, Irene & Bobby.

[Solar Destroyer]

Yup. They HACKED EZ BIG TIME!

Welcome, to the World Wide Web. :evil

[Bard77]

I'm STILL losing forums!

I lost my poetry forum and our game room overnight!

Every poem ever contributed.

All the history....

"Black Tuesday"

WTF!

[Bard77]

I'm STILL losing forums!

I lost my poetry forum and our game room overnight!

Every poem ever contributed.

All the history....

"Black Tuesday"

WTF!

[tekobari]

I lost my poetry forumPraise jesus.

[r0ss]

p079.ezboard.com/fezboard...=506.topic (http://p079.ezboard.com/fezboardimportantannouncements.showMessage?topicID =506.topic)

p079.ezboard.com/fezboard...=444.topic (http://p079.ezboard.com/fezboardfrm98.showMessage?topicID=444.topic)

p079.ezboard.com/fbugsfrm...=130.topic (http://p079.ezboard.com/fbugsfrm177.showMessage?topicID=130.topic)

http://img.photobucket.com/albums/v240/prospectplace/Krazy%20K%20Designs/ezgoldribboncampaign.gif

[Solar Destroyer]

I'm sorry to that, Rod. Hope you had it saved somewhere besides just EZ.

[Solar Destroyer]

meant = 'sorry to HEAR that'.

That's what I get for posting and watching COPS at the same time. :)

[Solar Destroyer]

AAAaaand so Naaaturallllyyyy the correction winds up at the TOP of the NEXT page. :wall

[Solar Destroyer]

still support the no-edit rule though. :smokin

[Zilvy]

Solar Destroyer is a Potem Toaster!!:iroller

[chynna]

:lol >>> Zil.

[r0ss]

How many servers have been restored?

At this point there is no such thing as a completely restored server.

Let me give you some background to put this in perspective. We have gone back into terabytes of non-back-up data looking for any board data we can find. Where possible we’ve put this data in the restore queue. When we learned of the hack we also dumped the contents of RAM from each server onto a hard drive. We have been restoring that data as well. With the main mode of data restoration gone (the regular back-ups) we have had to create new restoration processes from scratch. That means writing software, testing it and then putting it into production - all on no sleep. In some cases working slower IS safer - but more frustrating for you and for us.

At this time we are restoring from five different data sources. Each source requires a separate step or process to restore data. Roughly half the servers have finished the first step of restoration and are in the second step of what are ultimately five steps of restoration. No severs will be fully restored until they have passed through all five steps. We cannot give you specifics of what servers/boards are where in the process and we cannot have your board opt out of the restore processes.


Why am I still losing threads that were on my board after the attack?

We have isolated the problem and fixed three issues that were leading to thread loss. These fixes have been applied to all servers and you should not be seeing additional losses after today.
Thank you to the over 500 people who submitted information on this problem. We do not need additional information at this time, but if we get new reports of new data loss after tonight we may ask you for more information.

[z Irene z]

Threads seem to have been restored overnight to most of the forums here.