Opinion Forum

The Curse of Spam

April 16th, 2009

Anyone who runs a website with submitted content, like Opinion Forum, suffers from the curse of spam. We’re only about five months old, and it took the comment spammers a while to find us. Now we’re under daily attack. So far, we’ve been hit by almost exactly as many spam comments as we’ve received valid comments. Soon the spam comments will be far ahead of the real ones.

If there’s a hell with an especially hot corner, I hope all the comment spammers end up there!

Opinion Forum is protected in two very effective ways.

First, the website program has a feature for banned words and phrases. That protects us against comments with obscene and pornographic content. When a comment is received with banned language in it, it’s held for moderation. Then it can be edited or deleted, as appropriate.

Second, protection against spam comments is provided by the Akismet plugin program. It runs from a central service that checks all comments before they are published on the website. The program learns what is and isn’t spam. For example, if it mistakenly identifies a valid comment as spam, all you have to do is mark the comment as valid, and comments from that source won’t be identified as spam in the future.

Akismet lists comments it has identified as spam for you to review. It only takes a quick look and the click of one button to consign them all to oblivion. Most are readily identifiable as spam because of appearance and content. Some of the worst can be more than a full page of random words masking dozens and dozens of links to either commercial or pornographic websites.

After months of use, Akismet has identified only one valid comment as spam, and one especially crafty, normal-looking spam comment slipped through. That’s a phenomenal success rate.

Akismet can be used with any website that accepts submitted content.

By the way, this isn’t a commercial plug. Akismet is free for non-commercial use.

Articles written by Tom Carter
Tags: Akismet, spam
Categories: Life | Comments (6) | Home

6 Responses to “The Curse of Spam”

Brian | April 17th, 2009 at 6:23 am

That’s some incredibly sophisticated software. Did it come with the source code, or only the executable?

duggy | April 17th, 2009 at 6:41 am

Tom
I suppose it’s the nature of man to hinder and pick at others of the same species. Seems no matter how well things are working. someone will find reason to “fix it”. I fear that the Internet will become an even more unfriendly place in the future. It has long ago passed the novelty stage and is now a national asset. Control of the Web will soon be a major undertaking of friend and foe alike. World War Three will out of necessity be as much about Cyberspace as it will be about oil and Outer Space.

Tom | April 17th, 2009 at 7:30 am

Brian, Akismet is basically a WordPress plugin. In our case, it was already in the code of the theme Opinion Forum is based on. All that was necessary was to activate it and set it up. If a WordPress theme doesn’t already include it, it would have to be uploaded to the host server, then activated. However, as you can see from the Askimet website (link in the article), you can use it on almost any kind of site that includes submitted content.

One thing I didn’t mention before is that Akismet works against comment spam regardless of the language it’s written in. Believe it or not, we actually get spam written in Russian fairly frequently.

And yes, it is pretty incredible. It would be a nightmare to try to keep up with deleting spam comments that make it to the site. As I said, some of them are more than a page long, and with the column width of comments they would be several full screens in length on the site.

Tom | April 17th, 2009 at 7:37 am

Duggy, you’re right. The threat of cyber attacks is real, and it’s something we have to deal with now. You can read about one important defensive measure at Cybersecurity and Defense. It’s the kind of thing that makes a lot of people nervous, but it’s essential.

Kevin | April 17th, 2009 at 10:54 am

Comment spam is the reason I switched to using Haloscan script on my blog. For a long time we ran with the Movable Type comment script that came with the blog template – which I paid good money for. MT has plug ins for combating spam but they require a vigilence that I just wasn’t prepared to invest the amount of time necessary to stay on top of the game. Slowly but surely comment spam began to make it past the filters and so I just dumped it and went with Haloscan. It’s far from a perfect solution. For one thing I don’t like that comment threads aren’t inherently linked back to the post they sprang from – so that if someone Googles in on an old post and leaves a comment, I almost never know what post they’re ranting about. But… Haloscan is very, very effective at keeping the spammers away.

A huge part of the problem is that the IT skills and computing power of the big-time spammers matches or exceeds the IT skills and computing power of the bulk of those generating anti-spam plug-ins or managing them on individual blogs. Several years ago when I was much more involved with the MT techie community the MT code writers openly admitted as much. They said that many of the big-time spammers are actually involved with the anti-spam IT community so that they can stay on top of the technical advances and design ways to defeat them.

For example, the Blue Oregon blog also uses WordPress and is owned and operated by an IT pro. Even so, by virtue of the sheer number of page views (nearly 5000 per day) and unique visitors (nearly 3000 per day) they get much more focused attention from spammers than small fry such as myself and a fair amount of it gets by the filters.

I noticed that Joe Gandleman at The Moderate Voice, also using WordPress and getting much more traffic than Blue Oregon, has opted to use a new comment platform called Disqus which combats spam primarily by qualifying commenters – you have to have a valid Facebook account to post comments via Disqus. Back channel chatter from Blue Oregon insiders is that they are seriously considering going with Disqus.

Tom | April 17th, 2009 at 11:44 am

Kevin, I know a lot of people are using Haloscan for the same reasons, but I have the same reservations about it that you do. Akismet requires a small amount of attention, granted. But if you don’t do anything, Akismet dumps the spam comments after 15 days. Given the incredible accuracy of Akismet (so far), there’s very little risk of losing valid comments by just letting it dump them later.

I realize that someday the traffic and comment volume may make this a less than ideal solution. But until then, I’ll probably stay with Akismet to avoid creating obstacles to easy commenting, such as captcha boxes, mandatory registrations, etc.

I don’t know how well Akismet would work with platforms other than WordPress. They claim it can be used with almost any other platform, but I’ve never tried it.

You’re certainly right about the ingenuity of spammers. I just hope there’s always an effective way to defeat them. And, like I said, I hope there’s an especially warm place in hell for spammers.