Combating Internet Spam Spiders

More often than not, your site will be hit by a spam spider, or web crawler. A spider is a program that searches for information on the World Wide Web. Spiders are used to locate new documents and new sites by following hypertext links from server to server and indexing information based on search criteria. Among other things, it is looking for a standard formatted email address (name@domain.com). Once it finds this address, it will grab it, store it, and then sell it. Thus far, this technique has been a highly effectively method of gathering email addresses.

There are a number of procedures web designers can implement to protect against spiders. Of the many, here are three that work well:

I. Use HTML Symbols

HTML Symbols deceive spider software by making formatting modifications.

• All email addresses use the "at" symbol (@), and spiders look for this symbol. Because HTML symbols are written directly into the HTML code, they need to be interpreted by the browser for their actual meaning. The HTML symbol for the @ sign is "&#64."
• In the HTML code, replace the @ sign with the HTML symbol that represents it: <a href="mailto:name&#64;domain.com">
• This method is as innovative it appears. However, it is still sometimes caught by more sophisticated crawler software. Most crawlers are not this advanced, but they do exist.

II. Use JavaScript

JavaScript can be used to thwart more sophisticated crawler software.

Using Javascript, we can break email address into four separate components:

• Name
• The @ symbol
• Domain
• Extension

<script type="javascript">
name ="you"
domain ="domain"
extension =".net"
document.write('<a href="mailto:' + name + '@' + domain + extension +' " ' +' >e-mail link here ');
</script>

• Define the three variables (name, domain and extension) and simply concatenate those variables within a “document.write” statement. (NOTE: Use HTML's anchor tags within the write statement, otherwise your browser will not interpret the email link correctly.)
• Javascript is “client-side.” This means that although spiders may not be able to interpret your email address, browsers with Javascript capabilities turned off will not be able to either. Your email address will not be displayed to the user with Javascript disabled.

III. Use Server-Side Programming

Server-Side Programming enables users with Javascript disabled to send you email.

If your server supports server-side processing (Perl, ASP, Cold Fusion, PHP, etc.), simply design a feedback form and call a server-side script to process the data behind the scenes.

NOTE: If your server does not support any server-side programming language, you can use one of the aforementioned methods.