Technology Review: Spam Wars

<= ![endif]>

On the web a= t

Spam Wars=

The proliferation of junk e-mail = is threatening to overwhelm the Internet. Software companies are rushing to bu= ild defenses—but will the new technologies do more harm than good?=

Spam King: Alan Ralsky spewed tens of thousands of e-mail sales pitches per hour, bringing on the wrath of Verizo= n. (Photo courtesy of AP Photo/Paul Sancya)

By Evan I. Schwartz
July/August 2003

Operating 20 computer= s in an abandoned schoolhouse in Rockford, IL, Jay Nelson worked with relatives = to set up more than a dozen shell companies, renting equipment and Web hosting services using aliases such as “Art Fudge.” Nelson and his associates then “hacked into AOL e-mail accounts,” states one l= egal motion filed by AOL, and overwhelmed members with links to pornographic Web sites such as pamsplayhouse.com.

In 1999, AOL won a court injunction barring Nelson fr= om such activities and fining him $1.9 million; nonetheless, he and his collea= gues subsequently sent another billion e-mail messages—triggering 25 perce= nt of AOL’s spam-related customer complaints over the next two years.

Alan Ralsky, by contr= ast, seems almost respectable. While trying to overcome a past littered with fra= ud convictions, a court-ordered fine, personal bankruptcy, and a brief jail st= int, Ralsky in 1997 heard about a new Internet opportunity. Repudiating pornogra= phy to his wife, Ralsky rented mailing lists and set up servers in his basement, according to media interviews he gave last year. Pitching mortgages, vacati= ons, and online pharmacies and casinos on behalf of others, he boasted of thousa= nds of dollars per week in sales commissions. After moving into a $740,000 hous= e in a Detroit suburb, Ralsky set up another basement operation that was soon spewing tens of thousands of messages per hour, relayed through servers in Dallas and in Canada, China, Russia, and India. In 2001, Verizon Internet Services sued Ralsky, charging him with unauthorized use of its network.

Nelson and Ralsky are= just two of the many faces behind spam. But according to Jon Praed, an attorney = with the Internet Law Group, an Arlington, VA, firm hired by the plaintiffs in b= oth of these cases, big-time spammers have a common profile. “They have n= ot been successful in anything else,” he says. “They are hackers g= one bad, or they are crooks gone geek.” They also sit at the center of far-flung conspiracies to conceal their actions. (Neither Nelson nor Ralsky returned phone calls from Technology Review.)

The spam crisis is ha= rdly a secret. But few could have imagined it would get this bad this fast. More t= han 13 billion unwanted e-mail messages swamp the Internet per day, worldwide. = This tsunami of time-wasting junk will be a $10 billion drag on worker productiv= ity this year in the United States alone, according to San Francisco-based Ferr= is Research. In a perverse analogy to Moore’s Law of microchip processing power, the number of daily spam messages is doubling roughly every 18 month= s, according to the Radicati Group, a Palo Alto, CA, market research firm specializing in electronic messaging. Having risen from 8 percent of all e-= mail in 2000 to more than 40 percent by the end of 2002, spam has now reached a majority, according to studies from several anti-spam software companies. Conceivably, spam could soon represent 90 percent of all e-mail, says David Heckerman, who heads the Machine Learning and Applied Statistics group at Microsoft Research, which is working on anti-spam technologies. If that happens, he says, “a lot of people will just stop using e-mail.”= ;

“Spammers are g= aining control of the Internet,” says Barry Shein, president of Brookline, MA-based The World, which started in 1989 as the first commercial provider = of dial-up Internet service. Shein has been spending an increasing number of nights and weekends—the witching hours for spammers—trying to b= lock barrages of spam that appear so suddenly that they threaten to overwhelm hi= s service. He’s constantly adding new spammers to a “blacklist” used= to block all e-mail from rogue Internet addresses, but that’s a Band-Aid. “They change their network identities every couple of hours,” a= nd then sometimes launch “revenge attacks,” Shein says. And spamme= rs are ever alert to fresh prey: according to a study conducted by the Federal Trade Commission, someone who uses a brand new e-mail address in an online = chat room could get hit with spam as quickly as nine minutes later.

The problem could eas= ily grow beyond anyone’s control. “Our concern is not so much for t= he porn and the herbal Viagra as it is for the legitimate businesses,” s= ays John Mozena, cofounder of the Coalition against Unsolicited Commercial E-ma= il (CAUCE), an advocacy group.“There are 24 million small businesses in = the U.S. If just 1 percent got your e-mail address and sent you one message per year, you’d have 657 additional messages in your in-box every day. Th= at is our nuclear-winter scenario.”

To avert such a catastrophe, electronic warriors are fighting the scourge of spam using thr= ee principal tactics. The first involves the rapid adoption of spam-blocking-and-filtering software by consumers, corporate networks, and Internet service providers. Anti-spam software is expected to grow into a $= 2.4 billion industry by 2007, up from about $650 million now, according to a Radicati Group forecast. But that alone won’t win the war. The second, newer approach involves instituting more drastic changes in the way e-mail = and the Internet work, perhaps imposing new costs to send messages or developing the ability to trace e-mail messages like phone calls.

The third tactic is a= legal one, involving not only better law enforcement and prosecution of spammers = but even a ban on all unsolicited commercial e-mail. To beat back the persisten= t, rising tide of spam, it’s probably necessary to engage on all three fronts at once. “We move based on what we anticipate from the enemy, = and then the enemy reacts,” says Microsoft’s Heckerman. “We’re already up five levels of prediction.” Everyone expects further escalation—while hoping that e-mail as we know it won’t be destroyed in the process.

Spam Fighter: CipherTrust’s Paul Jud= ge, chair of the Anti Spam Research Group, concedes that defenses against spam = can lead to more being sent. (Photo courtesy of Scott Jolliff)

<= /span>

An Anti-Junk Arsen= al

As one of the most da= unting computer science problems to come along in years, the spam jam has triggered the Internet’s version of a Manhattan Project. Hundreds of software whizzes are forming teams and companies in search of the ultimate way to ha= lt mass proliferation (see Seven Ways of Sifting Spam). At the first-of-its-kin= d Spam Conference at MIT in January, the overcapacity crowd of almost 600 was spec= kled with PhDs writing scientific journal entries, young programmers wearing bea= rds and backpacks, and P.R. pros touting the latest anti-spam services and software. The scene struck some participants as rather pathetic. “The= re are some very bright people here,” The World’s Shein told the c= onferees, “and what are you spending your time doing? Blocking penis enlargement ads.”

Despite deep divisions among this assemblage on who has the best tools for eradicating spam, there’s broad consensus on one point: if there’s one thing worse than a piece of junk e-mail, it’s the prospect that a spam filter will stop a legitimate message from reaching its recipient. That’s why the= re are two important numbers one needs to know about the spam filters now in u= se or under development: the filtration percentage (the proportion of junk mail blocked) and the false-positive rate (the proportion of normal mail blocked= ). A 95 percent filtration rate is considered good, according to Paul Judge, hea= d of the Anti Spam Research Group, started in February as a new branch of the Internet Research Task Force, a professional society. Many filters claim ev= en higher filtration rates, he says, but those tend to run the risk of the unacceptable false-positive rates of .1 percent or higher—meaning that one in 1,000 normal messages would be lost.

Spam fighters are relentlessly adding new weapons to their arsenal. San Francisco-based Brightmail maintains one of the most widely used filters, which has been installed on corporate e-mail servers as well as the user networks of EarthLink, Verizon, Comcast, and Microsoft’s Hotmail. The filter processes about 10 percent of the world’s e-mail flow, says Enrique Salem, the company’s CEO. Brightmail has set up more than one million randomly generated “decoy” e-mail addresses, such as Dxodt19@anydomain.com. Since no human is attached to these accounts, no one= can possibly claim that their owners ever authorized a marketer to communicate = with them. Within days, weeks, or sometimes months, these phony addresses will b= egin receiving spam.

How can an e-mail add= ress that’s neither listed nor used start receiving spam? The answer is the “dictionary attack.” So-called spambots not only harvest e-mail addresses posted on Web sites but connect to the major Internet service providers and systematically send standard address verification requests to guessed addresses, beginning with “aaa, aab, aac,” or by trying “DrDebra25a, DrDebra25b, DrDebra25c.” Such programs are often included with spam kits sold by organized syndicates. Whenever these progra= ms fail to receive a “user unknown” type of message in reply, they= add that address to a list of valid addresses, to be sold to other spammers (see “Spreading Spam,” below).

An Internet service provider can sometimes detect such a breach and throw the attacker off the system, but the attacker will attempt to connect seconds or minutes later, = from a seemingly different Internet location. According to the Spamhaus Project,= a U.K.-based volunteer organization funded by a British Web hosting company, earlier this year both Hotmail and MSN were buffeted by such an attack at t= he rate of three to four tries per second, round the clock, for at least five months straight. (Microsoft, which runs both of the targeted services, says= it has identified the alleged perpetrators and is pursuing legal action in U.S. district court in San Jose, CA.)

Brightmail’s de= coy method is aimed at minimizing the damage of such attacks. When the in-box of Dxodt19@hotmail.com receives a message, Brightmail’s software compres= ses that message into a unique 512-bit “signature,” which is added = to the database of known spam. The database is updated constantly, and a new version of it is transmitted several times per hour to Brightmail’s m= ore than 600 corporate customers. Any message that comes reasonably close to ma= tching a known spam signature is automatically flagged as unsolicited. Eventually these pieces of presumed junk are deleted en masse. “It’s like a sting operation,” Salem says.

Brightmail excels in = its extremely low false-positive rate. It will block only about one in a million legitimate messages, for a rate of .0001 percent. The big shortcoming of th= is kind of filtering is that it doesn’t do a terribly good job of actual= ly blocking junk. A new piece of spam, or even a significant twist on an old s= pam, will probably make it through. Indeed, Brightmail’s Salem claims only= a 92 percent filtration rate—and large customers such as Microsoft and EarthLink peg the actual rate at more like 70 percent. That’s why Brightmail is only used as a rough filter—and why it doesn’t co= me close to tackling the overall problem.

Spreading Spam

A spammer harvests va= lid e-mail addresses using a “dictionary attack.”=

<= ![endif]>

Smarter Shields

Seeking a more perfect form of relief, tens of thousands of users have downloaded open-sou= rce filters (most popularly, Spam Assassin) or purchased commercialized versions such as McAfee’s SpamKiller. A collection of statistically valid rules created by humans, these “heuristic” filters stand guard at the user’s in-box and scan every incoming message for tip-off terms such = as “Viagra,” “V1AGRA,” or even “V*I*A*G*R*A,R= 21; plus improbable return addresses, strange symbols, embedded graphics, and fraudulent routing information, indicating the message is of dubious origin= s. After applying hundreds of rules, the filter scores each message, discarding those whose scores exceed a threshold value. Spam Assassin and SpamKiller typically exhibit filtration rates higher than 95 percent and false-positive rates of about .1 percent, according to Matt Sergeant of MessageLabs, a mak= er of Spam Assassin improvements.

This relatively high false-positive rate, however, is troubling to some users. After all, much legitimate e-mail has some of the same traits as spam. Sergeant concedes th= at newsletters that were requested by users will occasionally be discarded. Th= at flaw has led to novel solutions such as collaborative filters, in which use= rs vote as to which messages should be deemed spam.

SpamNet, from San Francisco-based Cloudmark, is one example of a program that deploys democra= cy in this way. An add-on to Microsoft’s Outlook e-mail program, SpamNet starts filtering spam automatically upon installation. If enough trusted us= ers designate a message as spam, that message ends up in the spam folders of Cloudmark’s entire base of 420,000 users. “When a new person jo= ins, they get the benefit of the community,” says Vipul Ved Prakash, Cloudmark’s founder and chief scientist. False positives are rarer un= der this approach, and users also have the option of clicking “unblock= 221; on any messages in their spam folders. But there are drawbacks: SpamNet dem= ands a higher level of user vigilance, and it requires that Cloudmark’s re= mote servers examine all incoming e-mail before passing it on.=

To fend off spam that penetrates other defenses, computer scientists have turned to the 18th-cent= ury probability theory of English mathematician Thomas Bayes. Published in 1763, two years after his death, Bayes’s “Essay towards Solving a Pro= blem in the Doctrine of Chances” provides a blueprint for determining the likelihood of future events. Since one person’s spam can be another person’s invitation to a pleasurable afternoon, Bayesian spam filters learn over time what each individual considers unwanted e-mail. When a user deletes several unopened messages about mortgage refinancing, for instance,= a Bayesian filter learns to discard e-mail with that kind of terminology. If = you typically do read such messages, however, the filter will take note of that= and consider it normal e-mail.

Because Bayesian filt= ers can be trained, their effectiveness improves over time, typically attaining filtration rates of 99.8 percent, along with a false-positive rate of a mere .05 percent. “If everyone’s filter has different probabilities = of different messages getting through, it makes it harder for the spammers,= 221; says Paul Graham, an independent Cambridge, MA, programmer. Last August, a = link to Graham’s article “A Plan for Spam” on slashdot.org jump-started a rush to Bayesian filtering. These kinds of filters, Graham s= ays, will break the business model of the spammer. It costs about $200, he continues, to send one million messages—an endeavor that typically yi= elds about 100 responses. If those 100 people spend an average of $2 each, the spammer breaks even. The goal, Graham says, is to drive response rates down= to around one in a million so that “it would no longer be economical for= a spammer to consider such a business proposition.”

Microsoft Research has taken this probabilistic approach even further. Standard, so-called naï= ;ve Bayesian filters treat each word or feature in an e-mail independently, but Microsoft claims its new filter, which is offered as an option in MSN 8 software, learns probabilities for words, phrases, and other distinguishing characteristics that commonly appear together. It might flag messages containing the phrase “make money from home” and “click here” that are sent from servers based in Hong Kong and that have ran= dom characters in the subject line. Microsoft’s Heckerman claims that, by correlating patterns, his filter exhibits an even lower rate of false positives.

The monkey wrench is = that spam is not an inanimate adversary, but rather a tool of wily and willful h= umans. In fact, the very effectiveness of spam filters may actually be making the problem worse. If half of a batch of spam gets thrown into the digital garb= age can, the spammer will tend to respond by sending twice as much spam the next time. “As you put more filters in place, spammers become more determi= ned, and the spam will increase,” says the Anti Spam Research Group’s Judge, who is the chief technology officer at CipherTrust, an Alpharetta, GA-based provider of e-mail security systems.

To balance the higher volume, Judge says, spammers simply find ways to lower their costs, such as enlisting servers based in China or India, where labor is cheap. What’= ;s more, as spammers mount a counterattack against Bayesian methods, spam is tending to look more and more like non-spam. For example, a message that sa= ys, “Hi Jim, have you seen the party pictures—take a look!” m= ay not raise red flags, because it doesn’t contain any obvious spam term= s. When spam begins to look exactly like messages from friends and colleagues, filters may fail.

<= ![endif]>

Coping with masses of junk e-mail poses a daunting technical challenge for computer scientists, who exchanged war sto= ries at MIT in January at the first ever Spam Conference. (Photo courtesy of Ger= ald Oskoboiny)

Crippling the Atta= ckers

That’s why anti= -spam researchers are cooking up more-systematic treatments. Referring to spam as= a “plague,” Mark Petrovic, vice president of R&D at Internet service provider EarthLink, notes that today’s e-mail system was desi= gned 20 years ago for small numbers of people who already knew one another. “The possibility of sending body part enlargement ads was unheard of,” he says. Stemming the tide of spam, he says, will “require= a cooperative solution to augment the basic way e-mail works.”

The most widespread of these measures is a blacklist of the sort used by Shein and other Internet service providers. Also maintained by startups such as SpamCop and NetBlock= s, and by nonprofits such as CAUCE and Spamhaus, blacklists are collections of Internet Protocol addresses, domain names, and server farms that have been implicated in spewing spam; any mail originating from these tainted places = will be blocked. But blacklists are imprecise: they often fail to keep pace with spammers, who constantly falsify their network locations, while sometimes b= locking legitimate users. Indeed, blacklists sometimes halt e-mail from entire countries with high spam rates. E-mail originating in China and South Korea= , in particular, has periodically been blocked from much of the Internet. <= /o:p>

The inverse of the blacklist is the white list—a preauthorized address book maintained by users. An option in AOL 8.0, for instance, causes any message from senders = not on the high-priority list to be discarded. This method also tends to trash e-mail you might want, though, and requires a high degree of maintenance; e= very time you make a new contact, you have to add a name to the white list. Aside from these drawbacks for their users, blacklists and white lists also are “wreaking havoc” on legitimate mass e-mailers, says Paul Soltof= f, CEO of SendTec, a direct-marketing firm. After all, many companies (Tech= nology Review among them) send out electronic newsletters and other promotional materials. These aren’t as obnoxious as the come-ons that most of us consider spam, and yet they are just as vulnerable to being blocked through= the widespread use of blacklists and white lists.

Another drastic anti-= spam measure strikes at the heart of the Internet’s culture: imposing new costs on sending e-mail. “Paying to send e-mail may be anathema to al= most everybody,” says Robert Hettinga of Internet Bearer Underwriting, a startup in Boston. “But eventually, bits of money will be attached to e-mail messages.” Just as paper mail requires postage, e-mail would require e-stamps. A charge of one-tenth of a cent per e-mail, for instance, would hardly be noticeable to ordinary users but would levy a $1,000 tax on someone sending a million messages at once. Any piece of e-mail sent withou= t an e-stamp would be automatically blocked. Others favor imposing a cost not in dollars but in the sender’s computer time. Your PC would have to solv= e a quick mathematical problem for each message it transmits, barely affecting senders of normal quantities of e-mail but crippling a spammer’s microprocessor. Such a “computational cost” approach is being developed at Microsoft Research and in an open-source effort called Camram = (see “Making Spam Expensive,” TR April 2003).

The World’s She= in proposes an Internet market trade association, which would be an “e-mail clearinghouse,” run by a group of e-mail providers. Such an organizat= ion would sell legitimate bulk mailers special license codes in return for royalties based on the size of the mailings they are sending. Spammers who = buck the system would be tracked down and sued by clearinghouse lawyers using fu= nds set aside from the royalty pool. “The goal is to monetize the process= ing of bulk e-mail,” Shein says. He derives the idea from the long-establ= ished model by which radio stations and performers pay royalties to songwriters b= ased on the formulas of another clearinghouse: the American Society of Composers, Authors, and Publishers. Elements of such a plan are already being adopted = by the big three of e-mail providers—Microsoft, Yahoo!, and AOL—who announced in April that they are banding together to develop a way of creat= ing a white list for legitimate marketers. The group has yet to announce whether participating marketers will pay to maintain a new infrastructure, but Shein guesses that things are heading that way.

For such a plan to wo= rk, future e-mail will have to be traceable. The telephone system has survived,= in part, because there have always been ways to track phone calls back to thei= r sources and find those who abuse the network. “Filtering e-mail without being able to establish identity is essentially futile,” says EarthLinkR= 17;s Petrovic. He cites the problem of spam masquerading as real e-mail. “= If my wife says, ‘I’d like to spend some time with you this evening,’ I will react differently than if a stranger says the same thing. I need to know who is talking to me before I can evaluate the meanin= g of the message.” Indeed, Petrovic adds, the anonymity of e-mail is centr= al to the spam phenomenon. If we cannot determine who is sending messages, all other spam-blocking measures will ultimately fail.

Establishing such traceability would require fundamental changes to the basic protocol that governs all e-mail transmission. Called the Simple Mail Transport Protocol,= or SMTP, it is the 20-year-old language that virtually all e-mail software spe= aks in order to move messages around the Internet. If all network providers swi= tch to an “authenticated SMTP,” as EarthLink’s Petrovic calls= it, only an e-mail with a verified return address and from a valid domain name would be able to get to its desired recipient.

The Legal Front

Technology alone will= never win the war. Ninety percent of spam is sent by fewer than 200 people, accor= ding to Mozena of CAUCE, the anti-spam coalition. That represents an astounding degree of concentration, but virtually everyone who fights spam for a living agrees it is roughly correct. The implication is clear: spam is a crime-fighting problem akin to the prosecution of the small number of malic= ious hackers who crack into networks. “These are human beings generating t= hese messages,” Mozena says. “It’s not as if the Internet is broken. You can’t address social problems solely with technical means= .” He believes that the spam plague is a criminal-justice dilemma that can be eradicated only with the active participation of legislatures and courts.

New laws, though, hav= e yet to make much of a dent. Last year, the European Parliament passed a directi= ve suggesting that member countries require marketers to ask permission from u= sers before sending pitches through e-mail. So far, Austria, Denmark, Finland, Germany, Greece, Italy, and Norway have enacted such “opt-in” anti-spam legislation. But since so much spam is sent from the United States through Asia-based servers, these laws have had little effect. In 2000, the U.S. House of Representatives voted 427 to 1 to pass an anti-spam bill. But instead of including a strict opt-in provision, the bill required consumers= to request the removal of their addresses from each marketer’s e-mail li= st. After privacy advocates denounced this “opt-out” bill as useles= s, it died without reaching the Senate. At least two spam bills are now alive = in Congress, but there is still no consensus among lawmakers on whether the government can effectively outlaw spam—or even that it should.

In April, the Federal= Trade Commission held a conference to help decide how best to approach this crisi= s. Brian Huseman, an FTC staff attorney, says the commission has prosecuted sp= ammers who have sold bogus wares, failed to live up to their claims, impersonated legitimate organizations, or engaged in other deceptive practices. But since the agency is mainly charged with prosecuting fraud cases, it is powerless against spam that sells legitimate products. “There is no federal law that prohibits unsolicited commercial e-mail,” Huseman says.

Until such a law is p= assed, lawyers will continue to rely on precedents from similar cases, says Jon Pr= aed of the Internet Law Group. He believes that indiscriminate mass e-mailing is “already illegal in all 50 states” based on centuries-old Common Law that prohibits unauthorized use of someone else’s property—= in this case, computer networks.

Armed with this argum= ent, AOL pursued porn spammer Jay Nelson, both before and after he and his cohor= ts violated the 1999 court order. Since spam cases can be prosecuted anywhere damage occurs, AOL chose its hometown district court in Alexandria, VA. In October 2002, the judge held the coconspirators in contempt and awarded AOL $6.9 million in damages and fees on top of the original $1.9 million findin= g, according to court documents. That figure was topped in May when EarthLink = won a $16.4 million judgment against Howard Carmack, a Buffalo, NY, spammer; a = week later, he was arrested on charges of identity theft. Praed says spammers ca= nnot skirt the payments by filing bankruptcy, and that the plaintiff can “hound” the guilty parties until the money is collected, preven= ting them from buying houses and cars. “We need to make the spammers reali= ze they made a mistake and to discourage others from doing it,” he says.=

Detroit-based spammer= Alan Ralsky, however, remains active. Instead of spending more time and money bringing Ralsky to court, Verizon last October decided to settle its case against the man that some call “the spam king.” In return for Ralsky’s paying an undisclosed sum and promising to avoid VerizonR= 17;s network, the lawsuit was dropped—leaving Ralsky firmly in business.

Furious anti-spam act= ivists posted Ralsky’s home and e-mail addresses online, and soon he was del= uged with piles of printed catalogues and junk mail. Yet he appears undeterred a= nd continues to add to his list of 250 million e-mail addresses. According to = his own statements, he is finding new ways to obscure his identity, laundering = his Internet location data through servers in Romania and obscure parts of Chin= a. Spamhaus and CAUCE consider the 57-year-old Ralsky one of the top five spam= mers worldwide. “I’ll never quit,” he told the Detroit Free Press. “I like what I do. This is the greatest business in the world.”

The war on spam won&#= 8217;t be won until guys like him are somehow forced to change their minds. <= /o:p>