On Link Shortening Services OR Spammers Are Winning

 

I am mostly finished with my new website shortenthatreallylongurlintosomethingsimpleandeasytoshare.info, a satirical (but working) URL shortening service. I was looking into how to design the site, and found something interesting.

The spammers are winning:

Huge Note: Lots, at least 30 of the services I checked out were down/gone/something else. The ones listed are ones that had an actual announcement about spammers. Tons of services have started putting in Turing (captcha) tests. Many have started checking their links against those in http://www.surbl.org/ or http://www.spamhaus.org/dbl/ or http://code.google.com/apis/safebrowsing/ or http://www.malware.com.br/ or other similar services, presumably some cron jobs that scans entries regularly. They have started preventing search engine robots from following, which helps prevent people from gaining link reputation illicitly.

Many sites have been acquired by other sites. (http://makeashorterlink.com/)

Articles about spammers and/or URL shortening services worth a read:

My favorite Services for whatever reason

So what Security Measures/Features should a good URL shortening service use?

  • Be Transparent – http://ur1.ca/ Lets their database be known publicly and forfeits its own rights, and this is what the other services should do too.
  • Provide API’s for both shortening and longer-ing. Let sites like http://untiny.me/ unshorten your site. I have programmed a link unshorten-er before, and trust me. API’s are easier than trying to figure out the redirect. (also: http://www.rexswain.com/httpview.html)
  • Provide a preview option. http://tinyurl.com/preview.php has a cookie services, and a few other services let the subdomain (http://peek.snipurl.com/) how a preview, not just redirect.
  • Rate Limiting – A Spam prevention technique.
  • Turing Test. If you are serious about getting rid of spam, force users to enter captcha to shorten a link. Many, many, sites such as http://ow.ly/ do this.
  • Registration. If users register once (with captcha), and are remembered thru cookies, the hassle is not too great. If users links are detected as SPAM, its easy to delete ALL of the links that they have submitted
  • Spam Site Checking. Scan the database for known spam links. (four services listed above). Delete them. If you have a user, or IP, you can block and delete similar, probably spam, links to for terms-of-use abuse.
  • Prevent Double Dipping – Do not let already short URL’s from shortened sites be shortened. This is a way the spammers are getting around malware link detection. They use an untrusted redirect site (the many we saw being taken down) that is then put in a popular shortening service that people, because they are idiots, trust.
  • Use a 301 Redirect, not a 302.
  • Allow custom (vanity) URL’s. Not security, just a nice touch.
  • If you have users/registration, let the users delete their URL’s.
  • Don’t use a “frame bar”. See http://searchengineland.com/the-growth-of-framebars-kevin-rose-on-the-diggbar-17416
  • List of links? http://crum.pl/multi has a place where you put a list of links into a site. I cant say I would use this, and can’t recommend, but it’s a useful feature to be available to the user. For me, this would be using my http://pinboard.in/u:squirrel tag, or date, collection, and linking to that.
Post a comment | Trackback URI