Dealing with a large amount of redirects
Oct 2013A while ago I was asked to create thousands of redirects for a customer. They had decided to change their domains and the URL structure of most of their pages, so dynamic or smart URL redirecting wouldn’t work.
Adding thousands of Redirect or RewriteRule in the Apache config (or even worse, .htaccess) always seemed like a lousy way of doing it. So I did some research to find out if there’s a better way of doing it, and surely enough there is. And it’s called RewriteMap, which lets you point to a file that contains the forwarding / redirecting URLs.
The beauty of this is not just that you can have a separate file for all your URLs, but you can also take your text file of URLs and created a DBM Hash File and then point the RewriteMap to this file. Why would you do this? Quite simple, the DBM Hash File method works much quicker, especially with long lists of URLs, and this is because unlike the text files, DBM is indexed, and this allows for more rapid access to the desired key.
Creating a DBM file is very straightforward:
httxt2dbm -f db -i redirects.txt -o redirects.db
And then adding the RewriteMap for example in your VirtualHost block:
RewriteEngine On
RewriteMap redirects dbm=db:/var/www/redirects/redirects.db
RewriteCond ${redirects:$1} !=""
RewriteRule ^(.*)$ ${redirects:$1} [redirect=permanent,last]
Simple, elegant and fast!