Ampersands, URL parameters and wget

So there’s this queueing system we wrote for the AJBL, which racks up processor intensive tasks like image crunching to do them in the background. We add commands to a database table as URLs that get called with wget from the app in the background, and it all works remarkably well.

Except this time the command had ampersands in it. It worked every time when you called it directly, but from wget it failed to pick up the second and subsequent params. Just getting to this point took longer than I cared to think about, and because all this happens indirectly in the background, it was a nightmare trying to work out what it thought it was doing. I ended up writing a logger just to be able to find out what it was thinking when executing things in the background like this.

There went another good few hours of my life I won’t see again – this seems to be a common refrain nowadays! Once I found out what was happening, it wasn’t much longer before I tracked down the cause, and of course [the marvellous Stackoverflow came to my rescue again](

Because the wget command is being executed in the shell on the box in question, the ampersands in the URL are being interpreted by the shell instead of being passed to wget. Simply wrapping the URL in quotes fixes it.

$cmd = ‘wget "’ . $cmdURL . ‘"’;

I don’t actually mind hitting problems like this; the satisfaction when you solve them is great. But it makes estimating projects even more of a joke than it already is, and the loss of momentum is horrifying.