NOTE: Please please please feel free to leave comments criticizing anything I write here or if you have any questions! If it can help me learn to do something better I’m all for it.

So, as much as I love using PHP and CURL, I’ve finally graduated to none other than iMacros because a majority of my bigger projects have started to require me to read pages which contain necessary javascript to view. Reverse engineering javascript is fun, when it works, but wasting valuable time reversing it doesn’t make sense.

If you haven’t heard of iMacros it’s a super powerful macro scripting program made specifically to automate web browser sessions. Now before you start thinking to yourself macros are for newbs who don’t know how to code, trust me this is far from newb. iMacros on its own is very newb, mixing iMacros + PHP/MySQL + other scripting = insanely complex.

My setup is this:

Fedora Core 11 w/ VirtualBox installed (a free VMware type application), and a WinXP Pro Virtual Machine w/ iMacros scripting edition installed along with the iMacros Firefox Plug-in.

The reasoning behind running Linux and a virtual machine is this, if a WinXP virtual machine starts leaking memory/freezes/dies, I can simply close the virtual machine and reopen it via Linux command line. I’m an avid fan of making sure stuff can run unattended for months. Although it makes things a bit more complex, it’s well worth the time investment in doing it right.

With that said, I plan to slowly add my entire setup to this post, along with some more complex coding that will allow you to multi-thread iMacros sessions.

iMacros + Firefox Plugin
The reasoning behind using the iMacros Firefox Plug-in versus the iMacros Browser is that the current iMacros browser doesn’t support threaded sessions. What this means is that the same cookies, cache, proxy settings will be used across every iMacros browser session. This is no good if you want to log in to 2 different accounts at once using 2 different proxy IPs. Firefox however allows you to create different browser profiles (just run ‘firefox -ProfileManager’ with firefox already closed to view the profile manager), and iMacros allows you to choose different profiles during Firefox’s execution. As a result you get browser sessions that are entirely different from one another.

Setting up WAMPServer.

1) Download and install WAMP Server at http://www.wampserver.com/

2) Setup the command PHP to execute the WAMP server version of PHP. You won’t need Apache, I just like WampServers version of the php config which already has most of the plugins I need installed which saves a lot of headache. To make the PHP command work on Windows Vista click the start button, right click Computer and go to Properties. Click advanced system settings, go to the Advanced tab, click environment variables at the bottom, under System Variables go to the variable PATH and add the PATH to the folder that contains the php.exe (my path is: ‘C:\wamp\bin\php\php5.3.0′;)

Setting up Firefox –

First off you’re going to need to download the Firefox version of iMacros here.

To make Firefox and iMacros run most efficiently I highly recommend you do several things.

1) Disable Flash – iMacros seems to have page loading issues when Flash is involved. Since you can’t play with flash in the iMacros Firefox Plug-in anyway, lets disable flash by going to Tools/Add-ons clicking the “Plugins” tab and going down to ‘Shockwave Flash’ and disabling it.

2) Install AdBlock Plus - why spend bandwidth and CPU processes running ads you’ll never see as well as get advertisers charged for those ads. Secondly, AdBlock Plus will allow you to block certain sites or scripts from loading that you specify. This again saves valuable resources and sometimes is necessary since iMacros still appears a bit buggy when trying to figure out whether a page is done loading or not.

3) Disable loading images automatically. 99% of the time there is no point to loading images either unless your system is some sort of image scraper. Although iMacros has an image removal function (‘FILTER TYPE=IMAGES STATUS=(ON|OFF)’) I prefer disabling images via Firefox instead. I highly recommend disabling images in Firefox by going to Tools/Options clicking the ‘Content’ tab and removing the checkmark on ‘Load images automatically’. If you have images that might be an exception, say a captcha image, you can simply click the ‘Exceptions’ button and add the url of the exception. I’ve also included an iMacro in the package above that will allow you to disable and re-enable images in Firefox.

4) Make every browser session you start a private browsing session. You can do this by going to: Tools/Options/Privacy in Firefox and then choose in the drop down “Firefox will never remember history”. The reasoning for this is that I’ve found the iMacros “CLEAR” command is unreliable.

The scripts/PHP code I’ve written:

iMacros Firefox Proxy  Setting Script (taken from: http://thepemberton.com/posts/archives/23 and modified a little bit)

To use it, simply replace {{IP}} and {{PORT}} with the IP and port of your proxy.

iMacros AdBlock Plus enable/disable script

To use it, simply install ABP and then modify {{status}} to ‘true’ or ‘false’ to enable or disable it.

iMacros Firefox images enable/disable script

To use it simply replace {{images}} to 1 or 2 to enable or disable images in Firefox.

iimfx.class.php

This file allows you to run iMacros from PHP with ease providing your running the iimRunner.exe. I haven’t documented it very well, but if you take a look through it you should be able to easily figure it out. Any updates, changes you see fit, please let me know, I’d love to improve on it. You’ll also need the 3 previous scripts above to use it appropriately as it’s customized for my setup.

The code of iimfx.class.php

<?php
 
class imacros {
	function __construct($proxyip = '', $proxyport = '', $silent = false, $noexit = false) {
 
		echo "--------------------------------------\nNew imacros session started!\nUsing Proxy: $proxyip:$proxyport\n";
		$this->proxyip = $proxyip;
		$this->proxyport = $proxyport;
 
		if (empty ( $this->proxyip ))
			echo "NO PROXY!!\n";
 
		$this->noexit = $noexit;
		$this->fso = new COM ( 'Scripting.FileSystemObject' );
		$this->fso = NULL;
 
		$this->iim = new COM ( "imacros" );
 
		$toexec = "-runner -fx -fxProfile default";
 
		if ($silent === true)
			$toexec .= " -silent";
 
		if ($noexit === true)
			$toexec .= " -noexit";
 
		echo $toexec . "\n";
 
		$this->iim->iimInit ( $toexec );
 
		if (! empty ( $this->proxyip )) {
			$dvars ['IP'] = $this->proxyip;
			$dvars ['port'] = $this->proxyport;
			$this->play ( $dvars, 'proxy' );
		}
	}
 
	function __destruct() {
		if ($this->noexit === false)
			$this->iim->iimExit ();
	}
 
	function play($immvars = '', $macro) {
 
		echo "--------------------------------------------------------\n";
 
		if (is_array ( $immvars )) {
			foreach ( $immvars as $key => $value ) {
				echo "Setting Value $key => $value\n";
				$this->iim->iimSet ( "-var_" . $key, $value );
			}
		}
 
		echo "Playing Macro $macro\n";
		$s = $this->iim->iimPlay ( $macro );
 
		if($s>0){
			echo "Macro successfully played!\n";
		}else{
			echo "--------MACRO ERROR!-------------------\n ERROR: " . $this->getLastError() . "\n";
		}
		return $s;
	}
 
	// This function retrieves extracts in your iMacros script if you have any. 
	function getLastExtract($num) {
		return $this->iim->iimGetLastExtract ( $num );
	}
 
	// Returns the last error :)
	function getLastError(){
		return $this->iim->iimGetLastError();
	}
 
	// Enables/disables images
	function setImages($images = 1) { // 1 = on 2 = off
 
		$dvars ['images'] = $images;
		$this->play ( $dvars, 'images' );
 
	}
 
	// Enables or disables adblockplus
	function enableABP($status = true){
 
		$dvars['status'] = $status;
		$this->play ( $dvars, 'abp.iim' );
 
	}
 
}
 
?>

• Previous Entry: Scraping Forms with DOM and PHP for Posting with CURL
• Next Entry: Toorcon 11 in San Diego (A review)