Documentation

About SiteCloner

SiteCloner is a PHP script which allows users to make static clones of pretty much any web site. It does not matter wether the original web site is made out of static HTML files or is built using a CMS like Wordpress or Joomla. The resulting clone will a collection of static HTML files including all used assets such as Javascript files, CSS file, images and font files. Generated clones can be browsed online, downloaded as a ZIP archive, send by email or uploaded via FTP.

SiteCloner is built using CodeIgniter 3, a popular and well-documented PHP MVC framework. For more information about CodeIgniter, please visit the official CodeIgniter homepage.

1 Installation

The SiteCloner script comes with a handy installation script. After uploading the script to your hosting account, navigate your browser to the /install folder (for example, if you have uploaded the script to the domain www.mydomain.com, you would need to open http://www.mydomain.com/install/ in your browser).

SiteCloner screenshot 1

In this screen you will need to enter the details of the MySQL database into which you want to install the script (please note you will need to create this empty database before installing the script, we advise using phpMyAdmin for this).

Once the installation process has completed, you will see a confirmation on your screen and you can continue to the login screen. Please use the following details to login:

  • username: [email protected]
  • password: password

After logging in, you can change the login details to whatever you'd like.

2 Configuration

SiteCloner does not need much configuration to get running. The main configuration items you might want to consider are 1 emails, 2 pretty URls and 3 folder permissions.

Emails
To be able to successfully send emails from the script, you might want to configure /application/config/email.php. This file contains all the possible CodeIgniter email configuration variables. To learn more about these configuration variables, we suggest having a look at the official CodeIgniter documentation for the email library (found here). For the best email delivery, we suggest configuring email.php to send email via SMTP. In addition, you should configure the "from" email address and name in /application/config/crawler.php; you want the email address here to match the domain from which you're sending email to prevent emails from ending up in the spam box.

Pretty URls
By default, all URLs (within the script, not within clones sites) will have the "index.php" file in them. Alternatively, you can configure CodeIgniter to use co-called "pretty URLs" instead, without the "index.php". To do this, please follow the instructions given by the official CodeIgniter documentation.

Folder permissions
Generated clones are stored in /files folder, so you will want to make sure the server can write to this particular folder (typically 755 permissions are sufficient; if you're experiencing trouble then try 777 instead). Zipped clones are stored in the /tmp folder, so the server should have write permissions to this folder as well (same thing, typically 755 will be ok, if not then try 777).

3 Managing Sites

The first step towards creating clones, is adding one or more sites under the Sites section.

SiteCloner screenshot 2

To add a new site, simply click the green button labeled "Add New Site" and enter the URL of the site you want to add.

4 Site Settings: exclude keywords

There might be situations where you want certain parts of a site being excluded from a clone. To this end, SiteCloner allows you to specify one or more exclude keywords. URLs containing one or more of these keywords are ignored by the crawler and won't be included in the clone.

To configure exclude keywords, click the little cog icon for the site for which you want to configure exclude keywords, as shown in the image below.

SiteCloner screenshot 3

Once clicked, a modal popup will appear which will allow you to enter the exclude keywords in the first tab. Please note you must save the settings by clicking the green button labeled "Save Settings" before closing the modal window.

SiteCloner screenshot 4

5 Site Settings: time out limit

SiteCloner has a built in timer, which keeps track of how long a cloning process has been running. Once the process exceeds the limit, the cloning process will halt. Once halted, you can always manually continue the process at any given time.

This time out limit is set for each site individually, and the default is 1800 seconds (half an hour). To change the time out limit for a site, once again click the little cog icon which will open the modal settings window. Inside the second tab, you can enter a time out value, in seconds, for the current site.

SiteCloner screenshot 5

6 Creating Clones

A new clone can be created from two separate locations. The first one is from the Sites page; by clicking the clone icon for the site you want to create a clone of:

SiteCloner screenshot 6

Please note that when creating a clone straight from the Sites page, you won't be able to specify any actions to be taken on completion or timeout.

The second method to create a new clone is from the Clones page. Using the dropdown on the right side of the page, you can select which site you want to clone, after which you can click the button labeled "Build A New Clone for". This method allows you to specify what SiteCloner should do when the clone is complete or timed out. More about this in the next section.

7 Events

When building a new clone from the Clones page, you can specify what SiteCloner should do in case the clone is completed or has timed out. When completed, you can either send the clone via email or upload it to a remote server via FTP. When timed out, you have the option of sending out an email to notify you of the cloning process having timed out.

To be able to send out emails properly, you might need to configure /application/config/email.php and we strongly advice to send emails via SMTP. When choosing to upload via FTP when a clone is complete, the script will only be able to upload if the supplied FTP details are correct. When incorrect, the upload will fail silently.

8 Downloading Clones

Once a clone is completed, you can download it straight from the Clones page by clicking the little download icon as shown below:

SiteCloner screenshot 7

9 Uploading Clones Via FTP

SiteCloner also allows users to upload a cloned site to a remote server over FTP. To initiate the upload, click the upload icon (as shown in the image below) and provide your FTP details in the modal popup. When uploading over FTP, depending on the site of the clone, this can take up quite some time. While uploading, be sure not to close the browser window or navigate to another page.

SiteCloner screenshot 8

10 Sending Clones Via Email

Clones can also be send as email attachment. To this end, click the little email icon and provide your email information. Please note that for emails to get sent correctly, you might need to configure application/config/email.php. For more information, please see section 2.

SiteCloner screenshot 9

11 Rebuilding Clones

A clones can be rebuild from scratch at any given moment by clicking the rebuild icon as shown in the image below:

SiteCloner screenshot 10

Rebuilding a clone will result in the original clone being destroyed.

12 Continuing Clones Process

In case the cloning process was manually cancelled or timed out, this particular clone can be finished at any given moment by clicking the continue icon as shown below:

SiteCloner screenshot 11

13 Editing Account Info

Next to settings for individual pages, you can also configure certain settings for an entire site. To configure these settings, locate the grey button towards the right edge of your screen labeled "Settings" and click it to display a dropdown menu:

SiteBuilder picture

Account information, such as username and password, can be edited by clicking the "My Account" link as shown below:

SiteCloner screenshot 12