How to create a wefightcensorship mirror site

The wefightcensorship website has been  designed to be easily duplicated by miror sites. You can host a copy of wefightcensorship on your webserver and be involved in the fight against censorship in the world. The more mirror websites, the more difficult it will be for censors to block WeFightCensorhip. The procedure outlined in this chapter explains how to create a secondary mirror of the wefightcensorship.org website on a machine located anywhere on the Internet. This script relies on OpenSSH and rsync. Alternatively, you can also install a website copying software on your server (such as httrack) and run it at regular intervals (using cron for example) in order to have an updated version of our website. Even simplier, you can use the autoblog script developped by SebSauvage.

Requirements

The following are required:

  1. A machine running Unix operating as a Web server. There is no restriction on the Unix type and distribution or on the Web server. It is up to the person who creates the mirror to configure it.
  2. The machine (in addition to the Web server) must have the following tools: rsync and OpenSSH

If you do not have all of the above, you should not begin the procedure.

Operating procedure

Technical requirements

The machine acting as a Web server must be installed and configured before the procedure is launched. This document does not cover the installation and configuration of software (virtual host, for example), or set up local users. This is specific to each server. There are few technical restrictions. A web server that can host static content is needed (for example there is no use of PHP language or a MySQL database). A user account must be available which allows the creation of files in the virtualhost hierarchy. This account will be used to operate the server synchronisation script and should be able to run the rsync synchronization command. In the following examples, we are using an account called www-data.

How synchronization works

Secondary mirrors are static web servers whose purpose is to make available to users the content of the website wefightcensorship.org. The hierarchy to be published is retrieved from a primary mirror site using rsync synchronization. Secondary mirrors must be authenticated on the primary mirrors using an OpenSSH key. The key serves both as an authentication of the secondary mirror and means of encrypting the data flow.

How to create an authentication key

If the user www-data does not have an OpenSSH key pair, here are the commands to create them:

www-data@mirror:~$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/var/www/.ssh/id_rsa):

Accept the default path, or choose a place where you can store the key. If you change the default path (for www-data it would be “homedir”), you must make sure that OpenSSH can access the keys. If necessary use an SSH agent for this.

Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /var/www/.ssh/id_rsa.
Your public key has been saved in /var/www/.ssh/id_rsa.pub.
The key fingerprint is:
79:01:ff:20:c5:65:0d:ff:0c:76:9c:db:34:a0:da:3f www-data@mirror
The key's randomart image is:
+--[ RSA 2048]----+
|        ....++   |
|         +....o..|
|        . +.  +o+|
|         oo+ . *+|
|        S....  .+|
|         .  .    |
|             E   |
|              .  |
|                 |
+-----------------+

Registration of the mirror with WeFightCensorShip.org

Access to the WeFightCensorShip.org synchronization servers must be requested and the public key (but not the private key) must be sent when your request is made. To do this you should send an e-mail to wefightcensorship@rsf.org with “creation of a mirror site” in the subject line. You will receive by return a synchronization script and the associated configuration files.

Creation of Web server hierarchy

The creation of the server hierarchy and the associated virtualhost are the responsibity of the owner of the mirror site. These vary according to the distribution, Web server and the operating procedures that are in effect. For the next phase we are assuming that the hierarchy is under /var/www/ mirror.wefightcensorship.org-443/htdocs. Adaptations can be made by the person using the mirror site.

Implementation of synchronization script

After the mirror site has been registered with WeFightCensorship.org, three files will be sent to you. First you will receive the synchronization script, followed by the associated configuration files. In accordance the usual practice, these will be place respectively in /opt/cybershelter/bin (synchronization script) and /opt/cybershelter/etc (configuration files). You should make sure that the script is permissioned for the user www-data. .

It is recommended that the first time the script is run it should be done manually. This will allow any configuration problems to be identified. Subsequently the script should be automated using a task in the crontab scheduler of the user www-data. For example the following crontab can be used for an hourly synchronization.

 www-data@mirror:~$ crontab -l50 * * * * /opt/cybershelter/bin/synchro-miroir-secondaire.sh

Configuration of the synchronization script

The synchronization script has two configuration files that allow the rsync parameters to be controlled. These are:

www-data@mirror:~$ cat /opt/cybershelter/etc/synchro-miroir-secondaire
MIROIR_PRIMAIRE=adresse.du-miroir.fournie
RSYNC_USER=synchro
REMOTE_PATH=/srv/synchro-miroir-secondaire

This file contains parameters set by WeFightCensorShip.org. They should not be changed, since this could stop the synchronization script from working. The first parameter is a list of primary mirror sites with which you can synchronize yours. This list is provided and cannot be changed. It may be updated automatically in the course of synchronization. The second parameter is the user account to be used for synchronization. This value is provided and cannot be changed. It may be updated automatically in the course of synchronization. The third parameter is the path on the server that contains the files to be synchronized. This value is provided and cannot be changed. It may be updated automatically in the course of synchronization.

www-data@mirror:~$ cat /opt/cybershelter/etc/synchro-miroir-secondaire.local
LOCAL_PATH=/var/www/$(hostname).wefightcensorship.org-443

This file contains the declaration of local variables that you can modify. The first parameter is the path to the local hierarchy of the site (the contents of vhost). The value should be adjusted to take account of the contents of the virtual host definition.

New versions of synchronization script and configuration file are sent during miror supdate. Files, and associated MD5 checksums, are available in the ${LOCAL_PATH}/synchro directory.

Miror owner can either update manually script and configuration file, or use symbolic links to use recevied files.

If you choose to use symbolic links, /opt/cybershelter/etc/synchro-miroir-secondaire should link to ${LOCAL_PATH}/synchro/synchro-miroir-secondaire, and /opt/cybershelter/bin/synchro-miroir-secondaire.sh should link to ${LOCAL_PATH}/synchro/synchro-miroir-secondaire.sh.

NB: if you re-enter an existing variable in the file /opt/cybershelter/etc/synchro-miroir- secondaire, its local value will be taken into account. To do so means, however, that no account will be taken of updates and this may change the way the synchronization script works.