Route53 Failover Mechanism!

Situation: you run your own website on an Amazon EC2 instance. Something happens: maybe the box runs out of resources (ddos, high traffic, you pick one!).
All of the sudden, your website is offline. Downtime means that Google will push it down in the ranking. So what do we do?.

Well, AWS Route53 has a new, and super cool mechanism, that allows you to set Health Checks. If the website doesn’t pass it, the DNS record will switch to a failover entry.

How can we achieve this?. Amazon itself posted a detailed guide here. Just for the record, here you will find details about when a website is considered healthy.

A couple details:

  1. I’ve used SiteSucker, a free OSX tool, that allows you to create a simple HTML backup. Running a static S3 backup is way cheaper than running two instances!.
  2. If your apache logfile grows **a lot** due to the HealthCheck hits, you can disable the logs, if the user agent is “Route 53”. Simply put the following in your .htaccess:

    SetEnvIfNoCase User-Agent .*Route 53.* dontlog

    Don’t forget about tweaking your website.com.conf apache file, to look like this:

    CustomLog logs/access_log common env=!dontlog

This is just… sooooo cool!

OSX: Fixing SSH hangs!

This glitch is pretty annoying. You’re following a log, while connected to a server (through ssh), and after a while, your ssh connection hangs.
The solution?.

Fire up Terminal, and type the following: nano ~/.ssh/config

Once there, fill up the following:

ServerAliveInterval 300
ServerAliveCountMax 120

That should keep your connection alive for the next 10 hours, without further issues.

Waiting until two async blocks are executed

The following snippet of code.. which is super interesting, is based on this post. This allows to dispatch a block, on Main Thread, once two async operations are completed.

dispatch_group_t group = dispatch_group_create();

dispatch_group_async(group, dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0), ^ {
  NSLog(@"Block1");
  [NSThread sleepForTimeInterval:5.0];
  NSLog(@"Block1 End");
});

dispatch_group_async(group, dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0), ^ {
  NSLog(@"Block2");
  [NSThread sleepForTimeInterval:8.0];
  NSLog(@"Block2 End");
});

dispatch_group_notify(group, dispatch_get_main_queue(), ^ {
  NSLog(@"Block3 :: %d", [NSThread isMainThread]);
});

dispatch_release(group);

Fixing High I/O usage on Amazon EBS

This humble wordpress blog is running on an AWS micro instance. We’ve got somewhere around 1k visitors each month, which is pretty awesome. But… to my surprise, the whole system is using over 14 million I/O operations.

I suspected there was something wrong with this… so i proceeded to do a small research.
By means of the application ‘iotop’, i managed to spot the I/O hog: apache!.

Specifically, i ran iotop with the following parameters:

sudo iotop -a -P

I ran a quick search on google, and found this post.  (Thank you George, for sharing your solution!).

Long short story, Apache’s APC plugin was using a memory mapped file, and it was writing… almost all the time.
The solution?. Edit your /etc/php.d/apc.ini file, and make sure that the mmap_file_mask parameter is se to use Shared Memory, as follows:

apc.mmap_file_mask=/apc.shm.XXXXXX

That should fix it!

Installing a Webserver on AWS EC2

I’ve recently lantean.co to AWS EC2. Amazon offers a free EC2 instance for a year…. so i decided to give it a shot.

The main reason i had to migrate to a self managed hosting is simple. Shared Hostings don’t allow you to fine tune several settings, such as the PHP Memory, and you might event not be able to login using ssh. What did i need to do?. It’s simple… let’s see…

 

Setting up the Environment

  1. Signup at Amazon Web Services. You’ll need a credit card.
  2. Create a new EC2 instance. Select ‘Micro’ as the type.
  3. Select the Amazon AMI. (I don’t trust 3rd party images!).
  4. Follow the wizard, and generate the SSH private / public keys.
  5. Setup the firewall, so only IP’s in your C class can connect through SSH, and everyone can hit the port 80.
  6. Connect to your box!
    ssh -i certificate.pem ec2-user@[elastic-ip]
  7. Setup a password for your root user
    su passwd
  8. Install Apache
    yum install httpd
    service httpd start
    chkconfig httpd on
  9. Install PHP
    yum instlal php php-mysql
  10. Install mySQL
    yum install mysql-server
    service mysqld start
    chkconfig mysqld on
  11. Secure mySQL
    mysql_secure_installation
  12. Install APC
    yum install php-pecl-alc

 

Setting up Apache

Assuming we’re not gonna host just a single website, but a couple of them… we’re gonna need to setup Virtual Hosts. With VirtualHosts you can serve as many domains as you need, using a single apache installation. Steps!

  1. Log into your instance and type… (replace domain.com with your own domain):
    su
    mkdir -p /var/www/domain.com/public_html
    mkdir /etc/httpd/conf.d/domain.com.conf
    nano /etc/httpd/conf.d/domain.com.conf/httpd.conf
    Add the following lines:

          <VirtualHost *:80>
              ServerAdmin your-mail@domain.com
              DocumentRoot /var/www/domain.com/public_html
              ServerName www.domain.com
              ServerAlias *.domain.com
              ErrorLog /var/www/domain.com/error.log
              CustomLog /var/www/domain.com/requests.log combined
          </VirtualHost>
  2. Enable htaccess in your virtual hosts:
    nano /etc/httpd/conf/httpd.conf
    AllowOverride All
  3. Enable logrotate:
    nano /etc/logrotate.conf
    Add the following lines:

       /var/log/httpd/*log
       /var/www/*/*log
       {
           size 5M
           missingok
           notifempty
           sharedscripts
           delaycompress
           postrotate
          /sbin/service httpd reload > /dev/null 2>/dev/null || true
           endscript
       }

 

Setting up mySQL

At last!. Let’s see how to create a mySQL database, add a new user, and how to import your mySQL dump, using nothing but bash.

  1. Create a new database and a new user
    mysql -u root -p     << You will be asked for your mySQL root-password!
    create database wordpress;
    create user 'wordpress'@'localhost' identified by 'password';
    grant all privileges on wordpress.* to wordpress@localhost;
    flush privileges;
  2. Import a database dump
    mysql -p -u wordpress wordpress < database_dump.sql

 
I hope you found this short guide helpful!