Background

I’ve been using the Amazon EC2 environment for a while now, mostly for basic development or testing environments. Since these environments have all been reproducible very easily we haven’t worried too much about getting them setup to resist an instance failure.

This week however, I’ve had three separate conversations regarding how we can get environments safely built for more complex applications:

• Team Foundation Server Beta 2
• Microsoft CRM / SharePoint Server Development Environments
• Oracle SE1

In each of these cases, the environment is only required for a short time while developing or demonstrating … then they will be shut down again.

Each of these needs some form of database storage … and since this will be a development environment we don’t want to be paying $1 / hour for the Amazon SQL Server instance, given that we are entitled to use our development licenses for this purpose.

One serious note about EC2 Drives

Hard drives on EC2 are split into three different categories:

• (C:) is the system drive, and is automatically attached when an EC2 instances. It should be noted that ONLY data on the C: is persisted when an instance is bundled!
• (D:) is a temporal storage drive, and should be used strictly for short term storage, as this does not get persisted when an instance is bundled.
• Additional Elastic Storage Blocks (ESB) may be added. ESBs are treated as additional hard drives, but they are stored within S3 and therefore will (usually) remain intact beyond the life of an EC2 server instance. As the capacity of the EC2 system drive is so low (10GB for a small instance) it is necessary to store all programs and data that need to be persisted within ESB drives.

I make this distinction right at the start of my post – because I got caught out by the fact that the D: drive does not get bundled … and had to start all over again 😦

Initial Setup

For this configuration we’re going to need the following setup on EC2

• 1 x 10GB ESB for storing installers and ISOs
• 1 x 5GB ESB for database
• 1 x EC2 instance for downloading files
• 1 x EC2 instance for Application installation

I’m working on the assumption that everything is installed on the same instance today – though obviously one could separate the various components of the installation, and use Amazon security groups as a way to restrict access and provide layers of security.

Preparation – Download the Installers

Maybe I’m being overly cautious, but I tend to setup a specific instance just for downloading the application installers and copying them to an Elastic Storage Block drive. This way any bloat that the download causes does not affect the size of the installed application AMI when we go to bundle it later.

To get started download installers for the following applications:

• MagicDisc (or your favourite ISO mounting tool)
• Microsoft SQL Server.
• Microsoft Team Foundation Server 2010 Beta 2
• Microsoft Visual Studio 2010 Beta 2

Copy each of these installers to an ESB drive, and then shut down the current EC2 instance.

Setup the Base AMI image

I decided to base my build on the “Amazon Public Images – Basic Microsoft Windows Server 2003” image, though in theory any of the Windows based images could be used. One note of caution though – if the author of the image is not Amazon, then ask yourself whether you trust the author not to have built in any rogue elements to the image.

Once the image is fully booted, add the new ESB drive (for storing the application database files) to the instance first, followed by the ESB containing the installers. The timing and order of adding these drives is important. If you load the new ESB drive before the instance is fully booted then the instance gets a little confused and loads the ESB as D: rather than E: – causing inconsistencies later when we create new instances from the bundled AMI.

After formatting and installing the new ESBs, the instance setup should look something like this:

• (C:) System drive containing the OS
• (D:) 160GB Blank instance drive
• (E:) 10GB+ Blank ESB drive for storing data
• (G:) 10GB ESB drive containing our installation media

Other components that need to be installed before going any further are:

• IIS
• ASP.NET

In order to install the additional windows components, it is necessary to mount the Windows 2003 media. The following article contains a list of snapshots that can be used to create Elastic Storage Block volumes containing the windows installation disks:

• http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1802

Installing SQL Server

On our new EC2 instance install MagicDisc (or some other ISO mounting tool) and mount the SQL Server installers.

The key configuration variations from the defaults were as follows:

• Setup the data directories for the SQL Server Instance to be stored on the E: drive.
• Ensure that all SQL Server features you require are installed. Since this particular install is for TFS 2010, I’ve enabled Full Text Search, Analysis Services and Reporting Services.
• Ensure that the SQL Server management studio is installed.

After the installation is complete, it pays to double check that all of the SQL Server folders on the E: drive have been assigned the correct permissions. I found with my install that I had to give the Network Service account some additional privileges in order to get the SQL Server services to start.

Installing Team Foundation Server 2010 Beta 2 (Pre-Configuration)

Installing Team Foundation Server 2010 has been described in many blog posts before, so I’m limiting the description in this post to the specifics of getting the install working on EC2.

Installation of TFS2010 is done in two steps; firstly the installer is run to setup the base application files, then the configuration tool is used to setup Team Foundation Server for use.

1. Mount the Team Foundation Server 2010 Beta 2 ISO image
2. Set the install location for the Features to the E: drive

Configuring Team Foundation Server

1. Create a local service account to run the Team Foundation Service. I’ve called mine TFSServiceAccount.
2. Start the Team Foundation Server Administration Console
3. Enter the local user account setup in step 1 (TFS Service Account) to run the Team Foundation Server services.
4. Select whether you are after a Basic or a Standard configuration scenario. Either way, all the defaults should be sufficient to get the installation working – though I did find I had to spend a bit of time sorting out IIS and the Reporting Server (since I’d make the mistake of installing SQL Server before IIS).

Setting up SQL Server Backups

Even though the SQL Server databases are being stored on Elastic Storage Block drives – there is still a chance that the ESB itself will fail. Don’t get me wrong … ESB is much safer than storing the data directly on the instance storage – but there is still a risk that it can be irretrievably lost. The following article by Michael Friss discusses the different backup strategies for SQL Server on an EC2 instance – and also includes a good PowerShell script for performing the backup and saving it to S3.

• http://friism.com/ec2-sql-server-backup-strategies-and-tactics

Any Comments …

I have to admit; I hit a number of issues performing this setup and had to restart the overall process three times. I’m sure there may be better ways to setup some of these features … especially the SQL Server components. I’d like to hear any feedback you’ve got, or what works well – so please leave a comment if you have any thoughts.