Add an EBS volume

Our AWS server as deployed in Deploy an EC2 server has a single root volume (i.e. a virtual hard disk) of 8GB, which is fine for our OS, our programs and our tiny sample Tech Club database. In the real world, we may want to work with much larger amounts of data, and to do this, we will need more than 8GB of space. In this section we will keep the root volume for the OS and the programs, and we will move the database data onto a new, separate data volume.

Separating the programs from the data gives us very valuable flexibility: it means we can change database data on the fly. Let's say we had a test instance with special configuration which should always have the live data from the previous day: we can simply backup the live data volume at the end of each day and then attach it to the test instance without affecting the test instance in any other way. This can all be automated using the AWS API as we will see in Automate backups.

Firstly we will make an image (i.e. a backup) of the AWS instance so that we can get back to where we started with no problems if anything should go wrong.

  1. Make sure no one is currently using your instance. This process will shut it down for a few moments
  2. In the AWS console, click EC2, then Instances to view your instance.
  3. Right click on your instance, then choose Image - Create Image
  4. In the screen that pops up, enter a name and a description for the image. I like to use the instance name, then underscore, then the date. Everything else can be left as it is
  5. Click Create Image
  6. Click on AMIs on the left side. You will see your new image with the status "Pending". In 3 or 4 minutes, it will change to "Available", and you're good to go

Create an EBS volume and attach it to the instance.

  1. In the AWS console, click on Volumes on the left, then on Create Volume
  2. In the new screen, enter the size you need, in GB. You pay per GB, but actually it's very cheap, at the time of writing 10 GB is around 1 EUR per month and IO between the volumes is included in the price
  3. Make sure the availability zone is the same as your instance, then click Create
  4. Now you will see 2 volumes: your 8GB root volume, and the new 10GB (or whatever) data volume. Now is a good time to name them accordingly. Hover your mouse over the blank name, and name the 8GB one "MyServer1-root", and the new data one "MyServer1-data"
  5. The new data volume is not currently attached to our instance. To attach it, right click on the data volume, and choose "Attach Volume". In the new screen, click in the Instance box to automatically show the MyServer1 instance and select it. Ignore the resulting device name because it gets renamed by Linux, as shown in the warning.
  6. Click "Attach"
  7. This concludes the AWS section. We have done the virtual equivalent of opening the server case and attaching a new hard disk

Format the new volume using XFS.

  1. Now we will see what the new device has actually been named by Linux. SSH into the server (ssh -i /path/to/my-key-pair.pem ubuntu@{Elastic IP}), then enter sudo fdisk -l to show the available disks. You should have 2 entries, 1 for each volume. Mine are "/dev/xvda" for the 8GB root volume and "/dev/xvdf" for the new 10GB volume. We will format the new volume using the XFS file system.
  2. Install XFS tools: sudo apt-get install xfsprogs
  3. Format: (make sure you enter the right volume!) sudo mkfs.xfs /dev/xvdf

Create a new directory "/dbdata" which will hold all database data, and mount the new volume onto it.

  1. Create dbdata directory in instance root: sudo mkdir /dbdata
  2. Change directory owner to postgres user: sudo chown postgres /dbdata
  3. Mount new volume to dbdata directory: sudo mount /dev/xvdf /dbdata
  4. Check the mount: df -h. Should see "/dev/xvdf 10G 33M 10G 1% /dbdata"
  5. Open the fstab: sudo vi /etc/fstab
  6. Add this line to the file which means that our volume will stay mounted even after a reboot: /dev/xvdf /dbdata auto noatime 0 0

Move the Postgres data directory and all current files to the new location.

  1. Stop the Postgres service: sudo service postgresql stop
  2. Open the main Postgres config file: sudo vi /etc/postgresql/9.3/main/postgresql.conf
  3. Scroll down to the first setting, "data_directory". Change the value to /dbdata/main
  4. We will keep the log files on the root volume to improve performance. Scroll down to about 50% and find "log_directory", comment it in, and change the value to /var/lib/postgresql/9.3/main/pg_log
  5. Move the current data files to the new volume sudo mv /var/lib/postgresql/9.3/main /dbdata
  6. Restart the Postgres service: sudo service postgresql start If you made a typo when changing the Postgres config file, it will tell you here

Congratulations, that's it! Now of course we need to test everything to make sure it still works. For interest, check disk usage again: df -h You will see that the root volume has slightly more free space and the data volume has slightly less.

In the next step we will look at making use of our new separation of the OS/programs from the data.