Automating new dev sites for new branches

Over the past few years we've moved away from using Subversion (SVN) for version control and we're now using Git for all of our projects.  Git brings us a lot more power, but because of its different approach there are some challenges as well. 

Git has powerful branching and this opens up new opportunities to start a new branch for each new ticket/feature/client-request/bug-fix. There are several different branching strategies: Git Flow is common for large ongoing projects, or we use a more streamlined workflow.  This is great for client flexibility — a new feature can be released to production immediately after it's been approved, or you can choose to bundle several features together in an effort to reduce the time spent running deployments.  Regardless of what branching model you choose you will run into the issue where stakeholders need to review and approve a branch (and maybe send it back to developers for refinement) before it gets merged in.  If you've got several branches open at once that means you need several different dev sites for this review process to happen.  For simple tasks on simple sites you might be able to get away with just one dev site and manually check out different branches at different times, but for any new feature that requires database additions or changes that won't work. 

Another trend in web development over the past few years has been to automate as many of the boring and repetitive tasks as possible. So we've created a Drush command called Site Clone that can do it all with just a few keystrokes:

  1. Copies the codebase (excluding the files directory) with rsync to a new location.
  2. Creates a new git branch (optional).
  3. Creates a new /sites directory and settings.php file.
  4. Creates a new files directory.
  5. Copies the database.
  6. Writes database connection info to a global config file.

It also does thorough validation on the input parameters (about 20 different validations for everything from checking that the destination directory is writable, to ensuring that the name of the new database is valid, to ensuring that the new domain can be resolved).

Here's an example of how it's run:


drush advo-site-clone --destination-domain=test.cf.local --destination-db-name=test --git-branch=test
---VALIDATION---
no errors

---SUMMARY---
Source path : /Users/dave/Sites/cf
Source site : sites/cf.local
Source DB name : cf
Destination path : /Users/dave/Sites/test.cf.local
Destination site : sites/test.cf.local
Destination DB name : test
New Git branch : test

Do you really want to continue? (y/n): y
Starting rsync... [status]
Rsync was successful. [success]
Creating Git branch... [status]
Switched to a new branch 'test'
Git branch created. [success]
Created sites directory. [success]
Updated settings.php. [success]
Created files directory. [success]
Starting DB copy... [status]
Copied DB. [success]
Complete [success]

There are a few other things that we needed to put in place in order to get this working smoothly. We've set up DNS wildcards so that requests to a third-level subdomain end up where we want them to.  We've configured Apache with a VirtualDocumentRoot so that requests to new subdomains get routed to the appropriate webroot.  Finally we've also made some changes to our project management tool so that everyone knows which dev site to look at for each ticket.

Once you've got all the pieces of the puzzle you'll be able to have a workflow something like:

  1. Stakeholder requests a new feature (let's call it foo) for their site (let's call it bar.com).
  2. Developer clones an existing dev site (bar.advomatic.com) into a new dev site (foo.bar.advomatic.com) and creates a new branch (foo).
  3. Developer implements the request.
  4. Stakeholder reviews on the branch dev site (foo.bar.advomatic.com). Return to #3 if necessary.
  5. Merge branch foo + deploy.
  6. @todo decommission the branch site (foo.bar.advomatic.com).

Currently that last step has to be done manually.  But we should create a corresponding script to clean-up the codebase/files/database/branches.

Automate all the things!