Deploying Drupal With Git and Capistrano

Submitted on Mar 23, 2013, 4:10 p.m.

I've been hacking on Drupal recently and so far, I like it a lot. I've also been reading a little about Drupal's deployment story, and decided that it might be fun to use Capistrano to deploy the projects I'm working on.

Turns out I'm not the only one that's thought of it. Kim Pepper's approach is excellent, and he's included his Capistrano tasks on Github.

Here's my take, which is a little different from Kim's in so far as I am not that concerned about the 'Rail-isms' that come with the default Capistrano gem (for the moment at least).

Obviously you'll need a machine with Ruby installed as well as the Capistrano gem. The Capistrano getting started docs and handbook are also very helpful.

1$ gem install capistrano

First we'll take a look at how I've organized the directories of the project - following a Rails-ish layout.

The top level of the project on my development machine, including git repository and capistrano files looks like this:

1.
2├── .git
3├── .gitattributes
4├── .gitignore
5├── Capfile
6├── config
7└── public

The Capfile is created by calling capify . in the project directory. The config subdirectory, and the deploy.rb file inside it are automatically created.

Here's the contents of the Capfile - which as you can see, simply loads the deploy file from the config directory.

1# Capfile
2load 'deploy'
3# Uncomment if you are using Rails' asset pipeline
4 # load 'deploy/assets'
5load 'config/deploy' # remove this line to skip loading any of the default tasks

Equally important is the the .gitignore file which ignores local settings and the files directories for Drupal.

1# .gitignore
2# Ignore configuration files that may contain sensitive information.
3public/sites/*/*settings*.php
4
5# Ignore paths that contain generated content.
6cache/
7public/sites/default/files

I'm including Drupal core in the git repository, as well as of course all of the project modules. I'm pretty new to Drupal, so not 100% sure if this is the best way to do this, but in addition to the actual project development, I'm performing core, theme, and module updates locally first, testing them on my dev and test machines, and then using the updated git repo to deploy the updates to the production site via Capistrano. This means I'll never perform a direct module or theme update on the live site itself.

On the test and production servers, I'm symlinking to the files (assets) and cache directories, as well as settings.php (which are all excluded from the git repository).

Here's what the directory structure looks like on the production server - starting at /home/foo/public_html/foo.com:

1.
2├── backup
3├── cache
4├── current -> /home/foo/public_html/foo.com/releases/20130323004159
5├── files
6├── log
7├── releases
8├── settings.php
9├── shared
10└── tmp

And here's a description for what's in each directory (Note: all of these directories are above, and outside of the Apache site directory - which is 'public, inside of current - more on that soon...')

  • backup - scheduled mysqldump sql files, and the sql dump file that's created during a deploy.
  • cache - my Boost site cache directory.
  • current - the currently deploy version of the site - which is a symlink to a directory that Capistrano creates for each deployment release.
  • files - all of the sites public and private assets, like uploaded images etc.
  • log - apache access and error logs.
  • releases - the Capistrano managed releases directory.
  • settings.php - the Drupal settings.php file, which is symlinked into current/public/sites/default/settings.php
  • shared - shared resources including the Capistrano created cached-copy of the git repository
  • tmp - a tmp directory.

Here's the structure inside the current directory:

1.
2├── Capfile
3├── config
4├── .git
5├── .gitattributes
6├── .gitignore
7├── log -> /home/foo/public_html/foo.com/shared/log
8├── public
9├── REVISION
10└── tmp

... which is nearly identical to our local development directory including the git repository and Capfile. The log, REVISION, and tmp directories are created by Capistrano.

The public directory is our live site and is the root of a standard Drupal installation and should look familiar to any Drupalist:

1.
2├── authorize.php
3├── cache -> /home/foo/public_html/foo.com/cache
4├── cron.php
5├── favicon.ico
6├── .htaccess
7├── includes
8├── index.php
9├── install.php
10├── misc
11├── modules
12├── profiles
13├── robots.txt
14├── scripts
15├── sites
16├── system -> /home/foo/public_html/foo.com/shared/system
17├── themes
18├── update.php
19└── xmlrpc.php

Note that cache directory is what Boost thinks is its cache directory, but is actually symlinked to the cache directory above. The system directory is a 'Rail-ism' and can be ignored.

And here's the layout of the sites directory.. .

1sites
2├── all
3│ ├── libraries
4│ ├── modules
5│ └── themes
6└── default
7 ├── files -> /home/foo/public_html/foo.com/files
8 ├── settings.php -> /home/foo/public_html/foo.com/settings.php
9 └── tmp -> /home/foo/public_html/foo.com/tmp

As you can see, files, settings.php and tmp are all symlinked back to the top of our directory tree.

Okay - and now the good news. Our capistrano deployment configuration is going to create all of this for us with just a few keystrokes from our local development or deployment machine.

Here's the complete deploy.rb file located inside the config directory:

NOTE: Updated to include the use of an environment variable or prompt for the target application, e.g:

cap deploy:drupal TARGET=live

1#SSH and PTY Options
2default_run_options[:pty] = true
3ssh_options[:forward_agent] = true
4set :use_sudo, false
5set :port, 5144
6
7#On Mac OS X
8ssh_options[:compression] = "none"
9
10#Application settings
11set :user, "foo"
12
13target_env = ENV['TARGET']
14
15if target_env.nil?
16 set(:target, Capistrano::CLI.ui.ask("Target name: ") )
17else
18 set(:target, target_env)
19end
20
21#Set the target
22if target.nil? || target.length == 0
23 set :application, "test.foo.com"
24elsif target == "live"
25 set :application, "foo.com"
26else
27 set :application, "#{target}.foo.com"
28end
29
30set :keep_releases, 5
31set :drush_cmd, "drush"
32
33#We need to use the --uri option for drush and cache clearing (see task:clear_all_caches)
34#because the cache files and directries are created by the web worker process owner, (usually
35#www-data for apache). Using the uri to invoke the cache clear, will invoke the cache clear
36#under the web worker process, and not the user running this deployment.
37set :drush_uri, "http://#{application}"
38
39set :deploy_to, "/home/foo/public_html/#{application}"
40set :app_path, "#{deploy_to}/current/public"
41#Shared resources like the Boost cache, files, tmp, settings.php are located here
42#and do not change across deployments. We symlink releases to these resources.
43set :share_path, "#{deploy_to}"
44
45set :scm, :git
46set :repository, "ssh://git@github.com:/foo/foo.git"
47set :branch, "master"
48set :deploy_via, :remote_cache
49
50#Servers
51role :web, "cloud01.foo.com" # Your HTTP server, Apache/etc
52role :app, "cloud01.foo.com" # This may be the same as your `Web` server
53role :db, "cloud01.foo.com", :primary => true # This is where Rails migrations will run
54
55after 'deploy:rollback', 'deploy:drupal:link_filesystem', 'deploy:drupal:clear_all_caches'
56
57namespace :deploy do
58 task :start do ; end
59 task :stop do ; end
60 task :restart, :roles => :app, :except => { :no_release => true } do
61 #Place holder for app restart - in Ruby apps this would touch restart.txt.
62 end
63
64 #Drupal application and project specific tasks.
65 namespace :drupal do
66
67 desc "Perform a Drupal application deploy."
68 task :default, :roles => :app, :except => { :no_release => true } do
69 site_offline
70 clear_all_caches
71 backupdb
72 deploy.default
73 link_filesystem
74 updatedb
75 site_online
76 end
77
78 desc "Place site in maintenance mode."
79 task :site_offline, :roles => :app, :except => { :no_release => true } do
80 run "#{drush_cmd} -r #{app_path} vset maintenance_mode 1 -y"
81 end
82
83 desc "Bring site back online."
84 task :site_online, :roles => :app, :except => { :no_release => true } do
85 run "#{drush_cmd} -r #{app_path} vset maintenance_mode 0 -y"
86 end
87
88 desc "Run Drupal database migrations if required."
89 task :updatedb, :on_error => :continue do
90 run "#{drush_cmd} -r #{app_path} updatedb -y"
91 end
92
93 desc "Backup the database."
94 task :backupdb, :on_error => :continue do
95 run "#{drush_cmd} -r #{app_path} sql-dump --result-file=#{deploy_to}/backup/release-drupal-db.sql"
96 #run "#{drush_cmd} -r #{app_path} bam-backup"
97 end
98
99 #desc "This should not be run on its own - so comment out the description.
100 # "Recreate the required Drupal symlinks to static directories and clear all caches."
101 task :link_filesystem, :roles => :app, :except => { :no_release => true } do
102 commands = []
103 commands << "mkdir -p #{app_path}/sites/default"
104 commands << "ln -nfs #{share_path}/settings.php #{app_path}/sites/default/settings.php"
105 commands << "ln -nfs #{share_path}/files #{app_path}/sites/default/files"
106 commands << "ln -nfs #{share_path}/tmp #{app_path}/sites/default/tmp"
107 commands << "ln -nfs #{share_path}/cache #{app_path}/cache"
108 commands << "find #{app_path} -type d -print0 | xargs -0 chmod 755"
109 commands << "find #{app_path} -type f -print0 | xargs -0 chmod 644"
110 run commands.join(' && ') if commands.any?
111 end
112
113 desc "Clear all caches"
114 task :clear_all_caches, :roles => :app, :except => { :no_release => true } do
115 run "#{drush_cmd} -r #{app_path} --uri=#{drush_uri} cc all"
116 end
117 end
118end

The Drupal specific tasks are in a separate namespace :drupal.

cap -T shows us the complete list of Capistrano tasks:

1cap deploy # Deploys your project.
2cap deploy:check # Test deployment dependencies.
3cap deploy:cleanup # Clean up old releases.
4cap deploy:cold # Deploys and starts a `cold' application.
5cap deploy:create_symlink # Updates the symlink to the most recently deployed version.
6cap deploy:drupal # Perform a Drupal application deploy.
7cap deploy:drupal:backupdb # Backup the database.
8cap deploy:drupal:clear_all_caches # Clear all caches
9cap deploy:drupal:site_offline # Place site in maintenance mode.
10cap deploy:drupal:site_online # Bring site back online.
11cap deploy:drupal:updatedb # Run Drupal database migrations if required.
12cap deploy:migrate # Run the migrate rake task.
13cap deploy:migrations # Deploy and run pending migrations.
14cap deploy:pending # Displays the commits since your last deploy.
15cap deploy:pending:diff # Displays the `diff' since your last deploy.
16cap deploy:rollback # Rolls back to a previous version and restarts.
17cap deploy:rollback:code # Rolls back to the previously deployed version.
18cap deploy:setup # Prepares one or more servers for deployment.
19cap deploy:symlink # Deprecated API.
20cap deploy:update # Copies your project and updates the symlink.
21cap deploy:update_code # Copies your project to the remote servers.
22cap deploy:upload # Copy files to the currently deployed version.
23cap invoke # Invoke a single command on the remote servers.
24cap shell # Begin an interactive Capistrano session.

Note that the drupal:link_filesystem task is not shown, since its description has been commented out. It's not on the list because there should be no reason to run this task on its own.

Kim Pepper's approach is to call the Drupal specific tasks from the after events in Capitrano's regular deployment process.

1after "deploy:update_code", "drupal:symlink_shared", "drush:site_offline", "drush:updatedb", "drush:cache_clear", "drush:site_online"

This is probably wise since calling cap deploy will force the Drupal tasks to be run as well. For now at least I'm calling the default Drupal task via:

1cap deploy:drupal

Of course before any of this will work you need to create the default /home/foo/public_html/foo.com directory and deployment account on your target application server, including the backup, files and tmp directories (as well as the cache directory if you're using Boost).

Calling cap deploy:check and cap deploy:setup will create the Capistrano managed directories. And for all of this to run smoothly, you'll likely need to copy the public SSH key for the deployment account into the ~/.ssh/authorized_keys file on the target application server, as well as the remote git repository account.

For changes that don't require a full deployment and new release directory, you can use the cap deploy:upload task to upload individual files to the live server, although needless to say this should be done with caution e.g:

1deploy:upload FILES=public/sites/all/themes/foo_theme/style.css

And there it is. It may look like a lot, but the payoff is large. You get nice, safe automated deployments and rollbacks with just a few keystrokes.