Throughout my many development projects I have produced applications in many different fashions. I’ve written code by shelling into the production server and editing files that were serving up content to end users in real time (YIKES!). I’ve developed directly on test machines and SCPed files directly up to production servers (YIKES AGAIN!). In the last few years, however, I’ve worked with different companies, many of whom have much more formalized development procedures, and I’ve learned a few things … to put it lightly.
One of the more profound techniques I’ve brought away from my tenure as a web software developer, is the use of a Virtual Machine or Virtual Development Environment as the first step in the development process. I won’t go into the details of how Virtual Machines are utilized in a development procedure (hopefully if you’re reading you are already using a VM), but I will touch on a few of the basic concepts as I describe my procedure for developing Facebook applications using a VM.
Virtual Machines are great for development. Each engineer gets their very own server to develop on, that mirrors the test and production servers. They can hack away at core code (or installation specific code) in a completely sanitized environment and only add their code back into the shared pool when they’ve tested it thoroughly themselves. They could even create their own continuous integration server if they wanted to. I say these things only to reinforce the fact that I like to use VMs and even on small-scale projects I implement them. This brings us forward to the point of this post:
Developing a Facebook Application on a Virtual Machine
NOTE: The examples below make some broad assumptions of your level of knowledge, like it assumes you have mod_proxy already enabled and it assumes that you already have your VM user talking successfuly to your test server user. This is just a high-level overview. Hopefully you can use your highly developed inference skills to fill in the gaps.
In my most recent incarnation as a “Web Developer,” I am the “Senior Developer” (read “Only Developer”) producing and maintaining a small-scale Facebook application. Please keep in mind that the following procedure is based on my current system at work and to follow it exactly would mean you have to have a similar infrastructure. I have a little Linux test machine on-site. It’s an old Dell desktop running Ubuntu DESKTOP, so nothing fancy at all. If you’re using an off-site test machine or Windows servers, you can probably still get some insight from this post, but the specific examples won’t be of much help.
Now, scale of the project aside, I like VMs, so I implemented one for developing this application. I set it up as a simple LAMP stack using Ubuntu Server 9.10 and turned on all the necessary pieces. I set up my standard framework and put everything into place. “This is great!” I thought. “I have completely streamlined this process! Look at this great little VM I have set up to develop on.”
Very soon after that I got my first ticket telling me we had to launch a new customer installation of the app. “All right!” I exclaimed. “I get to use the new VM!”
NOTE: We do a white-label application, so it needs to be completely devoid of our influence. No corporate name or logo apart from the client’s is allowed to be seen, in doing this we create an application on Facebook for each customer installation. This way we can have custom logos, description, and canvas page URLs per client. It get a bit unwieldy as we have to create TWO applications per client, a production application whose canvas callback URL points at our production system and a test application whose canvas callback URL points at our test server, but it works out well for our size.
The first step in launching a new client installation of our app is to create the test application. I log into our Facebook developer account and proceed to begin setting up the application. I get to the “Canvas Callback URL” section and a moment of severe disappointment grips me. The VM is completely internal and there is no way for Facebook to talk to it. Unless I’m willing to create a “Facebook VM application” for each of our developers (I know it’s only me, but I’m thinking of scalability here, people) and then there’s the problem of DNS entries, because every VM has an internal IP on our network, which is not visible to Facebook anyway.
BOO. It looked as if continuing to develop code directly on the test server was going to be the only way to test our Facebook application properly.
I wasn’t satisfied with that conclusion. I liked VMs. I liked sanitized development environments, especially when working with entry level developers. There had to be away around this.
I started down the road of trying to jury rig the application so it could function outside of Facebook. Meaning there didn’t need to be a Facebook frame around it and there didn’t have to be a valid Facebook session present. Then we could just access it via an internal IP like: http://192.168.54.75:/facebook-app or if we messed with our host file a bit http://nick-vm/facebook-app.
In the VERY long-term this could have been a viable option. Development would have been simple and the application could have had more breadth as it could have functioned as a standalone website. Due to time constraints, however, this was not going to work. It was surprisingly difficult to add code that could properly spoof a Facebook session and even if I had achieved that goal, it was turning my session code into spaghetti with all sorts of nasty environment based conditionals that added NO VALUE (and actually decreased value because of the increasingly unreadable code and creeping file size) to the final product that would live on the production server.
In a stroke of “genius” (read “Nick figuring something out”) I decided that the way to go about this would be to trick Facebook into thinking the content served to it was coming from the test server, when in fact, it was coming from my VM.
With some tricky BASH scripting and Apache’s very convenient mod_proxy module, I was able to create an automated check in/check out system that would tell the test server to take Facebook’s incoming HTTP request and forward it along to an internal machine … my VM, on a temporary basis. This meant that the content being served to Facebook would be coming from my virtual machine using the test server as the intermediary.
There was a number of things gained by this approach. The canvas callback URL in Facebook could permanently remain as “http://application.testserver.com/,” there would not need to be any changes to the application business logic, and core code could be developed by an engineer with no harm coming to code on the test server that was currently powering the test installations.
The check in/check out scripts were superbly easy to write too and were easily executable directly from the VM. Each script, when you boil it right down, needed to accomplish one thing. The “check out” script needed to alter the Apache virtual host on the test server to proxy the request through to the virtual machine – and the “check in” script simply needed to reverse this process.
So, the first thing we needed was a vanilla virtual host file that could be used as the proxy template:
<VirtualHost *:80> ServerName {appname}.testserver.com RewriteEngine On RewriteRule ^(.*)$ http://{user}-vm$1 [P] </VirtualHost>
Replacing the {appname} and {user} variables using a BASH script, AND making sure that the necessary entries were present in the test server’s host file, AND ensuring my virtual machine had a static internal IP, AND ensuring that my virtual machine had a virtual host file that accepted the HTTP request “nick-vm”, AND ensuring that Apache got bounced on both my virtual machine and the test server … made everything work perfectly. Facebook never knew the difference.
Here’s a snippet of the BASH check out script that shows how to edit this file. You’ll have to visualize in your minds how the variables get populated:
ssh testserver.com mv $VHOSTDIR/$APPNAME.conf $VHOSTDIR/$APPNAME.conf.checked.out.by.$USER ssh testserver.com cp -f $VHOSTDIR/vhost-checkedout-template.conf $VHOSTDIR/$APPNAME.conf ssh testserver.com sed -i "s/\{appname\}/$NAME/g" $VHOSTDIR/$APPNAME.conf ssh testserver.com sed -i "s/\{user\}/$USER/g" $VHOSTDIR/$APPNAME.conf ssh testserver.com chmod 644 $VHOSTDIR/$APPNAME.conf ssh testserver.com chown $USER:dev $VHOSTDIR/$APPNAME.conf
That would spit out a vhost that looked like this:
<VirtualHost *:80> ServerName app1.testserver.com RewriteEngine On RewriteRule ^(.*)$ http://nick-vm$1 [P] </VirtualHost>
The original virtual host file would be safe and sound – copied over to a file called “app1.checked.out.by.nick.” This would make it instantly apparent just by looking at the file system that the app was being edited by someone. This was also the way the check out script would know it could not check that file out until it was checked back in again.
After bouncing Apache, this virtual host would make sure all requests to app1 from Facebook went through to my virtual machine. Of course, the test server needed to know that “nick-vm” pointed to my VM. So, I gave my virtual machine a static IP and made sure the test machine’s host file had that record:
nick-vm 192.168.54.75
This is really the only “unscalable” piece of this system. For every engineer you onboard, you need to allocate a unique static internal IP for their VM and add the record to your test server’s host file. But the onboarding process will include creating accounts for them anyway, so you can just make this part of that procedure.
Once the app was checked out and the necessary code edited, all the check in script had to do was reverse the process:
ssh testserver.com mv $VHOSTDIR/$APPNAME.conf.checked.out.by.$USER $VHOSTDIR/$APPNAME.conf
Then bounce Apache on both the test server and the VM. Done and done.
Now, my specific check in/check out scripts did other things like packaging up databases and checking app code out of Subversion. They also fixed permission and did everything needed to properly configure an installation to run on a server. But the key point was it used the Apache virtual host files on the test server to create that connection between Facebook and the virtual machine.
A relatively large caveat to this system is that only one engineer can work on an application at any one time and the virtual machine version of the installation becomes visible to ANYONE in the world who adds the test application to their Facebook account. Since, it’s only a test application, this isn’t too big a deal, and can be combated by adding some sort of staging environment to your development process. This will ensure that clients never see test versions of the application, but it will also mean that each application installation will require three Facebook applications instead of two.
In the end I found this to be a pretty elegant solution. Multiple engineers could work on multiple apps and test them directly through Facebook. The biggest gain was that the core application code that was shared between all installations could be edited without ever impacting the stability of the core code on the test server. This meant that if you were changing core functionality, only one app at a time would be affected. This was a HUGELY significant, because clients could be testing their applications on the test server and I could be developing core code on my VM and the integrity of their applications would not be affected.
You may think that this is a bit of overkill for your application, but try to look at it this way: too many applications are not given the serious attention they need at the beginning and wind up being amateurishly written applications that have important jobs to do that they are incapable of doing. If you take some large-scale approaches in the beginning, you can save yourself A LOT of headaches later on. And we all do want our Facebook applications to become the next big thing … don’t we?
INTERESTING NOTE: I should add that I have successfully deployed this technique via VPN. So, I can have my VM running on my laptop at home, VPN into my office, and have the test server proxy the HTTP request from Facebook over VPN back to my VM at home. This means you do not even have to be at the office to check out an application installation and work on it. There might be an eensy-weensie loss of performance this way, but that’s a small price to pay to be able to write software in your skivvies.
