Config Migration

TLDR; I made a thing for migrating my configuration files, I like it, and it’s available for use here: https://github.com/AndyM84/config-migration

As the technology world settles into the ‘DevOps’ era and begins passing it out to other areas (looking at you, marketing automation), I figured I’d share one of my deployment “pet peeves” and solutions.  Further embracing my love of disclaimers, keep in mind that this is my own cockamamie solution, and I’m just as likely to be wrong as right.

The Problem

It seems there’s a bit of a gap in many companies’s deployment systems.  Numerous jobs I’ve held in recent years have worked hard to put various types of automation in place, but for whatever reason there wasn’t a lot of thought lent to automating configuration files during deployment.

In almost every case where this was possibly a concern, it was either expected that the system would gracefully fail when new configuration values were missing, or it would be left up to some poor release engineer to figure out which configuration property was missing and thus causing the newest in a long series of catastrophic upgrade failures.

Either way, expensive blocks of time were wasted or only protected because of extremely dedicated and entrenched individuals.  Is this a huge time-sink for release engineers?  Obviously that depends, but at the least this is hardly a ‘DevOps-y’ approach to things…

My Solution

With this in mind, I did what I do best/most-often, which is to create a solution before searching for something else that did the job.  My simplistic solution was two-fold:

1. Restrict my settings files to be non-nested JSON strings
   
2. Build a simplistic migration instruction set that allowed me to add, change, rename, and remove settings properties

#1 was mostly influenced by the infamous web.config files used by IIS/ASP.NET.  I love IIS and ASP.NET, but working through web.config files always felt to me like trying to navigate that corridor of horrors from Indiana Jones and the Last Crusade.

#2 is equal parts ego (thinking that I know better than everyone before me) and a strong desire to mimic the same sort of process used with database migration utilities, such as RoundhousE.

The result of this was a system based around files, named <OLDVERSION>-<NEWVERSION>.cfg (eg, ‘3-4.cfg’) that contained instructions similar to this:

coreVersion[str] + 0.1.0
siteName = A Site
database_host > dbHost
emailFrom -

This file would do the following things, in order:

1. Add a new setting, coreVersion, of type string with the value ‘0.1.0’
   
2. Change the value of the siteName setting to be ‘A Site’
3. Rename the database_host setting to be called dbHost
4. Remove the emailFrom setting
5. Update the configVersion setting to now be the integer 4 (was 3)

I’d already been in the habit of using a script to manage my configuration files (keeps me from having to worry about the format of the file itself), so all I had to do was insert this migration system onto the beginning of that script.  This means I can now run my typical ‘version’ update call and feel confident that any new fields added to my setup script will be present:

php ./scripts/configure.php --non-interactive --PcoreVersion=0.1.0.25

In Conclusion

After all of this jibber-jabber, this is ultimately a simple problem to which I’ve given a simple solution.  I’m sure there are other (better?) ways to handle this sort of thing, but until I find them I’ll be using this to alleviate at least one other thing in my CI/CD pipeline.

Just to make sure I’m remembering to share, I’ve published a PHP version of this system on my GitHub for your amusement.  I’ll probably throw this (or at least some version of it) into one of my many-splendoured frameworks at some point, but for now I think I’ll move on and get back to some actual work.  Enjoy!

https://github.com/AndyM84/config-migration

– Andy