Thursday, May 23, 2013

Strange Loop 2013 Workshop

I've been accepted as a speaker at strangeloop, a programmer conference that focuses on emerging languages, concurrent and distributed systems, new database technologies, front-end web, and mobile apps. The conference is in St Louis, Missouri, on September 18-20, 2013.

My workshop is entitled Your First Scala Web Application using Play! 2.1. Here's the quick blurb I submitted.

Wednesday, May 8, 2013

Failover with PHP & MySQL using IPTables

Not too long ago, I had to replace a MySQL server in our production environment. Its replacement would have the same CNAME, and so no configuration change was needed. This particular server is in a master-master replication set with another database server. The application is set up with automated failover between the two servers. While the failover was automated, I didn't want to just shut down the database server because its slave may miss some writes. I needed a way to fake shutdown to the web servers, but keep communication open between database servers.

Looking back, it would have been a lot easier to change configuration, wait 30 minutes for it to propagate, replace the MySQL instance, change the configuration again, and wait 30 minutes for it to come back up. At the time, however, I thought of a simpler solution. I would use iptables to reject all traffic from web servers to this one database server. Once the connection was rejected, the failover would take place, and the replication would continue to work.

Not only did my plan not work, it caused about 5 minutes of downtime on our production servers.