Thursday, January 24, 2013

VPC Migration: DNS

Shortly after starting the migration to VPC, I ran into an unexpected issue with DNS. I hope to give more a more informed view than I received prior to starting the migration.



DNS in EC2

One very nice thing about EC2 is the robust internal and external name resolution system. The hostname ec2-23-21-171-148.compute-1.amazonaws.com resolves to both 23.21.171.148 and 10.174.5.92, depending on where the resolution takes place. The private address is given when resolved inside of EC2, and the public address is given when resolved outside of EC2. Elastic IPs are assigned DNS names that behave in the same way.

This strategy for DNS allowed me to create CNAMEs for my services (ie db.lucidchart.com, mongo.lucidchart.com, etc). By doing this, I could replace individual servers with load balanced servers, replace instances on the fly, and more. The CNAMEs I created were all in Route53, Amazon's DNS service.  The basic fact is that I, and many of you, have been pampered by this infrastructure.

DNS in VPC

As it turns out, VPC does not have this DNS infrastructure. Not at all. Correction: VPC does have internal DNS, but it's disabled by default. See the AWS documentation.

For most instances - the ones without elastic IPs - this isn't an issue; I just used A records instead of CNAMEs. The issue, for me, was with the public instances. These instances have no public/private resolution of the elastic IP DNS names. In fact, they don't even have DNS names in the first place. This makes it difficult to do the nice ops tasks I named earlier.

One solution would be to setup your own DNS server(s) inside of your VPC. Once setup, you can change your DHCP option set to use your internal DNS resolution. I've never managed my own highly available DNS service, and I don't want to start now. So, this option was not for me.

My solution is to use Route53 to manage two separate sets of domain names, one public and one private. Every service is accessed either using the public name or the private name. For example, here is what some of my records would look like.
server1.private.lucidchart.com. A 10.0.0.1
server2.private.lucidchart.com. A 10.0.0.2 
server1.public.lucidchart.com. A 174.1.2.3 
www.lucidchart.com. CNAME server1.public.lucidchart.com.
internal-dashboard.lucidchart.com CNAME server2.private.lucidchart.com.
This solution seems to work just fine. I'm nearing the end of the migration, and I haven't run into any issues yet.

Summary

Just to be perfectly clear, let me summarize.
  • VPC instances do not have DNS names.
  • Elastic IPs in VPC do not have DNS names.
  • All Elastic Load Balancers do have DNS names.
While these are not killers, you ought to at least be aware of them.

7 comments:

  1. VPC can have the same public/private dns resolution, it's just off by default. Check your VPC settings.

    ReplyDelete
  2. Looks like you're completely right. Just checked it out:

    http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-dns.html

    I never saw the settings before now, despite my numerous searches on "VPC DNS".

    Thanks!

    ReplyDelete
    Replies
    1. Matthew,

      Could you update your post with this information at the top?

      Thanks!
      Brandon

      Delete
    2. Can you clarify? The docs you linked to don't mention public/private resolution. I've read them three times and I'm not sure how they apply to the topic at hand.

      Delete
    3. I just checked the link, and it doesn't mention anything other than public/private resolution. Perhaps you're looking for something else?

      From the section "Updating DNS Support for Your VPC":
      When you launch an instance into a VPC, we provide the instance with public and private DNS hostnames only if DNS hostnames are enabled for the VPC.

      Following this quote are the instructions to turn it on, both from CLI and Console.

      Maybe the fourth time is the charm :)

      Delete
    4. That says public/private HOSTNAMES not public/private RESOLUTION. My reading of the docs says if you check those boxes, you get hostnames. Ok, great. What are those hostnames? What do they resolve to, and how is that affected by whether the requester is inside or outside of EC2? It doesn't say, and there are zero examples.

      Delete
    5. I recommend you test it - it would take less than 5 minutes.

      In EC2, you get one hostname that gives you public AND private resolution. I don't think it works that way in VPC, though I wouldn't really know, because I wasn't aware of the option when I set up our configuration. Our entire infrastructure uses on private IPs now.

      I believe that you get a private hostname with private resolution for all servers, and a public hostname with public resolution for all publicly accessible servers. I don't think that a public hostname will resolve to a private address, but I'm not sure. You'll have to test.

      Delete