Checking Domain Availability - or finding out how the Internet works - Blog - domcomp.com
Date


the intenet

Domcomp has always been about doing one thing - finding the cheapest domain prices. Until now.

Our philosophy is that we’d rather do less but do it well. Without trying to sound too cocky, we believe that we do domain price comparison pretty well ;) However, on considering the wider picture of what our users truly needed, it became obvious that we could improve their experiences. Previously, on finding the cheapest registrar on domcomp, a user would then go to that registrar’s site to check the domain name availability. The checks are often slow and clumsy. What’s more, if the name for a particular TLD is taken but available for another, you’d then have to return to domcomp to check the cheapest price for the alternative TLD.

What if domcomp could tell you, instantly, whether your desired domain name is available across all TLDs? We set out to do exactly that.

But how?

So now we’ve discussed “why”, let’s think about “how”. There’s the age-old interview question “what happens when you hit enter after entering a URL”. We don’t need to know all the steps but let’s go over some important bits for our purpose. On entering "https://www.domcomp.com" in the URL bar, your browser sends a query to find the IP address (a numerical address like 168.1.2.3.123) corresponding to “domcomp.com”. Skipping some (long but very interesting) intermediate details, it suffices to say that the query will finds it’s way to the TLD Name Server. That is, each TLD, such as .com or .net has it’s own set of name servers and these servers contain Zone Files, a list mapping domain names such as “domcomp.com” to IP address. The Name Servers look up the queries name against these files and returns the address to your browser. These Zone Files are exactly what we want as it tells us whether a name has been registered, or not.

To check availability, all we need are zone files for each TLD. Indeed, we requested Zone Files for the TLDs that have them available (some require you to fax, yes fax, your request!) and update these daily. When a user types, “google.com” into the domcomp search bar, we can instantly check against our .com zone file to say that “google” is taken for ‘.com’. Availabile tlds are highlighted in green, taken ones are greyed out.


domain availability on domcomp.com

Job done? Not quite.

Unfortunately, it’s not so straightforward to obtain zone files. Different TLDs are managed by different organisations with their own rules governing usage and distribution of zone files. Some are easier to obtain than others whilst some are impossible. For instance, the bodies governing ‘.co.uk’ and ‘.de’, amongst many others, do not provide public access to their Zone Files. We are not so easily deterred by a minor setback. For tlds that we do not have zone files, we designed a framework for domcomp to check availability the old-fashioned way, via DNS look up. That is, if you type in “bbc.co.uk” into domcomp, we fire off queries for that address, much like your browser would do. If the domain is resolved then we know it’s taken. This has a few disadvantages compared to zone file look up. It is both slower than checking against zone files in our local database, or worse, inconsistent information returned from various DNS server. To solve this, we built a unique framework that fires multiple DNS requests, for each letter you type, and intelligently works out inconsistencies to provide users the most accurate results.

Further improvements

We’re pretty pleased that the speed of look ups on these TLDs closely matches our zone file look ups. In fact, go to domcomp.com, type a name in the search bar and see if you can tell which TLDs availabilities are resolved are using zone file and which use DNS method. We’re able to achieve near-instant result, simultaneously on multiple TLDs thereby achieving our original goal.

This is, however, not the end. We’ve noticed that there are some rare cases (we estimate <0.5% of domain names) where our methods are inaccurate. For instance, there are some TLDs (for example .xyz) that have many domains registered but not in the zone files or configured correctly for DNS. For these, we’re currently working on a third checking method of using WHOIS requests to provide more accurate results, when the first two methods fail.

If you have time, check our hard-work at www.domcomp.com - now providing domain availability checks, as well as price comparison :)