Various subdomain enumeration techniques are used to identify the attack surface of a domain or organisation.
The aim is to obtain as complete a list as possible of all the subdomains associated with the targeted domain.
At Vaadata, we carry out this type of enumeration almost systematically during our audits. This is done as part of a global reconnaissance phase, in order to provide our clients with an overview of the subdomains and servers exposed.
As well as being informative, this analysis enables the pentester to gain a better understanding of certain elements of the architecture of the client’s information system (such as dedicated servers or DNS wildcards).
It can also reveal technical information or data leaks relevant to the audit. For example, a subdomain dedicated to the documentation of a public API, an unprotected administration interface, or a subdomain used for phishing.
Two Different Approaches to Performing Subdomain Enumeration
There are two main types of subdomain enumeration techniques:
- Passive enumeration techniques: These make it possible to collect subdomains without interacting directly with the servers of the targeted domain. Their main advantage is discretion, as these methods do not generate any direct interaction and therefore do not activate alerts.
- Active enumeration techniques: These involve direct interaction with the target (such as HTTP or DNS requests). Although less discrete, they can be used to verify the existence of a specific subdomain or service.
In this article, we will explore the different techniques we use, both active and passive. Both types of technique are commonly used during pentesting. This is not an exhaustive list, but an overview of the most common methods.
What are the Different Subdomain Enumeration Techniques?
Manual enumeration
This technique simply involves visiting the website or web application of the targeted domain, then clicking on all the links present. Some links may redirect to different subdomains. The aim is to follow each link until you have explored every possible subdomain.
Although this is an active enumeration technique that is not very effective, it does provide rapid results in just a few clicks. It can also be used to identify web sites and applications designed for public access.
Enumeration using search engines
This technique is simple and easy to implement. It uses the advanced search operators of search engines to list indexed subdomains. Although it often fails to uncover a long list of subdomains, it provides a quick overview of the main subdomains linked to a web server and indexed.
It can be used as a complement to the manual technique. Being a passive method, it requires no direct interaction with the servers.
Let’s take Google as an example. There are a number of operators for targeted searches. The ‘site’ operator is one of them, and can be used to list the pages associated with a website.
For example, to list the subdomains of vaadata.com, you can use the following syntax:
site :.vaadata.comThe request ‘.vaadata.com’ indicates that we are looking for all URLs ending in .vaadata.com. Then all you have to do is look at the results to identify several subdomains.
Here’s a screenshot of the previous search where two subdomains are visible:
The operators may vary depending on the search engines used. For more information on this technique based on search engines, we refer you to our dedicated article: Exploiting google dorks to strengthen your security.
Certificate Transparency
Certificate Transparency is a security standard designed to monitor and audit digital certificates. Initially proposed by Google, it was subsequently adopted by the IETF (Internet Engineering Task Force) under RFC 9162.
The system is based on public logs listing all certificates issued by recognised certification authorities. The aim is to easily detect invalid or fraudulent certificates.
These logs are accessible to everyone and can be used to list the subdomains of a domain. We use several search engines to query these logs, including three main ones:
The screenshot above shows a search on the domain ‘vaadata.com’ via the ‘crt.sh’ site. In red, we see that certificates have been issued for various subdomains.
This method is passive, but you need to be careful: some of the subdomains listed may no longer be accessible or may have disappeared. It may be useful to use active tools to check whether a subdomain is still valid.
Bruteforcing Subdomains
There are two steps to brute force subdomains.
Step 1:
The first step is to generate a list of potential subdomains. Keyword dictionaries can be used to create a very large list of subdomains. These dictionaries, available online, are not based on specific permutations, but on a wide range of possibilities. It is also possible to create your own dictionaries using tools.
Another method is to generate variations or permutations from an existing subdomain. For example, if the subdomain ‘api.vaadata.com’ exists, it may be useful to test variations such as:
- api1.vaadata.com
- api2.vaadata.com
- api-dev.vaadata.com
- dev.api.vaadata.com
These two techniques can be combined to generate a more complete list.
Step 2:
Once the list has been generated, you need to identify the valid subdomains. This is done using tools such as ‘puredns’ or ‘dnsx’, which attempt to resolve the subdomains. Depending on the size of the list, this can generate hundreds or even thousands of DNS queries, making this method active and unobtrusive.
Ideally, you should use a reliable list of DNS resolvers to guarantee accurate results and avoid being blocked by too many queries.
Using online resources
There are many online resources for obtaining information on a particular domain. These are passive techniques:
- DNS aggregators: These are services that collect DNS data from numerous servers and allow searches to be carried out on this data (such as VirusTotal, DNSDumpster, Netcraft, etc.).
- Web archives: These services record the status of a site or domain at a given time. The best known is the Wayback Machine. By searching these archives, it is possible to find subdomains associated with a client.
- Shodan: This search engine specialising in connected objects can also provide information on subdomains linked to an IP or domain.
Other subdomain enumeration techniques
It is important to note that there are other techniques for finding subdomains that can be used in specific cases:
- Zone transfer only possible if the DNS server is incorrectly configured.
- CSP analysis of HTTP responses.
- SPF and CNAME data analysis
- Virtual host enumeration
Subdomain Enumeration Tools
Before concluding this article, here is a list of known tools for automating subdomain enumeration. These tools use some of the techniques mentioned above:
Amass
Amass is a free tool offered by OWASP for mapping the network attack surface of a target. It enumerates subdomains using passive and active techniques.
Amass relies on several APIs and data sources, such as search engines, certificate transparency and DNS enumeration.
SubFinder
SubFinder is a tool specially designed for subdomain enumeration. It uses only passive sources. Its main advantage is its speed. It can be supplied with a list of DNS resolvers to check the validity of the subdomains identified.
Gotator
Gotator is a tool for generating dictionaries using permutation techniques. It is widely used to create lists of potential subdomains.
Puredns
Puredns is a tool for quickly resolving subdomains and performing brute force. It also has an algorithm for detecting wildcard certificates, which helps to avoid irrelevant results.
SecList
SecList is a repository that contains many useful dictionaries for different scenarios. Keyword dictionaries can be used to generate lists of potential subdomains.
There are many other tools that we’ll let you discover for yourself: BGP, MassDNS, Subliste3r, etc.
Conclusion
As we have seen, there are several techniques, each with its own advantages. To obtain the most comprehensive results, it is ideal to combine several of these methods. For example, you can develop a custom script or aggregate the results of different tools.
Multiplying techniques and tools will significantly increase the chances of discovering more subdomains.
We’ll leave you with an example of a workflow using the different techniques listed above. This workflow combines active and passive methods to produce a final list of existing subdomains.
This is simply an example to give you an idea of the overall approach. It is of course not exhaustive and can be improved according to your objectives and your knowledge:
- Create a list of subdomains using the certificate transparency databases.
- Generate a list of potential subdomains based on a dictionary of keywords.
- Generate a list of potential subdomains based on permutations or alterations.
- Use any other suitable passive search technique.
- Combine all the previous results into a single list with unique entries per line corresponding to a potential subdomain.
- Run this list through a tool that resolves each entry to obtain a list of active subdomains.
- Use the search engine technique and add the subdomains to the list of active subdomains if they are not already there.
The use of different tools is of course recommended for the previous workflow.
Author: Yoan MONTOYA – Pentester @Vaadata