Introduction
With IT threats constantly evolving, businesses need to ensure that their information systems are protected against potential vulnerabilities. Vulnerability scanners have become essential tools for identifying these vulnerabilities before they are exploited in attacks.
Of the many scanners available on the market, Nuclei stands out for its flexibility, performance and open source approach.
In this article, we’ll look in detail at why Nuclei is an essential tool for detecting vulnerabilities. We will also discuss its main features and some concrete examples of how it is used.
Comprehensive Guide to Nuclei
Introducing Nuclei
What is Nuclei?
Nuclei is an open source vulnerability scanner developed by ProjectDiscovery. Designed to be both flexible and powerful, Nuclei automates the detection of vulnerabilities in IT systems (web apps, cloud infrastructure, networks, etc.).
Before continuing, it is essential to point out that the use of Nuclei can generate significant traffic and that it is illegal to use it without the prior authorisation of the owner of the system being tested.
Nuclei, an open source template-based tool
Nuclei relies on customisable detection templates in YAML format. These templates, as we will see later, enable a vulnerability to be defined and classified according to its criticality, type, category and a range of other parameters.
In addition, Nuclei is distinguished by its active community and the quality of its open source contributions, which enable templates to be updated regularly and new ones to be added continuously. There are currently over 8,000 templates available.
The Nuclei engine also uses template clustering. In other words, it is designed to optimise the number of requests it sends, in order to speed up scanning and reduce network traffic.
As an example, let’s imagine that 5 different template models each need to make a GET request to a connection page; the engine will only make one request, and the result will then be processed separately according to each of the different template rules.
Thanks to its ability to run thousands of tests, Nuclei is an indispensable tool for discovering vulnerabilities and correcting them before they are exploited in attacks.
Its modular architecture, adaptable to a variety of needs, makes it the benchmark vulnerability scanner, whether for basic security tests or more in-depth, specific analyses.
How does Nuclei Work?
In the rest of this article, we’ll see that Nuclei is easy to use. However, we will gradually discover that it reveals its full potential when you take the time to configure it or create your own templates.
Easy mode
In its most basic use, Nuclei simply expects a URL with the -u parameter, or, for a file containing several URLs, the -l parameter:
This scan provides a lot of information.
As no specific options were specified, Nuclei simply executed all the templates available in its default configuration (which does not correspond to all the templates available!).
The scan took around 2 minutes with 8169 Nuclei templates used on a single target. Note, however, that the server response time may vary.
Nevertheless, with this configuration, the notion of speed mentioned above is completely eliminated. In fact, the templates used by Nuclei cover a wide range of vulnerabilities, and most of them are not relevant to our target, making them useless.
Advanced features
Having explored the use of Nuclei in its most basic form, we’re now going to discover the many configuration options available to us.
Filtering templates
Template filtering is at the heart of Nuclei’s usage. It is therefore essential to know how to use it correctly. This will enable you to efficiently select the most suitable templates.
Nuclei is based on a tag system. When a template is created, one or more tags can be added to the YAML file. These tags can be used to classify templates according to different criteria.
To indicate them, simply use the -tag flag when running Nuclei, in order to filter the templates.
By specifying this flag with, for example, the cloud tag, 42 templates are loaded.
And to find all the tags available, you can use the -tgl flag.
Unsurprisingly, the cve tag is in first place, with 2456 templates currently associated with it.
Another regularly used option is -t/-template, with a file/folder path as the argument, to load the corresponding templates.
Choosing the most appropriate templates is not always a simple task. The best approach is still to select them manually, based on available information about the technologies used by your targets, and by choosing precisely the corresponding tags.
However, it is also possible to automate this process. Nuclei offers the -as or -automatic-scan option, which uses Wappalyzer to detect the technologies used, then automatically associates the corresponding tags.
Another approach is to first use the tech tag for the recon; then select the appropriate tags based on the results obtained. For exhaustive identification, the -headless option can be added to identify the technologies used on the client side.
Finally, you can filter the results by including or excluding certain severities using the -severity option. This can be useful when scanning several targets to ignore minor vulnerabilities.
Configuring and optimising scans
There are many options available for configuring and/or optimising scans. Here is a list of those we consider to be the most relevant:
| Option | Description |
|---|---|
| -timeout int | Waiting time before timeout \Default 5\ |
| -retries int | Number of attempts if the request fails \Default 1\ |
| -iserver, -interactsh-server string | Specify a self-hosted server |
| -H, -header string[] | Add specific headers to the request, in header:value format |
| -r, -resolvers | Resolver list |
| -rl, -rate-limit int | Maximum number of requests per second \Default 150\ |
| -mhe, -max-host-error int | Maximum number of errors per host before removing it from the scan \Default 30\ |
| -sa, -scan-all-ips | Scan all IPs associated with the DNS record |
Output Filtering
Nuclei offers numerous options for filtering the results. These filters not only help to refine the analysis, but also make it easier to use and store the data obtained.
Here is a summary table of the most interesting options:
| Option | Description |
|---|---|
| -silent | Allows only vulnerabilities to be shown |
| -ts | To add the time of the vulnerability scan |
| -jsonl | Output in JSON format |
| -json-export <file> | Output in JSON format, saved in the file |
| -me <dir> | Output in MD format, saved in the <dir> folder |
| -store-resp | Saves all requests/responses in the output folder |
Writing your own Nuclei templates
Although the Nuclei template repository is richly supplied thanks to a very active community, some CVEs with a public proof of concept may sometimes be missing.
It is also possible to have specific needs that the existing templates do not cover. In these cases, it is possible to create your own templates and publish them on the official Nuclei GitHub repository.
Let’s now look at their structure.
As mentioned in the preamble, all Nuclei templates use YAML. This is a human-readable data serialisation language, generally used to write configuration files.
It consists of 4 main parts:
- The template identifier
- Template information
- The data to be sent
- The response filter
We are going to use the template corresponding to CVE-2022-22965 \Spring4Shell. As a reminder, this CVE leads to a remote command execution (RCE) if all the conditions are met.
Template ID
The identifier is unique to each template and, as its name suggests, enables identification. This ID is displayed when the template conditions are met.
According to the Nuclei documentation, it must not contain spaces to facilitate parsing.
In our example, the identifier corresponds to the name of the associated CVE:
id: CVE-2022-22965Template information
The second section of a template is the info block. This block contains various general information about the template, such as the name of the vulnerability, the author, the criticality level, a description, references and other elements which provide a context for the vulnerability.
This block also includes the tags used to identify the template.
It frequently includes metadata, such as a Shodan or Fofa query (search engines that index devices connected to the Internet) to identify vulnerable targets. You can also specify the name of the vulnerable supplier or the product concerned.
In our example, the template concerns a CVE and therefore includes a classification section containing various information related to this CVE.
HTTP protocol
The third block actually breaks down into two sub-sections.
The first corresponds to the request that will be sent to our target, and the way in which it should be constructed.
The use of raw offers greater flexibility in the creation of the query body, by allowing the integration of DSL functions. These specific functions are designed to facilitate the creation of templates. Here are a few examples:
| Helper function | Example | Description |
|---|---|---|
| contains(input, substring interface{}) bool | contains(body, “login succeeded”) | Verifies if a string contains a substring |
| len(arg interface{}) int | len(body) | Returns the length of the input |
| regex(pattern, input string) bool | regex(“file-([a-z0-9]+\”, body) | Tests the given regular expression against the input string |
You can also access a set of variables, either manually defined or pre-established (‘builtins’). For example, in the case above, {{BaseURL}} represents the full URL (https://www.example.com/).
So, with this template, 2 requests will be made:
- A POST request, with a dynamically constructed part
({{BaseURL}},{{interact_protocol}}and{{interactsh-url}}). - A GET request constructed in the same way.
The second is the interpretation of the server response, and whether it meets the criteria for being vulnerable.
We have two matchers here, used to check a condition.
- The first checks that the protocol used for interaction with the remote server is
http - The second checks that the
User-Agent:Java is present in the request headers.
We also note matchers-condition : and, indicating that both matchers must be validated for the target to be considered vulnerable.
So, to summarise this template, the aim is to check whether the target is vulnerable to CVE-2022-22965 \Spring4Shell).
Two requests are made for each target, a POST and a GET. To validate the vulnerability, interaction with the remote server is required (the one provided by Nuclei or the one indicated using the -iserver flag).
The protocol used must be http, with User-Agent: Java in the headers.
If all these conditions are met, the server is potentially vulnerable and Nuclei will report the vulnerability.
Nuclei provides a range of tools for creating templates. You can find them in the official documentation.
Using Nuclei During Audits
Now that we have seen the basics of using Nuclei, let’s talk about its advantages during an audit.
Nuclei is particularly powerful for scanning a large number of targets, although this is not always necessary during our audits.
Firstly, we keep our list of Nuclei templates up to date. This tool is effective in detecting CVEs with an existing proof of concept. The Nuclei database, fed by an active community, contains a large number of templates. It also allows us to identify vulnerabilities that we might otherwise miss.
The second use of Nuclei is to provide us with a ‘report’ on our targets. It collects a range of information to guide our audit. For example, what type of API is in place on each target? What languages, frameworks and servers are used? Are there any files likely to contain sensitive information? This data allows us to better understand our target and identify potential vulnerabilities from the outset.
Nuclei can also be used as a starting point for more in-depth investigations. The results guide our manual efforts, enabling us to target the most promising or riskiest areas.
Furthermore, Nuclei’s flexibility, with the possibility of integrating customised templates, makes it a tool that can be adapted to specific audit scenarios. We can adjust it according to the specifics of each assignment, increasing its effectiveness and relevance.
So it’s a valuable tool to have in our arsenal. Its judicious use can considerably improve the effectiveness and scope of our security audits.
Reconnaissance and discovery of an attack surface
As previously mentioned, Nuclei can be very effective in the reconnaissance phase of a security audit.
This is one of the first steps we carry out. We may not find everything manually, and this is where Nuclei becomes very useful.
Take, for example, this web application that we have deployed locally (OWASP Juice Shop) and on which we have launched Nuclei with this basic command:
Without any specific parameters, more than 8000 templates were executed on our target. This command is useful to run at the start of an audit to gain a better understanding of the application.
This scan revealed several interesting paths.
Firstly, we discovered Swagger API documentation. This document describes a REST API, the endpoints, the necessary parameters, and the method to be used. It may be publicly accessible intentionally or by configuration error. Using this document, we identified an accessible endpoint.
The second file discovered by the scan is the robots.txt file. This file is commonly used by web applications to tell search engines which URLs to crawl.
Sometimes confidential paths are found in this file, as shown here. The path /ftp is indicated, leading to sensitive files stored on the application’s server.
The final discovery in our scan involves the application’s metrics. This file contains crucial information about the internal workings and performance of the system, such as performance statistics, the environment and debugging data. This file should not be publicly accessible.
In the application tested, we discovered details of the server’s capabilities and performance, as well as how it works.
Identifying a vulnerability with Nuclei
In this second scenario, we will see how Nuclei enabled us to identify a vulnerability, leading to the discovery of several medium and large-scale flaws. Exploiting one of them allowed direct access to server files via a “path traversal“.
This time we used the web application https://google-gruyere.appspot.com. This app is deliberately vulnerable and is used to raise awareness and provide training in cyber security.
Our scan revealed several vulnerabilities, most of which lead to access to the /etc/passwd file.
The exploitation here is simpler and more straightforward. All you had to do was visit one of the URLs identified to see that the /etc/passwd file had been accessed.
This file is often used as a ‘proof of concept’ because all users generally have read access to this file. This vulnerability allows direct access to server files without authentication. It is considered critical.
Although this type of scenario is rare during our audits, it can occur and expose our customers to serious consequences.
Author: Théo ARCHIMBAUD – Pentester @Vaadata