There are a lot of people in the bug hunting community that have begun to say that basic attacks are no longer worth the effort to pursue. Even a few well known bug hunters have chimed in to support the idea. I’m not so quick to agree. Recently, after I found myself listening to a talk about bug hunting methodologies, and up came a question about typical, or basic, attacks being a waste of time and whether or not we should just exclude them from our testing, I decided to put the idea to the test. The technique in question was directory enumeration (aka directory brute force attack or forced browsing). The argument was that the time spent, for the reward, was often balanced to the degree that it was a waste of time to search for directories and files. Having just uncovered a handful of .DS_Store files on a handful of private bug bounty programs, I felt it wasn’t prudent to dismiss this tactic. But rather than voice an unproven opinion, I decided to take on his argument directly. The conclusion of my efforts? Skip to the bottom if you can’t wait. SPOILER ALERT: Directory brute force attacks are not dead!
What is a brute force attack?
By definition, a brute force attack is when a malicious individual tries every single possible combination, or permutation, of a value for a parameter. The classic example would be and attacker brute forcing a password. An attacker would try ‘a’ for the password, then they would try ‘b’, and then ‘c’. They would continue this pattern until they had tried every allowable character. After single characters were exhausted and failed, they would then try two character combinations (aa, ab, bc, a1, a!, etc), until those combinations were exhausted as well. Given the possible character combinations (26 lower case, 26 upper case, 10 digits, and varied special characters), you have 62+ characters possible for each position in the password. So a password that had no special characters in it, and a length of 8 characters, would result in whopping 62^8 possible passwords. Iterating through all those possible combinations takes a very long time.
Typically attackers will not perform a full brute force attack because of the inherent time constraints. Instead, they will turn to lists of known common passwords. This is where the idea of dictionary based attacks comes into play. Instead of using a true brute force attack, an attacker can create a list of common passwords, and only try those. Often this is confused with brute forcing, but the reality is that by using a list of known values, you are dramatically reducing the set of possibilities. And by not trying every possibility, it is by definition, not a brute force attack.
But we aren’t talking about passwords. We are talking about hacking websites and web applications. So how does a brute force attack work there? Well, you can still do password brute forcing, but there is another common form of brute force attack on a web site. And this form of brute forcing comes into play when attempting to forcefully browse resources on the site or application.
For example, let’s say you have a web hosted service, and Internet users can leverage your service by creating a subdomain, like sampleapp.example.com. To manage their manage subdomain, they must browse to an admin directory. This admin directory is not visible in the main application. You just have to know that you need to go to sampleapp.example.com/admin to access the administrative portion of the subdomain. An attacker, with zero knowledge may attempt to browse to /admin just to see what happens. This would be forced browsing of the admin directory.
But it doesn’t have to be as simple as an admin directory. There are times when sensitive files have improper permissions set. Think about having a wordpress blog, and the wp-config.php file is accidentally set to world readable. For the uninitiated, the wp-config.php file contains credentials to the database. There are no links to this file on the blog, but if an attacker forcefully browses to example.com/wp-config.php and gets the file returned, they just gained credentials.
Another form of forceful browsing can entail IDOR attacks, but that is beyond the scope of this post so we will leave that out for now.
How To Perform Directory Enumeration
We now know what brute forcing is, so let’s combine that with forced browsing (brute force browsing). Imagine attempting to identify, through brute force, every directory and filename character by character, just in the hopes of finding a resource that helps you in your attack. The password example of only 8 characters was an immense number to begin with. But picture trying all possible file names that have a 255 character limit. This becomes vastly unrealistic.
So we can safely rule out pure brute forcing as the time involved would be astronomical. But, even in a dictionary attack with a reduced list of common filenames and directories, we can still have an intensely large number of possible URLs to check. Manually checking all those possibilities would still take an exorbitant amount of time. So we turn to tools to automate this for us.
The two most widely used tools for enumerating files and directories are dirbuster and gobuster.
Dirbuster is a java based application with a GUI front end that allows you to specify a target URL, word list of items to check, file extensions to check, and many other features.
NOTE: For the red team members in the audience, if are actually using this really noisy tool, expect it to be detected on volume alone. Further, if you do not modify the User Agent string, you will likely be blocked rapidly (the default contains ‘Dirbuster’). Avoid the noise and signature detection!
A big drawback to dirbuster is it’s memory utilization and noise level. Running dirbuster in multiple concurrent instances can result in resource consumption and a slowed OS. There are reasons to run multiple, but they are usually few and far between. The other concern, noise, comes from the word list size. The default ‘medium’ list is over 200,000 items long. And with a decent connection this whole list can be checked in under 10 minutes (possibly much faster). Most sites do not see 20k requests per minute. So take that into consideration when utilizing dirbuster. Throttling is your friend.
Basic usage entails launching dirbuster, pointing to the target URL, selecting the word list, checking boxes for directories and/or files (I strongly suggest removing recursive enumeration, as that can exaggerate the completion time greatly), and simply start the scan. Very easy, very quick.
Gobuster is relatively new on the scene, but widely used by bug hunters. Gobuster is written in Go language, and can be much faster and less resource intensive. However, it lacks a GUI front end that new pen testers may feel the need to use, and requires a deeper understanding of enumeration techniques (like checking for specific response types – 500, 301, 204, etc). Generally, you can consider gobuster to be a tool for more experienced penetration testers.
Typical usage is to specify a word list, target URL, response types that are of interest, and many other advanced features (like disregard certificate issues).
gobuster -w ~/Desktop/wordlists/buster_lists/Top100000-RobotsDisallowed.txt -u https://www.example.com -fw -t 150 -s 200,204,301,302,307,500 -k -o out.file
And here we are. If you’ve read the above, then you know what the attack below looks like, and some basic tools and their usage for performing the attack. So let’s go back to my goal: Determine if there is still a point to performing such a basic level attack.
I decided that for this experiment, I would take handful of public bug bounty programs that had been open for at least a few months, that way they were already vetted by several ranking bug hunters. This doesn’t test the low hanging fruit scenario, but rather whether or not this tactic is viable and also being overlooked.
Starting with the scopes, I created a list of domains to check. Next I began passing these domains to gobuster with a fairly small wordlist. I had a few hits for files worth checking (the ever present .DS_Store file). But these were largely of no concern as they didn’t reveal any sensitive information (or, sadly, files and directories).
After a couple hours of this (I wrote a script to automate the process), I got a strange hit. A site with a .DS_Store file, also got a hit for a .git file, but nothing else. This seemed anomalous and worth investigating further. A .git file could be quite nice if true. However, when I browsed to the .git file, things took a turn for the strange. Instead of a .git file, I got a clear text PHP file.
As you can see, this file identifies the commercial product in use to be a Kayako product. Doing some quick research, I learned that this product was intended to be a support system for users that have technical issues, or just basic questions. More or less, it was a ‘Contact Support’ Dashboard. Immediately I began investigating known vulnerabilities, but ultimately came up empty handed as the known attacks were not working. Not to be dissuaded, I took the information that it was a .php file, and decided to run with that. Launching Dirbuster, and using the ‘medium’ word list, I decided to not search for directories, but only for files ending in .php.
Shown below, there weren’t many hits, but seeing as you never know what may be divulged (calls to other pages, commented out items, etc), it was worth investigating each item. It was a small list after all.
I soon discovered that the key.php file has a string of random characters. Not wanting to jump to conclusions, I suspected this may be an API key, or even a product key. Excitement began to creep up on me as I did a Google search for ‘kayako key.php’. And wouldn’t you know it, the first result came up:
Reading this thread, the first two posts lead me to believe that the key file is required to allow the application to function. This lead to the logical conclusion that this was not an API key, but was in fact, a product key.
I submit the bug and it came back a P1 (critical).
So clearly this basic tactic netted a valuable bug. At this point I think it’s fair to say the tactic is clearly still viable. But that isn’t the only take away here.
(Queue the ominous horror movie plot twist music)
The bug bounty program this was discovered in, had 174 vulnerabilities rewarded, 155 mentions in the hall of fame, and existed for at least a year.
Let me say this another way. A bug bounty program, that is over one year old, had 155 bug hunters that discovered 174 valid bugs, still had a critical sitting out there. Not only that, but many of the bug hunters, in this program’s the hall of fame, are well known and high ranking. And professional courtesy prevents me from publicly outing them.
Just because it’s a basic attack, doesn’t mean it isn’t worth doing. Further, bug hunters are biased and go for the optimal amount of payout to time ratio. Meaning, they go for as much as they can make as quickly as they can make it, then they bail. Often times a bug hunter’s motives are not altruistic, and they would rather seek to pad their rankings and wallets. So doing a full audit of a bounty program will often yield items other bug hunters just do not deem as common enough, or high paying enough, to spend time investigating.
Due diligence is king.