DDoS and DDon'ts

Note: To skip the story and immediately go to the script that will fetch Cloudflare IPs and whitelist them using ufw, scroll down.

An interesting thing happened today: Someone contacted one of my clients and told them that he found a catastrophic regex backtracking vulnerability in one of their apps. This was interesting, because the app is a simple Next.js site that doesn’t use regex. Also, as far as I could see, Next.js doesn’t have any such vulnerabilities reported against it, so we were curious to figure out what was going on.

My client asked him to demonstrate the vulnerability, and he did. Sure enough, it brought the service down, but it also brought down the entire server, which was a bit odd. “Oh well”, I thought, “maybe it took up enough CPU to make the machine unresponsive”. The reporter then asked for a fee of 6 ETH to give my client information about the vulnerability and how to fix it, and gave references from other services. He even asked about possibly being hired by my client in a full-time position, to help with security.

Observing the vulnerability first-hand

I wanted to see the request for myself, though, so I could figure out which path it hits, reproduce it, and fix it. I asked my client to put the reporter in touch with “their security person” (me), so I could ask for another demonstration, this time being ready with logs to see what was going on.

The reporter agreed, and I logged onto the server to look around for signs of what happened in the last demonstration. I expected to find a URL with a long string component that triggers some regex to backtrack a lot, but I found no such thing.

Instead, I found 600,000 requests per minute.

That’s a DDoS

So this crafty person wasn’t a white hat hacker at all, he was a run-of-the-mill scammer who bought a few minutes of DDoS for a dollar or two and wanted to convince us that he’s exploiting some vulnerability to bring the service down. He was hoping for a quick payday, at which point we’d likely never hear from him again. Luckily for us, Cloudflare provides excellent protection against DDoS, and it was very easy to turn it on for my client’s services.

After having turned Cloudflare on, I asked for another demo, having prepared a few things.

The second demo

Before the second demo, the reporter remarked on how I had turned on Cloudflare, but that it wouldn’t help with mitigation. I asked why that matters, since this is a regex vulnerability, and asked him to continue with the demo. Having heard this, though, I assumed that he knew the server’s IP, and could just bypass Cloudflare.

He started the demo, and, sure enough, the server started getting a ton of requests, it was a bog-standard DDoS. Luckily, I had been prepared.

I had cobbled together a simple script to fetch Cloudflare’s IP ranges by downloading the files they publish, and then to use ufw to allow only those IPs for HTTP, and block everything else. I ran the script and, when the final reject rule had been added, all the requests stopped immediately.

Meanwhile, the scammer was getting more and more irate, complaining about me having enabled Cloudflare. He really didn’t like it when I told him I enabled Cloudflare because of an “unrelated” DDoS attack earlier today, but that the attack was small and unimportant, probably carried out by some script kiddy with a few cents to spare. At this point, I knew, and he knew I knew, and I knew he knew I knew, which is why it was so much fun to wind him up. He had already gotten angry, so after I called his attack “unimportant”, he stopped responding completely, probably also realizing that he wouldn’t be getting any money from us.

All in all, it was a lot of fun.

The script

The script that will block all HTTP requests from anyone except Cloudflare is below. It works by fetching Cloudflare’s IP lists, which are files that contain all the IP ranges Cloudflare will request stuff from. Then, it will use ufw to whitelist the IPs and block everything else on ports 80/443. This way, you can block everything except Cloudflare, and be sure that anything that does make it to your server is from Cloudflare themselves.

It won’t block SSH (port 22), to prevent you from locking yourself out.

Here it is:

#!/bin/bash

set -euo pipefail

# Get the Cloudflare IPs
curl -s https://www.cloudflare.com/ips-v4 -o /tmp/cloudflare_ips_$$
echo "" >> /tmp/cloudflare_ips_$$
curl -s https://www.cloudflare.com/ips-v6 >> /tmp/cloudflare_ips_$$

# Reset the firewall to clean stuff.
ufw --force reset

# Make sure the firewall is enabled and started, as the above command
# stops it.
ufw enable

# Allow my IP (change this to any IP you want to allow).
ufw allow from 1.2.3.4

# Allow SSH.
ufw allow ssh

# Allow traffic from Cloudflare IPs on all ports.
for ip in $(cat /tmp/cloudflare_ips_$$)
do
  ufw allow proto tcp from $ip comment 'Cloudflare'
done

# Reject HTTP/HTTPS, for good measure.
ufw reject 80
ufw reject 443

# Reload ufw.
ufw reload > /dev/null

# Show the rules to verify it worked.
ufw status numbered

Epilogue

It’s a bit sad that scammers like this guy exist, they give white hat hackers a bad name. When I thought there was an actual vulnerability, I was fairly thankful that he decided to disclose it to us, rather than use it, even though I thought 6 ETH was a bit steep for a vulnerability that “merely” caused downtime. I was disappointed to discover that he was a common scammer, but I’m also relieved that the DDoS mitigations I added work.

As always, if you have any comments or feedback, please Tweet or toot at me, or email me directly.