Various brute force prevention methods for Windows servers – pros and cons

Intro on brute force prevention tactics and some misconceptions

Protection from brute force attempts on Windows servers has always been a nightmare and would continue to be so if not… Yes, I admit, I will come up with a solution further down.

Most system administrators with self-respect start off with the best of intentions to actually keep track of brute force attempts but eventually give up because of the sheer number of attacks that occur daily.

Others, unfortunately, believe that a firewall takes care of the problem which it doesn’t, or that an account lockout policy is an answer. Neither of them is and I’ll show you why.

The firewall approach

Think about it. What does a firewall actually do? The role of the firewall is to block traffic on unwanted ports and to drop portscans and various SYN FLOOD attacks. That’s about it. A firewall is basically a harsh doorman deciding who gets in to speak with the guys on the inside and who doesn’t.

If an attacker actually connects on a valid port, the traffic is redirected/port forwarded to the server in question let’s say the webmail interface of a Microsoft Exchange Server or a Microsoft Windows Terminal Server or a Citrix Server. Once the attacker is there, the actual login request is handled by the server, not the firewall. The login process is managed by the Windows Authentication process (which in turn may be validated against Active Directory or a local user database using SAM). The firewall is already out of the picture since it has no connection with the Windows server apart from the TCP connection and keeping it alive really. They don’t communicate the result of the logon process with each other.

Also, changing from standard ports won’t help you much, will it? The login process is still managed by the Windows Server although you will get rid of a lot of portscans and “lazy background, script kiddie attempts” if you’re using non-standard ports. Basically, you get rid of the script kiddies, but the problem isn’t solved. The traffic is still redirected/port forwarded to the server that does the actual authentication.

Using for instance a Remote Desktop Gateway won’t handle the problem either. Using an RDP Gateway minimizes the attack surface, yes, but it is still reachable and the user logins still have to be validated. The problem is with any server that services logon requests basically, regardless of what ports and how they get there. That is Microsoft Windows server, Exchange Server, Citrix, Sharepoint, CRM, Terminal server and so on. The list can probably go on and on.

There’s also the risk that stuff stops working each time you apply some updates or patches to your Windows Servers if you start changing standard ports or standard configurations. It’s happened to me a few times and it’s not that amusing, to be honest when you’ve got 1000 users not being able to log in because you’ve just done your job and patched the servers to keep people’s data safe. Trust me, that’s not a good Monday morning.

The VPN approach

Yes. That’s a safer approach but also here we do have some issues. First of all, it’s not that easy to keep track of VPN certificates, set all of them up and manage all the licensing costs (that can be quite significant really ) and (sometimes costly) hardware you need to have in place. Historically there have also always been performance issues with most VPN solutions since all traffic is directed through one or a few VPN servers / connectors. Some of them also charge you for the bandwidth you want it to be able to use for VPN connections or charge you for the number of simultaneous VPN connections. A VPN solution can be quite costly as an initial investment and taking into account all of the administration involved in it.

You also probably won’t be demanding your users to have a VPN connection to the Microsoft Exchange OWA either since the whole idea of the OWA i that it’s supposed to be easy to reach from anywhere. I know there are some companies actually requiring VPN even for OWA and that’s just fine I guess but the more we’re moving our data and applications to cloud services, this hassle with different VPNs and stuff will eventually fade into the dark corners of the Internet (that’s my personal belief anyways). The thing is that your users don’t want to be tied down by complicated VPN clients and stuff. Users nowadays are used to “stuff just working”, and it has to be easy and intuitive for them. The days of the “System Administrators from Hell” implementing all kinds of complex solutions to keep stuff secure and forcing users to have very specific and complex ways of accessing data are over. They were good times, good times but they’re over. Deal with it.

The IDS/IPS approach

Using a centralized IDS/IPS This is a more efficient method, yes. The downside is, that most of these systems require you to change your infrastructure and get specific, costly hardware, licenses and costly consultants to get it up and running. And someone needs to monitor it, take care of it, and so on. There are parallels to the VPN approach here although an IDS/IPS does a whole lot more such as examining all the network traffic, examining it for malicious code and so on. I’m not sure actually if an IDS/IPS can communicate with the Windows Server Authentication Process so I actually won’t say anything about that. I would presume they can, otherwise, I fail to see the point (from the brute force login perspective, that is) and you’d still need to handle the logon attempt on the Windows server.

The Account Lockout Policy approach

The account lockout method is also flawed due to the fact that an attacker can quite easily cause a DOS (Denial of Service) simply by hammering your server with an invalid login request but with valid usernames, thus rendering the accounts unusable for valid users. Basically, all he (or she)  needs to know is the user logon name and in many systems, it’s not that hard to guess (try the CompanynameUsername or the mail address for the user since it’s quite often also a valid login name if you have a look at the properties of the user in Active Directory Users and Group snap-in)

The Cloud Computing approach

We are shifting more and more of our data and applications into various Cloud Services (like it or not but, it’s a fact and you know it). This way we do get rid of some of these problems on our own servers. Hopefully, your Cloud Service provider actually has a plan for these scenarios and has the necessary surveillance software and systems in place. If you’re using a Cloud Computing platform based on Windows Servers, you should actually ask your provider how they handle brute force attempts on their servers. Most likely they will give you one or more of the scenarios described above and, as I’ve shown you, they are not adequate to handle the task at hand. They’re just not up for the job. Feel free to ask your own provider and see what answer you get. My guess is .. mumbo jumbo, but basically, they don’t have anything in place really, more or less.
You could even try logging into your own account with your own username but the wrong password loads of times and see what happens. Will it be locked out? Will your machine be locked out? How does your Cloud Service Provider respond and are you informed in any way that an intrusion attempt has been made using your account? How many times can anyone try to access your account without you being notified of it? And from where are they trying to get to your data and why?
Personally, I know of only one Cloud Service Provider that has also taken these questions into account, and that’s Red Cloud IT in Sweden.

Is there a solution then?

Yeah. I told you so in the beginning and even if choose not to use what I suggest, I highly recommend that you start thinking about these things properly because these problems will accelerate in the future. Just take a look at all the hacktivism with DDOS attacks going on out there. It’s just a start because the Internet is still young.

First of all, and this is extremely important you realize, that it doesn’t matter if you hosting your own servers or if you’re using VPS (Virtual Private Servers) hosted somewhere else or even if you’re a Cloud Service Provider. The basic principle stands: if you are providing any kind of service to users using the Windows Authentication mechanism you should be reading this, and hopefully my point has come across.

If you’re having brute force attacks on your Windows systems today and I’m pretty sure you do (just turn on login auditing and I’m sure you’ll see you have more than you actually thought you did, *for some odd reason this is NOT turned on by default in Windows*) there’s a few things you should be doing (that I’m guessing you’re not because you’re not a cyborg and you need to sleep, meet your friends and family and actually be doing something productive during your work hours). On the other hand, if you are doing all of these things I’m guessing you have quite a large IT staff with a lot of time on their hands. Good for you. Call me and I’ll apply for a position.

First of all. Block the attack.

You need the attack to stop! Instantly. This is of course your first priority That’s basically blocking it in the firewall, either in the local Windows firewall or the external one. It’s actually up to you which way is the easiest one. The reason is that you don’t want to be wasting CPU and RAM and bandwidth on these people (or botnets)  and of course, you don’t want them to actually succeed in logging on (should you have a lousy password policy in place ) or even them disguising a real intrusion attempt behind a DDOS attack to fill your logfiles and hide in there. (Yes, it’s not an uncommon method). There are also quite a few reports of DDOS attacks being used to disguise the actual reason for it which is to find out what security measures are in place for future reference. The “know your enemy principal”.

Second. Trace the attack. From where did it come?

Second, you need to find out from where the attack originated and what username was used. This is because you want to know if it is a competitor trying to hack you and access your corporate data or if you find yourself in the interesting position of your own username trying to log in from sunny Brazil and you’re just not in Brazil (although you’d love to be). You’re in Chicago looking at winter. Something’s up.
You also want to see if it’s a former employee trying to log on and so on… This is stuff you need to know and keep track of since there may be legal issues involved further down the line.

Points one and two, you want to be handled in real-time. There’s no use for you to find out two days after the attack that something actually happened. You want it stopped, reported and handled as it happens.

The legal stuff.

Third, you need to decide what to do with your information. Should it be handed over to the legal department, your boss, or the police, or is it just “nothing” and can be discarded?

So. “What would you suggest as a solution then”? 

The easiest and most cost-efficient way to handle brute force attacks on Windows servers is to have an automated system to block, track and report each attack and that’s where Syspeace comes into play.

Syspeace is a locally installed Windows service, thus using a minimum of system resources,  that monitors the server for unwanted login attempts and blocks the intruders in real-time in the local firewall based on the rules you’ve set up. For instance “if this IP address has failed logging on 20 times during the last 30 minutes then block it completely for 5 hours and send me an email about it”

This means that you can for instance set up a blocking rule that is your “Account lockout policy – 1” in your rules and that way simply block the brute force attack but not lock your users’ accounts and cause them unnecessary disruption.

Since Syspeace monitors the Windows Authentication login process, it doesn’t matter what firewall you are using or what ports you’re using. The monitoring and blocking are done where the actual login attempts are made and therefore caught and handled automatically.

Once the intruder’s IP address is blocked, it’s blocked on ALL ports from that server. This means that if you have other services also running on it (like FTP or well… anything really) those ports and services are also protected instantly from the attacker. Not giving them the chance to find other ways of gaining access to that server through exploits.