Troubleshooting Syspeace

An interesting support case came to our attention recently.

A customer claimed that Syspeace wouldn’t block according to the rules.

The bruteforce attacks would continue , even after they should have been blocked.

We checked the ususal culprits (verify that the .Net is fully patched, that the customer is running the latest Syspeace version, verify that logging is enabled and that the firewall is turned on )

The rules were added as expected in the firewall but they didn’t have any effect.

After a lot of troubleshooting the root-cause was found.

The customers server did indeed have the firewall enabled but only in one of the firewall profiles (public, private, domain) and unfortuantely, the network used was not the one the firewall was enabled for, hence, nothing was blocked as expected. The rules were added but did not take effect in the expected amount of time

So, as a general troubleshooting tip , check how your firewall is enabled and verify that it indeed is the correct network profile in there, or, enable the firewall for all three profiles.

The usual troubleshooting tips we give are described in the manual in the troubleshooting section

1. Make sure you’ve enabled the firewall (as described in Firewall), firewall enabled, prefferably on all profiles.

2. Make sure you’ve enabled the auditing (as described in Windows login detection prerequisites).

3. Verify that the server can reach https://s.syspeace.com/ping . (You should see a message saying Hello from Stockholm. and the local time of the server and recommended Syspeace version)

4. In some instances, when running Terminal Server or Remote Desktop Services there’s actually the scenario where the Windows server itself fails to obtain the source IP address of the login attempt (you can verify this by checking the Windows event log and look for Source Network Address: ) Sometimes, that entry is empty, thus disabling Syspeace from actually having anything to block. Syspeace will attempt to corroborate the IP address from some other logs. If it doesn’t find any, there is not much that Syspeace can do. (Update: starting with Syspeace 2.7, these attempts can be detected too.)

5. In any applicable firewall or antivirus software, allow Syspeace access to https://s.syspeace.com/ (port 443).

6. Verify any proxy settings, if applicable.

7. Some methods of Windows authentication actually attempts to log in several times. Two failures may be part of one log in attempt. Syspeace has no way of knowing how many attempts were intended and has to work with the actual failures. Due to counting failures instead of attempts, rules may be triggered seemingly ahead of time.

8. One way of quickly verifying functionality is to use a workstation (not whitelisted) and attack your server with the net use command from the command prompt. After the number of tries defined in the current rules, the workstation should be blocked from communicating with the server. Example of the command: net use * \server name or server IP addressanyshare /user:syspeacetester “anypassword”

9. If you want to submit logs to us, start Syspeace, go to Management → System settings, enable logging and start the service. The log file is created in a subfolder of the Syspeace installation folder.

10. When submitting logs,
Please create a .zip file of the logfiles, include any relevant information from Windows Eventlogs (application, system and security and when applicapble, the Syspeace eventlog ) and also create a .Zip-file of the database and email them directly to the devteam . The email address can be found in the manual

11. If your server doesn’t pick up the source IP address in your eventlog , please have a look a this blog article

12. If your database has grown above the size limit of 4 GB, in the current version ( 2.5.2) you will have to manually delete the database and set up your Syspeace again. Please refer to this post on the matter
by Juha Jurvanen

#msexchange Brute force attacks prevention on #Webmail #OWA with #Syspeace #hacking #security

Preventing brute force attacks against Microsoft Exchange Server and OWA Webmail

If you’re running Microsoft Exchange Server your also quite likely to have the Microsoft Exchange OWA (Webmail)
interface up & running to enable your users to use Activesync and access their email, calendars and contacts
over an easy-to-use web interface accessible over the Internet. This is just as relevant if you’re managing your
own Exchange Server or if it is a hosted Exchange at a service provider. If your provider doesn’t have a
solution for this, you may find yourself in a very difficult situation one day as explained further down.
Since the Exchange Webmail (OWA) is reachable and visible over the Internet, this of course also means that
anyone is able to try to log in to your Exchange server over the same OWA interface. They may not succeed to
login but they may try to overload your server by sending lots of login request or have your users undergo a
Denial of Service attack (a DoS attack).

Brute force attacks used as Denial of Service attacks

The OWA in itself (or does Windows Server for that matter) doesn’t have any brute force prevention mechanisms
built into it but the actual user validation is done within the Active Directory infrastructure by your domain
controller(s). Within the Microsoft line of products this is actually true for most of them such as Terminal
Server (RDS, Remote Desktop), Sharepoint, SQL Server and so on and also for Citrix since user validation is done
in the same way.
If you have for instance set up Account Lockout Policies to disable a user account after 5 failed attempts ,
anyone with knowledge of your name standards (email addrees, AD login) can basically run a script against the
server using a specicif username (or hundreds of them) and deliberatley usoing wrong passwords, thus locking the
legitimate users account and disabling them from loging in at all (in essence, they can’t even login to anything
that uses the Active Directory validation, not even their own workstations in the Office)
If such an attack is made from a single IP address, it is fairly easy to block it manually (simply block the
attack in either the external firewall or the local firewall of the Exchange server).
In reality though, this is not how such an attack occurs. Should someone really want to disrupt ypur services,
they will do this from hundreds or thousands computers at the same time and making it impossible to block
manually.

Using Syspeace as a countermeasure

With Syspeace , this is all taken care of automatically. Syspeace monitors the Windows Serevr logs for failed
login requests and if an IP address tries to login against your servers ( Exchange, Terminals Server and so on)
and fails for instance 5 times within half an hour, the IP address is automatically blocked from communicating
at all with the affected server on any level (so if you’re also running other services , they will not be able
to target them either once blocked).
Each attack is blocked, traced and reported via email that contains the source IP address, the username used,
country of origin and previous attacks from the same IP address.

Here is actually an example of how the email notification looks like (with IP address and domain name intentionally removed)

Blocked address *.*.*.* (ip-*-*-*-*.*.secureserver.net) [United States] 2015-01-14 18:45:00 Rule used (Winlogon):
	Name:			Catch All Login
	Trigger window:		4.00:30:00
	Occurrences:		5
	Lockout time:		02:00:00
	Previous observations of this IP address:
	2015-01-14 16:44:50	****lab
	2015-01-14 16:44:52	****labroator
	2015-01-14 12:53:44	****ron
	2015-01-14 12:53:46	****demo
	2015-01-14 12:53:48	****canon

Syspeace also delivers daily and weekly reports of blocked threats.

Within Syspeace, there is also reporting tools for access reports, a Global Blacklist for infamous offenders and
much more.

Installing and setting ups Syspeace

Setting up Syspeace is very easy and only takes a couple of minutes, without the need for changing your
infrastructure or bying very expensive dedicated hardware. Most likely , you will not even need to hire a
consultant for it.

Syspeace runs as Windows Service and support a variety of Windows Servers such as Terminal Server, Exchange Server, Sharepointm Windows Serevr 2003 to Windows Serevr 2012 R2 and more and it starts detecting brute force attacks immediately after you set it up and press the start button.

Please download a free, fully functional 30 day trial and see for yourself how a very big problem can be very easily solved.
Should you decide to keep using Syspeace, the licensing cost is equivalent to an antivirus product and the
licensing model is highly flexible, enabling you to decide for yourself ofor how long you wish to run Syspeace.

Syspeace, SHA-2 certificates and Windows Server 2003

Recently, the SSL certificate used for syspeace.com, the Syspeace Licenses site as well as backend Syspeace services was reissued with a signature using the SHA-2 hash algorithm.

The SHA-2 hash algorithm replaces the earlier, deprecated SHA-1 and moving forward is recommended by the CA Security Council.

However, some users on Windows Server 2003 have seen issues using the new certificates, due to Windows Server 2003 as shipped not being able to work with SHA-2 certificates. For this reason, we are reissuing our SSL certificate, now again using the SHA-1 hash algorithm.

We intend to once again move to SHA-2 when it is feasible to do so.

#infosec How to block an ongoing dictionary attack / brute force attack against Windows Servers, #MSexchange and more

How to block an intrusion attack against Windows Servers for free

If your server or data center is targeted by a brute force attack a.k.a dictionary attacks , it might be hard to figure out how to quickly make it stop.
If the attack is from a single IP address you’d probably block it in your external firewall or the Windows Server firewall and after that start tracking and reporting the attack to see if needs following up.
However, if the attacks is triggered from hundreds or even thousands of IP addresses, it will become basically impossible to block all of them in the firewall so you need something to help you automate the task.

This is where Syspeace comes into play.

Fully functional, free trial for brute force prevention

Since Syspeace has a fully functional trial for 30 days, you can simply download it here, install, register with a valid email address, enter the license key into the Syspeace GUI and the attack will be automatically handled (blocked, tracked and reported) as soon as the Syspeace service starts up.

In essence, the attack will be blocked within minutes from even connecting to your server.

The entire process of downloading, installing and registering ususally only takes a few minutes and since Syspeace is a Windows service it will also automatically start if the server is rebooted.

If the attack is triggered to use just a few login attempts per attacking IP address and for a longer period of time in between attempts, I’d suggest you change te default rule to monitor for failed logins for a longer triggerwindow , for example 4 days so you’d also automatically detect hacking attempts that are trying to stay under the radar for countermeasure such as Syspeace.

The Syspeace Global BlackList

Since Syspeace has already blocked over 3.6 Million attacks worldwide , we’ve also got a Global Blacklist that is automatically downloaded to all other Syspeace clients.

This means that if an IP address has been deemed a repeat offender (meaning that it has attacked X number of Syspeace customers and Y number of servers within Z amount of tme), the attackers IP address is quite likely to already be in the GBL and therefore it will be automatically blacklisted on all Syspeace-installations, thus making it preemptively blocked.

Syspeace does not simmply disable the login for the attacker, it completely blocks the attacker on all ports from communicating with your server so if you’ve got otther services also running on the server (such as an FTP or SQL Server) the attacker will not be able to reach any if those services either. The lockdown is on all TCP ports.

More Syspeace features, supported Windows Server editions and other services such as Exchange Server, Terminal Server, SQL Server …

You will also get tracking and reporting included immediately for future reference or forensics.
Syspeace supports Windows Server editions from Windows 2003 and upwards, including the Small Business Server editions. It also supports Terminal Server (RDS) and RemoteAPP and RDWeb, Microsoft Exchange Serevr including the webmail (OWA) , Citrix, Sharepoint,
SQL Server and we’ve also released public APIs to use with various weblogins. All of this is included in Syspeace. Out of the box.
We’ve got a IIS FTP server detector in beta and also a FileZilla FTP Server detector and we’re constantly developing new detectors for various server software.

Download and try out Syspeace completely free

Even if you’re not being attacked by a large brute force attack right now, you can still download the trial and have Syspeace handle attacks for you in the background. Who knows, there could be more invalid login attemtpts than you think, such as disabled or removed users that have left the company or very subtle, slow dictioanry attacks going on in the background that actaully might be quite tricky to spot if your not  constantly monitoring logfles.

On this blog, http://syspeace.wordpress.com, we’ve written a lot of blog articles on how Syspeace works and a lot of other articles regarding securing your servers that we hope you’ll find useful.

How to battle slowgrind #bruteforce attacks against #msexchange #windows server #remotedesktop #sharepoint with #Syspeace

Syspeace automatically blocks attacks that occur according to the rules.
The default rule is that if an intruder fails to login more than 5 times within 30 minutes, the intruders IP address is blocked, tracked and reported for 2 hours and simply is denied any access to the server.

A new trend though has emerged and that is for bruteforce attackers to “slowgrind” through servers, trying to stay “under the radar” really from IDS/IPS HIPS/HIDS such as Syspeace.
They’ve got thousands and thousands of computers at their disposal so they’ll basically just try a few times at each server and then move on to next one in the IP range or geographical location hoping not to trigger any alarms or hacker countermeasures in place.

An easy way to battle this is actually simply to change the default rule in Syspeace from the time windows of 30 minutes to for example 5 days.

This way , I’m pretty sure you’ll see there are quite a few attackers that only tried 2 or three times a couple of days ago and they’re back again but still only trying only a few times.

With the “5 day” windows, you’ll catch and block those attacks too.

Here’s actually a brilliant example of an attack blocked, using a 4 day window.

Blocked address 121.31.114.99() [China] 2014-08-11 15:06:00
Rule used (Winlogon):
        Name:                   Catch All Login
        Trigger window:         4.00:30:00
        Occurrences:            5
        Lockout time:           02:00:00
        Previous observations of this IP address:
        2014-08-11 13:05:51     aksabadministrator
        2014-08-10 22:06:48     aksabadministrator
        2014-08-10 06:39:12     aksabadministrator
        2014-08-09 15:39:52     aksabadministrator
        2014-08-09 00:32:05     aksabadministrator

Syspeace has blocked more than 3 285 300 intrusion attempts against Windows Servers worldwide so far.

#Syspeace stops due to license server inaccessable on #Windows Server 2003 #infosec

Syspeace service stops due to license server not reachable / inaccessibility on Windows Server 2003

We’ll actually update the troubleshooting section with info for Windows 2003 Servers but here’s why this can occur.

Apparently root certificates are not automatically updated on Windows Server 2003:

http://support.microsoft.com/kb/931125

The automatic root update mechanism is enabled on Windows Server 2008 and later versions, but not on Windows Server 2003. Windows Server 2003 supports the automatic root update mechanism only partly. (This is the same as the support on Windows XP.) And because the root update package is intended for Windows XP client SKUs only, it is not intended for Windows Server SKUs. However, the root update package may be downloaded and installed on Windows Server SKUs, subject to the following restrictions.

If you install the root update package on Windows Server SKUs, you may exceed the limit for how many root certificates that Schannel can handle when reporting the list of roots to clients in a TLS or SSL handshake, as the number of root certificates distributed in the root update package exceeds that limit. When you update root certificates, the list of trusted CAs grows significantly and may become too long. The list is then truncated and may cause problems with authorization. This behavior may also cause Schannel event ID 36885. In Windows Server 2003, the issuer list cannot be greater than 0x3000.

This can be resolved for Syspeace by manually installing the gd-class2-root.crt certificate from this page: https://certs.godaddy.com/anonymous/repository.pki

#infosec #Syspeace for intrusion prevention for #windowsserver instead of specific applications or services such as #FileZilla FTP Server or #WordPress

Syspeace for intrusion prevention for the entire server instead of specific applications or services such as FileZilla Server

If you’re managing a server and host various applications and services all of them are reachable for your users and and customers but most likely, and quite often, they’re also reachable for others to try to log in.

To be cost effective, you could be using using a Terminal Server (or Remote desktop Server) and you’ve also got for instance a FileZilla FTP Server to ease file transfers (or the Microsoft IIS FTP server, my hunch is that these two are the most common ones if you’re running a Windows Server environment) and there’s a web interface for the remote applications and so on . There might also be other services on the same server/servers.

Built in intrusion prevention in applications or Windows Server

Some software actually have brute force prevention built into them, such as the FileZilla FTP Server (although, keep in mind that is it not enabled by default) and there could be other software installed that have intrusion prevention built into them. Not within Windows Server though and there are quite a few articles on this blog explaining how it works such as this one about securing your Exchange OWA

An atacker will first portscan your server, search for open ports and try to figure out what services and applications you’re running on them. Even if you’ve changed the default ports, quite often the application will actually reveal itself in the header what it is and what version it is.

You can for instance simply do a telnet session to the port in question and see what your applications actually reveal about themselves.
Simply start a telnet client and connect to the port you’re interested in such as port 25 for SMTP (email) or port 21 for FTP and you’d probably get at least some information on what is running on the server. To gather more detailed and complex information, you probably be using software like nmap.

After that, tbey’ll simply use automated scripts to try and login. If there is a block in some way on for instance FileZilla FTP Server they’ll simply move on to the next port/service , like the RDWeb interface for Remote Desktop and RemoteAPP services and continue the attack since they’d only been blocked on the FTP level so far (usually port 21) Here’s a >previous article describing parts of the anatomy in a hacking attack written by Juha Jurvanen.

If you’re hosting a multiple software and srevices on a server and each of them have brute force prevention builtin , they’ll only block the attack within their own part of the system.
FileZilla will block the brute force on FTP but nothing else.

Using Syspeace as your HIPS , Host intrusion Prevention System for Windows Servers

A key difference using Syspeace as a HIPS (Host Intrusion Prevention System) is that it will block the attacker entirely on all ports if they trigger any of the detectors, rendering the attacker unable to communicate at all with your server on any port (even ping), thus automatically protecting any other service you have running on it.

To illustrate this with something in the “real” world.
If you’ve got a house with multiple doors, the attacker would first try their keycard/key in one of the doors to try to gain access into the house until an alarm is triggered and they would have to move on, but only for that specific door.
After that they’d keep using the keycard/key on the next door and so on.
With Syspeace, they’d only be able to use the keycard on the first door until the alarm is triggered and after that they would be automatically blocked from even trying to use the keycard on any of the other doors since the doors would have “magically” disappeared for them and would be out of reach for them. It would be as if the actual building itself would have disappeared for them.

Download a fully functional, free Syspeace trial for intrusion prevention or even if you’re under attack of a brute force or dictionary attack

Have a look at the Syspeace website and try the fully functional trial for it and see how it can help you to easily and quickly brute force protect your server. We’ve had users downnloading Syspeace and implementing it in minutes during a dictionary attack to have Syspeace automatically deal with it and to block, trace and report the attack. Since the trial is fully functional and free and it only takes a few minutes to set it up, it can be an easy solution to handle an ongoing attack.

Syspeace supports Windows Server 2003 and on (including the Windows Server Small Business versions), SQL Server, Remote Desktop, Exchange Server, Sharepoint, Exchange OWA, RDWeb , Citrix and more. Out of the box. It actually also support Windows 7 and Windows 8 but please refer to his article on when Syspeace is actually useful for you and when it’s not.

Syspeace has blocked more than 3 126 500 brute force and dictionary attackas targetaed agains Windows Servers worldwide.

The Syspeace team has also developed a FileZilla FTP Detector that is in beta and also an Microsoft IIS FTP detector.
We’ve also released a detector for selfhosted WordPress and we’ve released the Syspeace API for .PHP and .NET to enable our users to develop their own intrusion prevention for applications instead of being forced to develop protection into applications themselves from scratch.
The Syspeace API can also be used to protect spcific websites if you’re hostng multiple websites.

#infosec Syspeace support for #FTP on #IIS and #Filezilla in beta

This is just a short newsflash that the Syspeace devteam has been working on adding detectors for #Microsoft #IIS FTP server and for #Filezilla FTP server.

Using the Syspeace engine to prevent bruteforce attacks against #windowsserver #msexhange #Sharepoint #remotedesktop #Citrix has proven to be highly efficient and the need for more detectors grows steadily the more users we get.

We’ve blocked,tracked and reported over 3 Million #bruteforce and #dictionary attacks against Windows Servers worldwide so far.

We have a constant dialogue with Syspeace users over mail or Uservoice to see what new detectors our users need and one of the most frequently asked for is FTP support.

If you have ideas for new features or detectors, please join us at Uservoice or drop us an email.

We’ve already publically released the Syspeace API to enable you to write your own webapplication detectors and have Syspeace handle bruteforce attacks for you.
For more information on how to do this, please refer to the Syspeace Detector API page .

#infosec #Syspeace has blocked over 3 Million #bruteforce attacks

Today, we reached a new milestone for Syspeace. We have now blocked, tracked and reported over 3 Million #bruteforce attacks against #windowsserver #msexhange #Sharepoint #remotedesktop #Citrix #SQLserver worldwide!

#infosec #security About using #Syspeace against #DDoS attacks for #sysadmin

Syspeace and DDoS attacks

We had a discussion the other day about Syspeace and if it would help in a DDoS attack.

Essentially a DDoS attack is about overloading a server with massive traffic thus making it unreachable for the services the way it is supposed to be.

This can be accomplished in numerous ways.

If for instance 10 000 computers in a botnet are targeted at downloading a specific image or file from a public website without a login, Syspeace would not be the tool for you. Not at the moment anyway. Syspeace is designed to monitor failed login attempts and handle them by custom rules to protect your Windows servers by completely blocking the attacking address in the local firewall. This will protect your server on all ports soo if you other services running on it, they would also be blocked for the attacker.

DOS/DDoS by using Brute force / dictionary attacks and how Syspeace would react

The two different methods in the brute force/dictioanry attack department would be the following.

Single login attempt method

If the same 10 000 copmuters try to login to your server (an Exchange weblogin, RDS/ Terminal Server, Sharepoint, Citrix and so on ) with a brute force / dictionary attack the server would stop responding due to the overload on CPU/RAM and the network would also be filled.

If each and one of these 10 000 computers only tries once to login , Syspeace wouldn’t react since that would esseantially mean that all logins (or IP addresses essentialy) would be blocked at the first thus disabling anyone to login.

If you’re a hosting provider or outsouring provider and you have a number of customers at static IP addresses you could whitelist the customers IP addresses and set up a Syspeace rule to block at one failed login and in that manner have the attacka partially handled by Syspeace.
However, if you’re a Cloud Service provier this won’t work in reality since your customers could be coming from any IP address anywhere.

Multiple login attempt method

The second method would be to have each and everyone of these 10 000 computers constantly trying to login multiple times and such an attack would be blocked by Syspeace.

Bare in mind though, this would not sort out the network being flooded but it would help you protect your server from crashing due to overloaded CPU/RAM usage and it would buy you time to contact your ISP and see if they can help you mitigate the attack (with specific tools or increasing your bandwidth for instance)

To a certain extent , the Syspeace Global Blacklist would probably also have you preemptively protected against some of the IP addresses attacking you already.

If you don’t have Syspeace at all it’s not unlikely you’ll also be having a lot of user accounts locked out if you you’re trying to use lockout policies. Here’s a previous blogpost on why that is

Future features in Syspeace

One of the things we’ve already released are public APIs for customers with their own applications, webapplications and loginforms so we enable them to use the Syspeace engine to easily handle brute force attacks. For more information on how to implement it on your website or appliaction , please refer to the Syspeace Detector API page

We do have some ideas on how also to have Syspeace help in the first scenario (1 login/computer attack) but we’ll get back to you on that after we’ve implemented quite a few new more features and functions that’s already in our roadmap.

To have your Windows servers protected against malicious login attempts and have it set up in minutes without changing your infrasctructure , please visit the Syspeace download page

By Juha Jurvanen