|
Last blog posts
Login |
Blog: Network Security
Created by TaneliOtala on Thu 11 of May, 2006 [18:03 UTC]
Last modified Thu 19 of Nov, 2009 [07:25 UTC]
Description: NetSec, or Network Security -- observations, tips, tricks for small to medium sized networks
denyhosts -- almost best thing since sliced bread
Denyhosts
Almost too good to be true... What's good, is that it allows you to get almost rid of dictionary attacks on your SSH port... What's not so good, is that as of recent, the dictionary attacks on POP3, IMAP4, TELNET, FTP are significantly on the rise... and Denyhosts does not make it particularly easy to block the other protocols... If I manage to get the other protocols blocked, I'll publish the regex'es for those. Meanwhile, if you notice it in your log files, rememember that: iptables -A INPUT -s x.x.x.x -j DROP always works....
Using Pion for network security
Pion, by Atomic Labs can do some pretty incredible tricks on detecting anomalies, hacking attempts, etc. by passively looking at the network traffic.
Imagine, being able to program (graphically, no program code) any kind of web/smtp/voip event, and being able to take action real-time. Check out http://atomiclabs.com — I will write actual recipes soon.
Russian (caravan.ru) link spammers
Tiki link spammersSo tired... seriously, first email spam (BTW, how about a presidential candidate that has capital punishment for spammers?) and now trackback link spammers. There is this russian network, caravan.ru, that is relentlessly pounding on my dozen or so tikiwiki sites, adding a trackback link spam every few second. Here is what it looks like in your logs: 217.23.147.210 - - 03/Sep/2007:08:53:10 -0700 "POST /tiki-view_blog_post.php/1/1 HTTP/1.1" 200 - "-" "WordPress/1.9" 212.24.48.34 - - 03/Sep/2007:08:53:16 -0700 "POST /tiki-view_blog_post.php/1/2 HTTP/1.1" 200 - "-" "WordPress/2.0" 217.23.143.226 - - 03/Sep/2007:08:53:31 -0700 "POST /tiki-view_blog_post.php/1/2 HTTP/1.1" 200 - "-" "WordPress/2.0" 217.23.133.242 - - 03/Sep/2007:08:53:39 -0700 "POST /tiki-view_blog_post.php/1/5 HTTP/1.1" 200 - "-" "WordPress/1.9" 217.23.151.130 - - 03/Sep/2007:08:53:45 -0700 "POST /tiki-view_blog_post.php/1/5 HTTP/1.1" 200 - "-" "WordPress/1.9" 217.23.143.224 - - 03/Sep/2007:08:53:50 -0700 "POST /tiki-view_blog_post.php/3/19 HTTP/1.1" 200 - "-" "WordPress/2.1.2" 217.23.132.114 - - 03/Sep/2007:08:53:54 -0700 "POST /tiki-view_blog_post.php/3/19 HTTP/1.1" 200 - "-" "WordPress/2.1.2" 212.24.48.31 - - 03/Sep/2007:08:53:58 -0700 "POST /tiki-view_blog_post.php/2/7 HTTP/1.1" 200 - "-" "WordPress/2.0" 217.23.143.26 - - 03/Sep/2007:08:54:02 -0700 "POST /tiki-view_blog_post.php/2/7 HTTP/1.1" 200 - "-" "WordPress/2.0" 212.24.48.52 - - 03/Sep/2007:08:54:04 -0700 "POST /tiki-view_blog_post.php/3/9 HTTP/1.1" 200 - "-" "WordPress/2.1.2" Let's see, what are the options?
Well, let's just first plain block the caravan.ru network out: iptables -A INPUT -s 212.24.48.170/24 -p tcp --dport www -j droplog iptables -A INPUT -s 81.176.0.0/15 -p tcp --dport www -j droplog iptables -A INPUT -s 62.213.64.0/18 -p tcp --dport www -j droplog iptables -A INPUT -s 212.158.160.0/20 -p tcp --dport www -j droplog iptables -A INPUT -s 217.23.128.0/19 -p tcp --dport www -j droplog iptables -A INPUT -s 212.24.32.0/19 -p tcp --dport www -j droplog iptables -A INPUT -s 85.255.118.92/24 -p tcp --dport www -j droplog iptables -A INPUT -s 72.232.191.50 -p tcp --dport www -j droplog
SPAM on the rise...
Is it just me, or have others noticed that SPAM is on the rise?
It is really annoying when I get 100 SPAMs per day on the email account I use on my cell phone... Time to "tighten the screws" again! Links:
So, I go through my sendmail.mc configuration, and notice that I could add a few blacklists again. FEATURE(dnsbl, `relays.ordb.org', `Rejected -- see http://ordb.org/ for reason')dnl FEATURE(dnsbl, `sbl-xbl.spamhaus.org', `Rejected -- see http://www.spamhaus.org/SBL for reason')dnl FEATURE(dnsbl, `bl.spamcop.net', `Rejected -- see http://spamcop.net for reason')dnl FEATURE(dnsbl, `dnsbl.sorbs.net', `554 Rejected see http://dnsbl.sorbs.net')dnl FEATURE(dnsbl, `list.dsbl.org', `Rejected -- see http://dsbl.org for reason')dnl FEATURE(dnsbl, `block.rhs.mailpolice.com', `Rejected -- see http://rhs.mailpolice.com')dnl FEATURE(dnsbl, `cbl.abuseat.org', `Rejected -- see http://cbl.abuseat.org')dnl FEATURE(dnsbl, `l1.spews.dnsbl.sorbs.net', `Rejected -- see http://spews.org')dnl Basically I ended adding dnsbl.sorbs.net and l1.spews.dnsbl.sorbs.net Results are good:
Good work for a quiet Sunday morning, though I wish I could find a good milter in open source, to do better — and not rely so much on blacklists.
Orienting WiFi antenna without help?
I keep adding new Internet links to my house system... some as real links with traffic shaping, load balancing, etc.
...Some, just as backup links. I confess it has gotten a little bit out of hand, having just added the sixth link. So, MetroFi in Sunnyvale, CA started offering a free city-wide WiFi connectivity since December 2005 — and I just could not resist it. I got a handy +8dBi directional antenna (by Airlink), a 7ft antenna cable, and a PCI WiFi card... a little hole into the wall, hang the directional antenna right underneath the solar panels, point to the WiFi link on the lamp post... Now the problem, how do you orient the antenna? And, as a geek, can you do it with a one-liner? watch "iwlist eth1 scan 2>/dev/null|grep -o \"Signal level=[0-9-]* dBm\"|head -n 1 | festival --tts"
Then I just go outside, I can hear the dBm readings loud and clear, so all that's left is a little bit of patience while getting the antenna to point in the right direction. Note, that there is a little bit of lag time on the card scanning the APs — you should get two or three readings between re-pointing the antenna
Collecting all SSH RSA keys, and redistributing them
Here's a handy script for those who like to create a revocable, per-user trust system...
This script will reach into all of your boxes, where you already run ssh (and have already run ssh-keygen). It will retrieve all of the .ssh/id_rsa.pub keys, concatenate them into a single file, and deliver that file to all the hosts as the .ssh/authorized_keys file. Presto, all your hosts now share the same set of RSA keys, and you don't need to type passwords again. If one of the machines gets compromised, you can revoke the key and redistribute easily again. NOTE: There are much better ways of doing this — but this is simple, quick, and efficient... # List your hosts (hostname, or FQDN) separated with spaces
HOSTS="hewey dewey lewey"
# Base file name
KEYFILE=authorized_keys
# Wipe out old temp file
rm $KEYFILE.new
# Collect (concatenate) public RSA keys from all hosts
# If you have errors, you may either be lacking the key -- use "ssh-keygen -t rsa"
# Of you might have a key that has changed (remove the offending key)
for i in $HOSTS ; do {
echo "Host: $i"
ssh $i cat .ssh/id_rsa.pub >>$KEYFILE.new
} ; done
chmod go-rw $KEYFILE.new
# Push newly built collection of all keys into all hosts
for i in $HOSTS ; do {
echo "Host: $i"
scp -p $KEYFILE.new $i:.ssh/$KEYFILE
} ; done
Dynamic Firewall, cleanup time
If you used the Dynamic Adapting iptables firewall of previous posts...
...You will have also noticed, that it tends to easily mark an entry twice into the firewalls. Here is a simple Perl script to clean out the dupes out of the input queue: #!/usr/bin/perl
# Chain INPUT (policy DROP 6290 packets, 2646K bytes)
# num pkts bytes target prot opt in out source destination
# 1 15 1232 droplog tcp -- * * 151.164.60.251 0.0.0.0/0 tcp dpt:22
# 2 6 448 droplog tcp -- * * 151.164.60.251 0.0.0.0/0 tcp dpt:22
# 3 5 384 droplog tcp -- * * 210.0.137.196 0.0.0.0/0 tcp dpt:22
my $num, $pkts, $bytes, $target, $prot, $opt, $in, $out, $source, $destination, $prot2, $port;
my $p_prot, $p_source, $p_destination, $p_prot2, $p_port;
my @cmds;
foreach (`iptables -vnL INPUT --line-numbers`) {
($num, $pkts, $bytes, $target, $prot, $opt, $in, $out, $source, $destination, $prot2, $port) = split;
if ($target eq "droplog") {
if ($p_prot eq $prot && $p_source eq $source && $p_destination eq $destination && $p_prot2 eq $prot2 && $p_port eq $port) {
push @cmds, "iptables -D INPUT $num"
}
}
$p_prot = $prot;
$p_source = $source;
$p_destination = $destination;
$p_prot2 = $prot2;
$p_port = $port;
}
while ($val = pop @cmds) {
print "$val\n";
`$val`
}Notice how the script pushes the commands in the stack, so they come out in reverse order, so the numbering does not get munged up... little gotcha. It is amazing to notice how much the attacks have gone up since I posted these scripts... I am guessing that it's just good net citizens helping me in testing the efficiency
I was following conversation on LogAnalysis mailing list (LogAnalysis@lists.shmoo.com) and one of the members suggested that finding out if a web site is defaced is one of the valuable things to find out.
Realizing, that all my hosted web sites are stored in a MySQL table (for automated management) — what would it take to add the functionality? Just add one column for the main page "page TEXT" (and a flag "verify CHAR(1)" in case you don't want checking on a domain)... and add 50 lines of Perl... !/usr/bin/perl
# NOTE: Your wget needs to be at least v1.8 to handle PHP based pages
# NOTE: Could use Algorithm::Diff -- if it was more widely available
use strict;
use DBI();
my $debug = 0;
# Connect to sites database
my $dbh = DBI->connect("DBI:mysql:database=floyd;host=localhost", "USERNAME", "PASSWORD", {'RaiseError' => 1});
# Query out of the sites database all proper domains (not subdomains), that have verify flag set to yes
my $sth = $dbh->prepare("SELECT domain,page,contact FROM sites WHERE verify = 'Y' AND domain NOT LIKE '%.%.%'");
$sth->execute();
while (my $ref = $sth->fetchrow_hashref()) {
my $domain = $ref->{'domain'};
$debug && print "$domain\t";
my $page = `wget -q -O - $domain`;
my $orig = $ref->{'page'};
if ($orig) {
if ($orig eq $page) {
$debug && print "Complete match\n";
} else {
open(OUTFILE1, ">/tmp/verify1.$$.txt") or die "Can't open /tmp/verify1.txt: $!";
print OUTFILE1 $orig;
close(OUTFILE1);
open(OUTFILE2, ">/tmp/verify2.$$.txt") or die "Can't open /tmp/verify2.txt: $!";
print OUTFILE2 $page;
close(OUTFILE2);
my $res = `diff -abBdw /tmp/verify1.$$.txt /tmp/verify2.$$.txt`;
my $count = () = $res =~ /\n/g;
$debug && print "CHANGE LINES: $count\t\t";
if ($count {'contact'}\nCc: root\nFrom: root\n";
print MAIL "Subject: Warning: your site $domain index page has changed substantially ($count)\n\n";
print MAIL "diff of the original page:\n$res\n";
close MAIL;
}
}
} else {
$debug && print "No imprint -- storing initial\n";
$dbh->do("UPDATE sites SET page = " . $dbh->quote($page) . " WHERE domain = " . $dbh->quote($domain));
}
}
unlink("/tmp/verify1.$$.txt");
unlink("/tmp/verify2.$$.txt");
$sth->finish();
$dbh->disconnect();Add the flile into your crontab, and you're set... CREATE TABLE floyd.sites (
domain VARCHAR(64) NOT NULL PRIMARY KEY,
verify CHAR(1) NOT NULL,
path VARCHAR(64) NOT NULL,
contact VARCHAR(64) NOT NULL,
email VARCHAR(64) NOT NULL,
aliases VARCHAR(255),
created DATE,
expires DATE,
page TEXT
);MySQL table (text column) is such an easy place to store an imprint of the index page... Perl is ideal for parsing the text, wget(1) for pulling the page, and diff(1) for diff'ing the pages (though I could have used the embedded Perl equivalents — but I wanted to just put this quickly together, instead of romping around in CPAN). Once again happy to have all my data (and now some more) in a MySQL database. This blog entry will also appear in my Network Security (NetSec) blog with more details.
Using Swatch to make adaptive firewall
If my previous posts have gotten you to take a look at your firewall hits...
You will have noticed that a lot of the attacks are scripted — for example dictionary password attacks, where the attacker is hitting your SSH port with a few dozen name/password pairs. The bad news is, if you are running SSHD service, then any attempt to hit your SSH port is legitimate. The good news is, if you see a number of repeated attempts, all failing, then it's an attack. So, is there a way to pick up on the repeated failed attempts, and adjust the firewall to block the attempt out, and perhaps report to DShield? Sure, enter Swatch and throttling. Here is my .swatchrc: perlcode my $ssh_regex = '.*sshd.*authentication failure.*rhost=([^ ]+)';
perlcode my $secure_ssh_regex = '.*sshd.*Failed password.*from ([^ ]+)';
perlcode my $ftp_regex = '.*ftp.*authentication failure.*rhost=([^ ]+)';
perlcode my $xmlrpc_regex = '([^ ]+).*POST .*xmlrpc.php';
watchfor /.*/ and /$ssh_regex/
echo
throttle threshold 4:60
exec "iptables -I INPUT 1 -s $1 -p tcp --dport 22 -j droplog"
watchfor /.*/ and /$secure_ssh_regex/
echo
throttle threshold 4:60
exec "iptables -I INPUT 1 -s $1 -p tcp --dport 22 -j droplog"
watchfor /.*/ and /$ftp_regex/
echo
throttle threshold 4:60
exec "iptables -I INPUT 1 -s $1 -p tcp --dport 21 -j droplog"
watchfor /.*/ and /$xmlrpc_regex/
echo
throttle threshold 4:60
exec "iptables -I INPUT 1 -s $1 -p tcp --dport 80 -j droplog"Just run Swatch with: swatch --tail-file="/var/log/messages /var/log/secure /var/log/httpd/access_log" What you get is:
As soon as they're added to the firewall (as a blacklisted IP address), remembering that these are scipted attacks, the subsequent hits will be collected and reported to DSield. So, you get the benefit of: reporting the scripted attacks to DShield, as well as bit more security from blocking 95% of a dictionary attack. Using the Swatch throttling module gives you the benefit of not having your firewall crammed full of statements for every single hit. On an irregular interval I clean up my firewall (iptables), by just re-initializing it so it doesn't grow to enormous size.
DShield
Distributed Intrusion Detection System Is the first thing I want to tell about.
It is a worthy cause, though I think it does not fully live to its potential — I will explain. DShield is a volunteer system, where you can take your firewall logs, process them (clean up), and then email them for aggregation. DShield will take your logs, essentially all the hacking attempts that went bonk against the wall and correlate them. When you operate a firewall, you are blocking certain ports (TCP and UDP) for access, i.e. you are not allowing incoming connections to those ports. Generally speaking you should firewall every incoming port, unless you are running a service (such as HTTP for WEB, or SMTP for email, etc). In blunt terms — nobody should be able to talk to your computer, unless you talk to them first — sort of like "don't talk to strangers" policy. You would be amazed to note, however, how many hits (hacking attempts) I get on my firewall, on a continuous basis. Against a single publicly visible IP number, I get anywhere between 50 - 500 attempts per hour, every hour of the day. My numbers are lower than usual firewall numbers, since I run so many services, i.e. I consider attempts to connect to my Web (HTTP) server legitimate — most people would not be running a web server, and thus all those hits would be reported as well. Once you install the appropriate scripts that you can download from DShield, your firewall, whether it is a firewall-in-a-box or a Linux-turned-firewall-router, all these attempts are shipped of in an email to DShield, and they will be aggregated for analysis and "Fight Back," where the administrators of particularly noisy sources will be notified. My favorite Fight Back reply is: Date: Mon, 7 Feb 2005 16:25:43 -0500 This user has been locked in the trunk of a 1980 Cadillac along with his PC and has been driven up and down a very bumpy road for several hrs and we believe that the problem is now resolved. Thank you for the report I have been contributing my logs to DShield for years, donate your logs for good cause too! Also, click Here to find out right away if your current source address is listed as a hacked address. Visit file galleries to download the script I use to scrape iptables files.
Background
I run, for fun and profit (educational, not monetary profit), numerous web sites, email and mailing list services, DNS hosting, etc. for friends and my household.
See Otala.Net for some of the sites. Looking at the logs has given me countless times insight onto what's happening on the internet — latest attacks, new style attacks, usage patterns, scalability issues... you name it. This has been invaluable at my work (CTO of SenSage, and earlier at GlobalCenter) and when I have consulted to other companies. I have gathered numerous tools, and written some more... and intend to now start publishing these tools for the good of anyone who wants to protect their networks (or build similar tools). I also have an idea of a new network security product, that would help alert and diagnose internal network hacks — something I have seen on the rise in recent years. Feel free to email back to me, if you find inaccuracies, wish me to accelerate some topic, or just answer questions. I will respond within reason (but I won't design your network for you, at least not unless you convince me). Taneli Otala
Page: 1/1
1 |