In general, to make our time more effective we all must do this type of analysis, since we all know that it is becoming increasingly difficult to read logs on a daily basis. In other words it's hard to read gigs of log data each day and not all of us have teams of people reading logs. I commend you for writing the Perl program and developing your query tool in access.
It makes perfect sense; some vendors are doing this now in an attempt to maximize the effectiveness of their security personnel (including my company). If I may, suggest something a little more robust than Access (file size restrictions etc). Not only firewall data, but why not multiple types of data. These could be entered into the system. It's all about how the system parses and understands the data submitted. For example, how your script would possibly understand the bytes in the payload (http) is a function of parsing that statistic.
I don't think man or machine can make an accurate decision based purely on the number of bytes in a packet thusly the need for more information. For example, you know you have an animal in a box, the animal breaths and weighs 15 pounds, fur is coming out of the box, but you still don't know if it's a black cat (hat) or your loving lab puppy. I also, think this should be left to the IDS/IPS to perform these functions aka "peek into the packet".
I enjoyed your comment on "List all successful TCP connections for everyone who had more than 1 explicit denied connection". Building more or less what's called event chaining or what some vendors are calling "event correlation". Linking events seems to be a very good way of reducing false positives, however it's not an end all be all to discovering new attacks/attackers. Anomaly detection via statistical analysis would be an effective method for discovering these new attacks.
Matthew F. Caldwell, CISSP
Chief Security Officer
Guarded Net, Inc. "The home of neuSECURE"
www.guarded.net
em:mattc_at_guarded.net
-----Original Message-----
From: Terry Ziemniak [mailto:tmz_at_hawk.swc.com]
Sent: Wednesday, December 11, 2002 11:00 AM
To: focus-ids_at_securityfocus.com
Subject: Firewall Activity analysis
All,
I have been working on firewall activity analysis for Pix firewalls for
a while. I have written a perl script that parses the log files and
puts all of the data into an Access database. This allows me to run
queries such as "List all successful TCP connections for everyone who
had more than 1 explicit denied connection".
This is an explicit (rigid ?) way to flag bad behavior. However I was
wondering it makes sense (now that all of the data is in a database) to
attempt statistical analysis of this data to flag bad behavior.
I could look at the HTTP bytes, or number of connections, or time (etc)
and flag source IPs that deviate from the norm by a certain amount. I
could do this without setting hard limits (such as 'list the top 1%
incoming HTTP users') which would limit that amount of IPs flagged as
bad. Of course this would be applicable to any protocol.
The goal of this is to flag suspicious communications that should be
more thoroughly investigated.
At this point this is a mental exercise but I was wondering if anyone
had any thoughts or opinions on the matter.
PS - This is based on my somewhat tenuous grasp of statistical analysis.
Thanks.
Terry
Received on Dec 11 2002
Intro
