Totally agree with you.
Also, some times (or should I say most of the time) those marketing ppl.
are trying "too hard" on pushing their product(s); That they will even
tells you that Earth rotate against the Moon (just joking here).
Raistlin wrote:
>>It claims to have a 100% accuracy , no false positives.
>>
>>
>
>It's really simple to build a system with no false positives. Just leave it
>unplugged. It generates no false positives, since all the positives (none)
>are true positives.
>
>Unluckily, this doesn't say a word about the performance of the system, does
>it ? :-)
>
>If correct positive assignments are A, false positives are B and false
>negatives are C, accuracy is A+D/A+B+C+D, precision is A/A+B and recall is
>A/A+C (in document retrieval terms; i'm not aware of an established IDS
>terminology, but the concepts are similar on the whole). A 100% accuracy has
>no meaning whatsoever. The absence of false positives means a 100%
>precision, but we cannot pretend marketing people to read the Communications
>of the ACM, can we ? :-)
>
>What you really want is a high signal-to-noise ratio (many true positives
>among the positives), so a high precision, that's right, but also a high
>recall (many of the attacks must be detected). You can plot precision vs.
>recall in a ROC curve. They have done it that way in biology and medicine
>for years, and the graph usually shows an inverse proportionality. 100%
>precision means a very, very low recall, if any (unless you have designed
>the perfect intrusion detection system, and I'd challenge that even on
>theoretical grounds ;).
>
>A high precision, per se, means absolutely nothing. A nonexistent IDS is
>totally precise: it never generates a false alert. It never generates an
>alert, also :)
>
>Stefano "Raistlin" Zanero
>System Administrator Gioco.Net
>public PGP key block at http://gioco.net/pgpkeys
>
>
>
--
Roy Lo
Freelance Consultant
E-mail - roylo_at_sr2c.com
Sun Certified Network Administrator (SCNA)
Sun Certified System Administrator (SCSA)
Cisco Certified Network Associate (CCNA)
Received on Dec 11 2002