Assured: The Isode Blog

Measuring False Negatives and IP Reputation

Richi Jennings noted in a recent post on the Ferris Blog (BorderWare Claims Amazing Reputation Filtering) the claim from BorderWare of getting 98.3% detection using IP Reputation (DNSRBLs), and that other sources suggested 75%.

Isode has been making measurements of false negative rates, published in a white paper, “Measuring the False Negative Rate for Isode’s M-Switch Anti-Spam.”

Our measurements suggest that the (public) DNSRBLs we use hit about 90% of spam. Well-managed DNSRBLs seem an effective way to detect spam, because they have a very low false positive rate. We use DNSRBLs to mark messages (rather than reject at the SMTP server), so we can examine quarantine to check for false positives.

A further 5% can be hit by two other reputation mechanisms:

1. SPF (which is well known) is reasonably effective, but can produce some false positives, particularly in conjunction with mailing lists.
2. SURBL detects URLs within messages, using an underlying RBL mechanism.

Isode’s M-Switch anti-spam can hit most of the remaining spam with a variety of other spam markers and content scoring (using Support Vector Machine derived tables). General-purpose content scoring appears to work very well for many users, but aggressive checking leads to false positives for others, which can be mitigated by use of whitelists.

It seems conceivable that rates higher than 90% can be achieved using public DNSRBLs, although experience suggests that some (poorly managed) DNSRBLs lead to false positives.

This has been cross-posted from the Ferris Blog.

New Evaluation Guide: Directory Access Control using Security Labels

Isode's M-Vault now supports Directory Access Control using Security Labels, this is a feature we introduced with R14.2 and which we've talked about in a number of our recent Directory Whitepapers.
To illustrate how this works in practice, we've produced a new evaluation guide that leads an evaluator through:

* Setting up security label and security clearance controls for the directory
* Testing authentication restrictions and object access permissions
* Testing object addition restrictions

The evaluation page for this feature, together with links to documentation, can be found on the evaluation page.

As new features are added to Isode software and new documents added to our evaluation library, this guide will change. To make it easier for evaluators to keep track of potentially important changes and additions, we've made an RSS feed available for this guide.

Measuring the False Negative Rate for Isode's M-Switch Anti-Spam

A key feature of any anti-spam solution is how effective it is at removing spam. A perfect anti-spam system would have a zero false positive rate and a zero false negative rate. In practice, this is not usually achieved, and systems will invariably trade off the two measurements.

A new whitepaper on the Isode website describes how false negatives can be measured and looks at false negative rates from the beginning of this year for Isode's M-Switch Anti-Spam.

"Measuring the False Negative rate for Isode's M-Switch Anti-Spam"

The graph below shows the false negative rate from January 2008.

Isode R14.2v1 now available

We're pleased to announce that a new version of R14.2 is now available for download from the Isode website.

Details of features and fixes in R14.2v1 can be found in the accompanying release notes.

The binaries for this release can be downloaded from the Partner Index Page or by following the relevant links from the Evaluation section of the website.  You'll  need a Partner or Evaluator login and password to download the binary files and the accompanying Release Notes which details the features and fixes in R14.2v1.

If you wish to evaluate Isode software and do not have an evaluation login, you can obtain one by filling in this short evaluation form.