Logfunneld – handling lines of events

From the got_record sub, we need it to do something other than print out. We need to start down the road toward parsing and processing data.

There are a few dispositions that we can entertain within the context of an event and what to do with it.  We need it to:

• Take an input line or record of data
• run patterns against it
• when patterns match, we need to be able to do things
• And we need to recognize that some patterns should stop processing while others may be processed more than once.

Here is our old got_record call back:

sub got_record {

    my $record = $_[ARG0];

    print "$record\n";

}

We need to establish a hash array that has keys made up of patterns and subroutines handles corresponding to parsers for the event. This hash array we will call %PEZ. In recognizing the disposition of events, there are many lines of events from logs that are unusable or superfluous or don’t need to be handled. In effect, you want to process that patterns that pinpoint the events you want to ignore, up front. In this way, you discard quickly in %PEZ avoiding any heavy lifting if possible.
So, in the new got_record sub:

sub got_record {

   my $record = $_[ARG0};

   my $PEZ_recnumber = 0;

   my $PEZtotal = #${keys %PEZ};

   while (( my $pattern, $handler) = each (%PEZ)) {

      $PEZ_recnumber++;

      if ($record =~ $pattern) {

	$handler->($record);

        }

      else {

        next;

        }

    }

    return;

}

What this does is to run through each key value pair in the %PEZ hash and match the pattern to the input $record. If the $pattern regmatches to the $record, it executes the subroutine coderef in the value part of the %PEZ Hash and passes the $record to it as an argument. The %PEZ is organized like this:

%PEZ = (

   qr/LINK\-3\-UPDOWN/                    => \&link-3-updown,

);

Notice that we use qr//. A qr// is used to precompile regular expressions making patterns run significantly faster than compiling at runtime as “LINK\-3\-UPDOWN” would be. Either pattern is acceptable and can be mixed within %PEZ.
As you may recognize, this event is from Cisco devices as a syslog message and is propagated via a record in syslog like:

Oct 13 12:33:48 router.yada-yada.subnet.dummy.net Oct 13 12:33:48: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/20, changed state to down

So, the ink-3-updown sub would look like this:

sub link-3-updown 

   $input = $_[ARG0];

   # Pattern is qr/LINK\-3\-UPDOWN/

   #  Oct 13 12:33:48 router.bogus.net Oct 13 12:33:48: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/20, changed state to down

   # I add the pattern and one or more `samples in the handler subroutine to make it more readable and more reentrant.

   # So, you can tokenize $input by spaces as:

   my @Spaces = split("/s+", $record);

   # or by colons...

   my @Colons = split(":", $record);

   # or Percent

   my @Percents = split("\%", $record);

my $Node = $Spaces[3];

if ($Spaces[8] eq "Interface") {

   my $Subobject = $Spaces[9];

   }

elsif ($Spaces[6] eq "Interface") {

   my $Subobject = $Spaces[7];

   }

else {

   my $Subobject = "Cannot find Subobject...";

   print STDERR "Cannot find Subobject for $record\n";

   }

if (/changed state to down$/ =~ $record) {

   my $Severity = "Major";

elsif (/changed state to up$/ =~ $record) {

   my $Severity = "Clear";

   }

else {

   my $Severity = "Unknown";

   }

my $Alerttext = $Percents[1];

# For the basics, we will add in pushing this to a DB in a later post

print "$Node #Subobject $Severity $Alerttext\n";

return;

}

The beauty behind the %PEZ is that we store the pattern and the coderef of the subroutines in the Hash. Also, because the subroutine handle is exposed within the context of executing code within Perl, we can read these
subroutines in as part of configuration. But think about this. If I can load in external subs at run time, I can also do so on the fly. In later posts I’ll put up a control portal and present
some commands that enable patterns to be added, changed, or deleted in %PEZ but we can also add, change, or delete the Handler subroutines in real time.
If we wrap the sub in an eval, we can even test it dynamically. This will enable an administrator to add new handlers and patterns on the fly without having to HUP or restart the process.

Previous Post -> EMD Part 1 - logfunneld

Start of Thread -> Netcool