Scott O'Brien

Projects, Ramblings and Resources of my (online) life

Compliance with Cisco IOS Devices & Bulk Changes

2014-07-19 00:11:38 +0000 UTC

One of the biggest problems in the environment I work in is that almost all of the deployment of our 300+ devices is that everything has been hand crafted.  Usually this isn’t such a big problem, but add that with a design decision to route right to the access layer of our campus network with a multi-VRF network and you can start to see how mistakes, or changes in design along the deployment has meant inconsistencies.  Not only that, but when it comes time to go and change something, that means going through and altering near 300 devices, a massive pain that is hard to scale.

Usually this problem is solved with network management solutions.  We have Cisco NCM that’s used for this task, but I have to admit, it’s horrible and hard to do anything other than the basics (not to mention end-of-life’d, probably for good reason).

Now that I’ve got Rancid backing up my configs to git, I decided to write a (very simple) compliance manager myself that will allow us to build compliance checks in standard python.  This works by using my IOSDevice wrapper to load up the configuration files, then dynamically loading a list of classes that subclass the compliance check, and start loading them up to perform the check.

By over-riding specific methods (like checkRequired), you’re able to determine if the check is required on that device, then give a status of if it’s failed or not.  There’s methods for generating fix config (that will later be able to be executed on the end device) then the ability to run commands to check if it’s succeeded.

The reason I decided to write this is because we’re soon going to be ripping out and merging VRF’s and making our routing a lot simpler in the campus network (inter-vrf gateway, etc) and there is no way I’m going to be doing this by hand on each device.

If you’re a python head, check it out!

Switchport Mode Access ........(211/301)
  * b123-ba-asw-01
  * b17-mr-wlc-01
  * sbs-18a-asw-01
  * beg-340-asw-01

User Accounting with Netflow

2014-07-08 05:54:52 +0000 UTC

At UOW we had a challenge.  We wanted to allow proxy-free internet, but wanted to keep an eye on how much data was being consumed by what sort of users.  For this we built Project Herbert

It uses netflow from inside our network and some syslog monitoring scripts to match up our private RFC1918 address space to the users who have it at that time, process the flows in near-realtime so we can adjust throttling and firewall policy to be reactive with the environment.

The idea was to build this as a distributed system and allow it to scale-out to deal with more load

Project Herbert

Logging NAT Translations on the Cisco ASA

2014-05-08 06:14:09 +0000 UTC

It’s often handy when dealing with infringement notices and the like to have NAT translations logged.  Sure a better way would be to record netflow from these devices (and include the translations) but for a quick syslog solution, you can always:

logging enable
logging list ToSyslog level critical
logging list ToSyslog message 305011


Messages will look something like:

May 08 13:01:20 %ASA-6-305011: Built dynamic TCP translation from inside: to outside:


Finding overlaps in address space

2014-02-24 06:05:36 +0000 UTC

We have an interesting problem at my workplace, we have an MPLS VPN design for separation of security zones (e.g., staff from students.) and we don’t have MPLS support on our edge. With a L3 to the edge design though this means that while every edge switch has its own address space (per VRF), it also has a /30 uplink (once again, per VRF) back to the PE device.

While this (rightly or wrongly decided) slightly more complicated design in itself isn’t a problem, I’ve been working on programatically putting all of this data into an IPAM (IP Address Management) solution. Doing this from the devices themselves (as apposed to a spreadsheet where it was previously kept) has provided the best way moving forward, so good in fact, that my IPAM started throwing exceptions when duplicate IP addresses and overlapping spaces were attempted to be added in the system.

The following Python script uses my Cisco IOS python library to be able to identify IP address overlaps from a bunch of my saved device configs.

It works by the following:

  1. Adds all L3 interfaces addresses and secondary addresses to a list
  2. Sorts list such that larger subnets are at the front, smaller (/32’s and the such) are at the back of the list.
  3. Walks through each subnet, if the network addresses is not in the routing table, add it in, if it is, add it to a list of colliding subnets.
  4. With the list of colliding subnets (subnets that have been multiply defined, or are of overlapping size):
    1. If they are /30’s and there are only two places of definition, skip
    2. if the subnet values are not equal OR an address appears twice in the subnet.  Print it out as a colliding address (space)

You can find the script as a ghist here (note:  This assumes the devices have been picked and loaded as a Picked list of IOSDevice’s)