My experiences of managing a Cisco switch with Puppet

December 8th, 2012

One recent pet gripe of mine has been having to add a new VLAN into our datacenter for our vSphere platform.  Not that I trust my DCs switches with puppet just yet, this is a proof of concept post about how we could be using puppet to centrally manage this configuration and push it out across our DC.

Before
We’ve got a pretty basic topology going on in our DC, it’s just a VSS with the other switches pretty much being nothing more but layer 2 for the most part.  The dot1q trunk back to the VSS carries all VLANs from our end of rack switches.  When we add a new vlan in the DC to trunk to the ESX machines, we would add the VLAN in all the DC switches (not running VTP) then add the vlan to the trunk port on each port patched to the ESX hosts.  (we’re not using any link aggregation to the ports connected to the same ESX host, the ESX hosts themselves have their own load balancing method.. If you know any for/against doing it like this please comment and let me know)

Setting up the Puppet lab

Introduced in Puppet 2.7 is network device management.  This more or less is an expect script to manage interfaces and vlans on IOS devices.  For this lab, we will be using cisco IOU with the following topology

Setting up the Devices

Ideally, you would have a few puppet nodes that manage a few devices each to spread out the load, for the purposes of this exercise, I created a single vm running Centos6 with both puppet-server and puppet installed.  For this machine to manage the switches, I added the following into the device.conf file

[[email protected] ~]# cat /etc/puppet/device.conf
[dc_sw1]
  type cisco
  url telnet://puppet:[email protected]/
[dc_sw2]
  type cisco
  url telnet://puppet:[email protected]/
[dc_sw3]
  type cisco
  url telnet://puppet:[email protected]/
[dc_sw4]
  type cisco
  url telnet://puppet:[email protected]/

Signing the devices

To update the devices, you have to run puppet device.  The first time you run it, a certificate will be created that needs to be signed on the puppet master.

[[email protected] ~]# puppet device --verbose
info: starting applying configuration to dc_sw4 at telnet://puppet:[email protected]/
info: Creating a new SSL key for dc_sw4
warning: peer certificate won't be verified in this SSL session
info: Caching certificate for ca
warning: peer certificate won't be verified in this SSL session
warning: peer certificate won't be verified in this SSL session
info: Creating a new SSL certificate request for dc_sw4
info: Certificate Request fingerprint (md5): E8:A6:35:9D:BF:CE:3D:BC:E0:E4:C2:5B:00:CE:9F:DB
warning: peer certificate won't be verified in this SSL session
warning: peer certificate won't be verified in this SSL session
warning: peer certificate won't be verified in this SSL session

so we’ll need to sign our devices

[[email protected] ~]# puppetca --sign dc_sw4
notice: Signed certificate request for dc_sw4
notice: Removing file Puppet::SSL::CertificateRequest dc_sw4 at '/var/lib/puppet/ssl/ca/requests/dc_sw4.pem'

Setting up the switches for Puppet

If you look up to the device configuration file, we need to create a local user for puppet to log into the switch (remember, it acts much like an expect script)

username puppet privilege 15 password 0 cisco
line vty 0 4
 privilege level 15
 password cisco
 login local
 transport input all
!

The Configuration

so with no more ado, we can easily simply abstract the behaviour of these ports using puppet syntax :)

define esxport( $port ){

        interface {
                "${port}":
                        mode => trunk,
                        duplex => full,
                        description => "ESX Host",
                        allowed_trunk_vlans => "3,4,5,8,9"
        }

}

node "dc_sw1" {

        esxport { 'e0/2': port => 'Ethernet0/2' }
        esxport { 'e0/3': port => 'Ethernet0/3' }

}

node "dc_sw2" {

        esxport { 'e1/0': port => 'Ethernet1/0' }
        esxport { 'e1/1': port => 'Ethernet1/1' }
        esxport { 'e1/2': port => 'Ethernet1/2' }
        esxport { 'e1/3': port => 'Ethernet1/3' }

}

The Ugly

A lot of the states don’t yet seem to be supported by this module.  This means even the default trunk mode of dynamic desirable will cause issues when Puppet is pulling device information and you’ll have to manually specify “switchport trunk encapsulation dot1q” and “switchport mode access” before setting puppet free on the devices.

Results

[[email protected] ~]# puppet device --verbose
info: starting applying configuration to dc_sw4 at telnet://puppet:[email protected]/
info: Caching catalog for dc_sw4
info: Applying configuration version '1355007108'
notice: Finished catalog run in 0.20 seconds
info: starting applying configuration to dc_sw3 at telnet://puppet:[email protected]/
info: Caching catalog for dc_sw3
info: Applying configuration version '1355007108'
notice: Finished catalog run in 0.21 seconds
info: starting applying configuration to dc_sw2 at telnet://puppet:[email protected]/
info: Caching catalog for dc_sw2
info: Applying configuration version '1355007108'
notice: /Stage[main]//Node[dc_sw2]/Esxport[e1/1]/Interface[Ethernet1/1]/description: defined 'description' as 'ESX Host'
notice: /Stage[main]//Node[dc_sw2]/Esxport[e1/1]/Interface[Ethernet1/1]/duplex: duplex changed 'auto' to 'full'
notice: /Stage[main]//Node[dc_sw2]/Esxport[e1/1]/Interface[Ethernet1/1]/mode: mode changed 'access' to 'trunk'
notice: /Stage[main]//Node[dc_sw2]/Esxport[e1/1]/Interface[Ethernet1/1]/allowed_trunk_vlans: defined 'allowed_trunk_vlans' as '3,4,5,8,9'
notice: /Stage[main]//Node[dc_sw2]/Esxport[e1/3]/Interface[Ethernet1/3]/description: defined 'description' as 'ESX Host'
notice: /Stage[main]//Node[dc_sw2]/Esxport[e1/3]/Interface[Ethernet1/3]/duplex: duplex changed 'auto' to 'full'
notice: /Stage[main]//Node[dc_sw2]/Esxport[e1/3]/Interface[Ethernet1/3]/mode: mode changed 'access' to 'trunk'
notice: /Stage[main]//Node[dc_sw2]/Esxport[e1/3]/Interface[Ethernet1/3]/allowed_trunk_vlans: defined 'allowed_trunk_vlans' as '3,4,5,8,9'
notice: /Stage[main]//Node[dc_sw2]/Esxport[e1/2]/Interface[Ethernet1/2]/description: defined 'description' as 'ESX Host'
notice: /Stage[main]//Node[dc_sw2]/Esxport[e1/2]/Interface[Ethernet1/2]/duplex: duplex changed 'auto' to 'full'
notice: /Stage[main]//Node[dc_sw2]/Esxport[e1/2]/Interface[Ethernet1/2]/mode: mode changed 'access' to 'trunk'
notice: /Stage[main]//Node[dc_sw2]/Esxport[e1/2]/Interface[Ethernet1/2]/allowed_trunk_vlans: defined 'allowed_trunk_vlans' as '3,4,5,8,9'
notice: /Stage[main]//Node[dc_sw2]/Esxport[e1/0]/Interface[Ethernet1/0]/description: defined 'description' as 'ESX Host'
notice: /Stage[main]//Node[dc_sw2]/Esxport[e1/0]/Interface[Ethernet1/0]/duplex: duplex changed 'auto' to 'full'
notice: /Stage[main]//Node[dc_sw2]/Esxport[e1/0]/Interface[Ethernet1/0]/mode: mode changed 'access' to 'trunk'
notice: /Stage[main]//Node[dc_sw2]/Esxport[e1/0]/Interface[Ethernet1/0]/allowed_trunk_vlans: defined 'allowed_trunk_vlans' as '3,4,5,8,9'
notice: Finished catalog run in 14.22 seconds

my $0.02

I must say, I’m very disappointed in this module so far.  It shows great promise and makes a once tedious task relatively effortless to manage, however with the time invested to find out what is and what is not supported, I think it’s far too early to invest in such a solution.  The idea of setting something like an expect script loose on my kit also worries me.  It’s much better to have an API or a promise that the input/output the expect script uses won’t change in a future release then do something unexpected (pun intended there.)

I guess if we were using an OS like Junos we could have created apply-groups like this to abstract the configuration in much the same manner, at least down to the switch level.  Very interesting for a new take on managing these things though

EDIT:

I’ve been thinking about this a lot since I posted it.  I think I was too harsh on the tool.  It seems even Cisco’s own tools work by ssh’ing into the box to make their changes.. while not ideal, for these old IOS devices around, it seems to be the accepted thing to do.  It’s exciting times ahead in this space though, I can feel it!

14 responses

  1. Brice Figureau comments:

    Hi,

    Thanks for your detailed article about Puppet Network Device. First, let me state, as the author of said code, that in my view it was much more of a proof of concept than production ready code. My point was for some other people to help build something bigger, adding more network device or feature.

    That just didn’t happen; my spare time being limited I couldn’t really bring the network devices stuff up to what I’d like it to be.

    Regarding some of the shortcomings you’re talking about, you can setup encapsulation in puppet (add encapsulation => “dot1q” to your interface resource). Also, if you found any issues, please report them as bug in the puppet redmine bug tracker. That way I could have a look, and possibly fix then.

    Amd finally, yes, the cisco network device is more like a giant smart expect script, but that’s because to my knowledge IOS doesn’t support any public API. For instance the F5 module uses the F5 API to do all the configuration.

    One subtle difference with an expect script, is that Puppet is idempotent, while an expect script can only deal with things you took into account.

    Also, note that all the network device gears with PND become full-fledged puppet host, so that means you get reports, inventory and such for free when using this tool.

  2. scottyob comments:

    Hi Brice,

    Thanks for the clarification. As a proof of concept I think it’s amazing. I’ve had plenty of debates recently about Puppet’s place in the networking world and I’m always of the opinion when managing things like switchports it’s just as suited as using puppet on servers. I was testing with a L2IOU image, I’ll submit the bug reports for what I’ve found hopefully this weekend if it helps you out.

    I must confess, I’ve never done anything with the reports, inventory and such in puppet before, this proof of concept was to see how it would go solving my specific problem in our datacenter.

    I’m not sure if you’ve seen it, but you might like to check out http://www.cisco.com/web/solutions/trends/open_network_environment/index.html?CAMPAIGN=Cisco+Open+Network+Environment&COUNTRY_SITE=us&POSITION=sl&REFERRING_SITE=Cisco.com+homepage&CREATIVE=open+network+environment2+homepage+spotlight It looks like we might be getting an API yet *fingers crossed*! It’s not your work I don’t love (keep up the good work, it’s great) it’s the platform not lending itself to easily integrate with a tool like this. While it solves my problem, personally I’m not going to use it with the worry of cisco changing the way I/O works in a later version, it doesn’t feel me with great confidence to embrace this solution.

    I am curious though, when writing the module, did you ever consider getting/setting the values from snmp?

    Thanks again for reading and love your work! :)

  3. Daniel Hooper comments:

    Cisco API? Can you use SNMP to do this?

  4. scottyob comments:

    Made an edit to the post.. while not perfect, when talking IOS, it’s the best we have

  5. srivatsa comments:

    Hello Scott,
    Detailed and wonderful explanation. I’m new to puppet and got things working after 2 weeks.
    I followed your article to setup my switch and everything works well, only thing I’m not aware of is where should the manifest (.pp) for the device be place? and under what name?

    Thanks,
    Vatsa

  6. srivatsa comments:

    I just figured out I need a site.pp (which dint exist) and get my manifest there.

    Now I wanted to know is there a way to specify what device we are using like a switch or a router, if so how and where?

    Thanks,
    Vatsa

  7. cast iron tea pots comments:

    Undeniably believe that that you stated. Your favourite justification seemed to be at the web the easiest factor
    to bear in mind of. I say to you, I certainly get annoyed
    whilst people think about concerns that they just do not
    understand about. You managed to hit the nail upon the highest and also
    outlined out the whole thing without having side-effects
    , folks can take a signal. Will probably be back to
    get more. Thank you

  8. Marcella comments:

    Gold ended the year where it started, while
    unloved bank shares had a superb run. An exemption was
    made during the confiscation for dentist gold, gold?
    This is the game within the game is to transform an ailing professional
    football club into a winning team and a successful business.
    You can save thousands of dollars orr more per acre.

  9. new balance m576 comments:

    I was wondering if you ever thought of changing
    the page layout of your site? Its very well written; I love what youve got to say.
    But maybe you could a little more in the way of content so people could
    connect with it better. Youve got an awful lot of text for only having one or
    2 pictures. Maybe you could space it out better?

  10. http://support.thufri.com/entries/62570707-Small-Loans-Secrets-You-Have-To-Know comments:

    What a information of un-ambiguity and preserveness of
    valuable know-how concerning unexpected emotions.

  11. Adeel comments:

    Hello Scott,

    My name is Adeel and i am Student of Electronic Engineering .I saw your Blog it is very very nice and helpful for me.
    I saw that u have great command on Puppet (Configuration Management Tool) . I saw your post ” My experiences of managing a Cisco Switch with Puppet”
    I have a project in my university which is basically related to your Post.the task of my project is
    “Automate Cisco Switch using any Configuration Management Tool”.
    The aim of the project is that Switches and Routers have some Configurations and I have to use any Configuration Management Tool so that i will configure all the switches and router through a server or configure through a single click.

    So please i need your little help about this project i will be very thankful to you.

    Best Regards
    Adeel Ahmed

  12. raleigh seo experts tracks back:

    raleigh seo experts

  13. ข้อสอบข้อเขียนนิเทศ จุฬา comments:

    Hi there colleagues, its impressive post on the topic of cultureand
    completely explained, keep it up all the time.

  14. YDUQKCEELIJIDJEI comments:

    There are actually loads of particulars like that to take into consideration. That is a nice point to convey up. I supply the thoughts above as normal inspiration however clearly there are questions like the one you deliver up where a very powerful thing shall be working in trustworthy good faith. I don?t know if greatest practices have emerged round issues like that, however I am certain that your job is clearly identified as a fair game. Each girls and boys really feel the influence of only a momentpleasure, for the remainder of their lives.

Leave a comment