The problem: everyone’s source of truth is a show run

I’ve been exploring Ansible at work to better audit a handful of campus networks with branch offices. We’ve had Ansible in our datacenter environment for a while, campus, though, got left behind, and it shows.

Why Ansible for networking (and why “IaC” actually matters here)

It’s tempting to think of Ansible for networking as “a way to run CLI commands from a script instead of typing them.” and while that’s half true, its underselling what you get. Simply put; instead of describing a sequence of commands to run, describe the state you want the device to end up in. That framing has helped me so much with the purpose and value of IaC and how it is a catalyst for automation. Now none of this works without having a source of truth but that is a topic for a future post.

Compare these two mental models:

Scripted CLI (imperative):

configure terminal
interface GigabitEthernet1/0/12
switchport access vlan 20
description Branch-Printer
exit

Run this twice and you MIGHT get an error on the second run or if the interface config changed between the first and second time you ran the script then you could be reverting another change that may have not been documented. The commands don’t check anything before and unless you are keeping copies of configs that are timestamped you dont know if something changed between run one and run two.

Declared state (Ansible, ios_config):

- name: Ensure access port config on Gi1/0/12
  cisco.ios.ios_config:
    lines:
      - switchport access vlan 20
      - description Branch-Printer
    parents: interface GigabitEthernet1/0/12

This can be run hundreds of times, because the module checks current state before it decides whether to push anything. You’re set a state and ansible tries to meet that state, if its already there nothing happens (and you know nothing happened via logs).

Design: inventory, a baseline, and a shape you can reuse

Before any playbook, you need two things sorted: how devices are grouped, and what “correct” means for each group.

Inventory first. Group by role, not by building:

[campus_access]
sw-branch01-ac1 ansible_host=10.20.1.11
sw-branch02-ac1 ansible_host=10.20.2.11

[campus_dist]
sw-branch01-dist1 ansible_host=10.20.1.2

[campus:children]
campus_access
campus_dist

[campus:vars]
ansible_network_os=cisco.ios.ios
ansible_connection=network_cli

Now a baseline is good to have, think stuff every access switch in the environment should agree on: NTP servers, logging hosts, AAA/RADIUS config, banner, SNMP location string, maybe a default VLAN policy. This is where config drift has happened in my case so its the highest value standardization (because who wants to manually check and entire campus switch topology).

Baseline example:

- name: Enforce campus access baseline
  hosts: campus_access
  gather_facts: no
  tasks:
    - name: Baseline NTP servers
      cisco.ios.ios_config:
        lines:
          - ntp server 10.10.10.10
          - ntp server 10.10.10.11

    - name: Baseline RADIUS
      cisco.ios.ios_config:
        lines:
          - radius server RAD-PRIMARY
          - address ipv4 10.10.20.5 auth-port 1812 acct-port 1813
          - key {{ radius_key }}

Variables (like radius_key) live in group_vars/campus.yml, vaulted. In the same way you dont hardcode passwords or secrets into simple bash/powershell scripts.

Build: the baseline role, and diffing before you commit

The actual unit of work I built out first was a campus_baseline role — NTP, logging, AAA/RADIUS, banner, SNMP. Nothing exciting, which is exactly why it was the right starting point: low risk, high drift potential, easy to verify.

The loop that makes this trustworthy is: template → check_mode → diff → apply.

ansible-playbook baseline.yml --check --diff --limit sw-branch01-ac1

--check tells you what would change without touching the device. --diff shows you the actual config lines that would be added or removed. Both of these have been super helpful in verifying what I’m doing and helps me trust the automation a little more than just hitting enter and hoping for the best. Also once the diff looks right, drop --check and your done, no need to retool the script or worry about a different edge case.

Gotcha worth knowing up front: on IOS, ios_config sometimes reports changed: true even when the resulting config is basically identical— line ordering, whitespace, or a re-sent key line that the device can’t tell you it already has (because it’s encrypted on the device and Ansible has no way to compare it) will trip a false positive. Don’t take changed at face value on your first few runs; read the actual diff. You can tighten this up with match: line vs match: none settings and by being deliberate about which lines you send, but expect some noise before you dial it in.

Tradeoffs: Ansible converges, it doesn’t react

It’s worth being honest about what this doesn’t do. Ansible is not a control loop. It doesn’t watch the network and correct drift the moment it happens, it converges state at the moment you run it, and then it’s done thinking about that device until you run it again. If someone makes a CLI change five minutes after your playbook runs, Ansible has no idea until the next run.

It’s also overkill for a day to day admin work. If you need to check one switch’s interface counters right now, just SSH and show interface is faster than writing a task, running a playbook, and parsing the output. Ansible is better on repeatable, declared, fleet wide state think like upgrading the firmware on a bunch of devices or setting up a new branch office.

Ansible answers “What should this be?”

Results

The visible output of all this is unglamorous but real: a Git repo where the campus baseline lives as YAML, with commit history showing who changed the RADIUS key and when, instead of a Slack message nobody can find six months later. Onboarding a new branch switch stopped being “SSH in and copy paste from the switch next to it” and became “add it to inventory, run the playbook.” One push, one diff review, and forty access switches agree with each other again.

Reproducing this

I built and broke all of this against an EVE-NG lab running on a Proxmox cluster before touching anything real, a safe way to find out that your ios_config line ordering assumptions are wrong before you find out on a live switch. In the future I will be posting lab guides on these sorts of topics.

Takeaway, and what’s next

Ansible answers one question well: what should the network be? It’s a declared state tool, and once you stop treating it as “CLI-in-a-loop” and start treating it as rules devices follow. What it doesn’t answer is: what is the network, right now? That’s a different problem, the live state, not the declared state. Conveniently that’s what I’m covering next, using Netmiko to pull real time info instead of enforcing a target config.