According to Gartner, InterCloud is a Software-Defined Cloud InterConnect (SDCI) provider. With this said, it is easy for anyone outside InterCloud's core IT and engineering functions to imagine InterCloud’s implementation of building and service delivery processes to be similar to the fully automated industrial process of a Toyota, or of a cloud based company such as our cloud service providers partners. Actually, our network configuration automation processes are "work in progress". However being “THERE” has always been the goal and still is and for more than two years now, it has been actively pursued. During 2018 we have reached a step where our service delivery architecture are structured “as code” and delivery process is +80 % automated.
The goal of this series of posts is to share with our customers, partners, colleagues and everyone else interested in the topic of automation, especially automation of network and it’s support systems, our journey towards a fully automated network. The first part of this series will focus on the analysis we have made of the existing tools in the market.
State of the art network automation
In the IT context, automation is not something new, if anything else IT is about automation. Sysadmins have been creating and using scripts and different kind of programmatic ways to solve their repetitive burden for ages. Network automation have always been more complicated as network operating systems usually do not present open interfaces to address them programmatically with well known scripting languages such as Perl, Bash or Python etc. There is barely any support to execute even basic procedures in their native language other than CLI scripting. However even network automation has been a thing for some time now, in the nineties with SNMP (I haven’t met a lot of people using the SNMP SetRequest command though. However automated configuration was part of SNMP specification) or more recently with NETCONF. Nowadays, it is possible to look at network automation issue with out of the box solution approach even if some integration will certainly be required. The network automation approach we are going to discuss is the one that have embedded devops philosophy, which enables us to view the whole network as code, that could be versioned, executed, rolled back, continuously integrated and/or delivered and/or interacted with using the same set of tools as system and software engineers. It does not necessarily include Network Virtualization Functions (NFV) or Virtual Network Functions (VNF) even if the approach we are going to discuss can also be used in solving some of NFV/VNF problematics or be part of the tools. Even when narrowed down the way we did, the approaches are still diverse. I will try to categorized them in these 3 different groups :
The do it yourself building block approach
In this scenario, basically the automation teams goes with a programming language a set of libraries / frameworks compatible with the network equipments they are dealing with. This approach is well suited for specific network tasks. For instance each time a new application is in production, vlans are created on switch A and B, a SSH, HTTP(S) flow are opened on external and internal firewall, and a new pool is added on load balancers. Network administrators will make a program or collection of scripts (maybe) in Python using NETCONF library if their equipment is compatible with it or send REST API request if the set of equipments provide an API or a mix of both enabling the configuration of vlans on switches, rules on firewalls and pools on load balancers. This approach requires human intervention to fill in the configuration parameters and validate the implementation. There is usually no transactional workflow involved which make it difficult to rollback the chain of operations when it fails on one device.
Network administrators have kept themselves at this step for long following their sysadmin peers who have been configuring web servers, databases servers or databases using the same kind of approach. This has changed when sysadmins have adopted the IAC (Infrastructure as Code) approach using tools such as Terraform, Ansible, SaltStack, Git, CI/CD emulating the software engineers and configuring a complete system infrastructure in one transactional stack. Then network administrators have started using the same tools but with the old approach : configuring a set of functions on equipements filling in the parameters and checking the implementation. System and software engineers have moved long ago from this step and have become fully integrated and fully automated coining the concept/philosophy/job title of “devops”.
Needless to say this approach is not maintainable, scalable or even suited for more complex environments and tasks.
Integrate a generic automation tool approach
A generic automation engine is most of the time a configuration management tool :
- configuration management tools : like the ones referred to above which will be responsible for the actions (collect artefacts, generate configuration, implement them etc.)
- auditing tools : LogStash, Splunk, Syslog etc. which will analyse the history in order to generate new triggers for actions
- databases : Influxdb, Mongo or any relational database management systems which will store actions and triggers or be used as “source of truth”
- Rules engines : Kapacitor is an example and they are used to match triggers to actions
- Workflow editors : airflow for instance will help design a path to go through for each triggers/actions pair in order to complete a more complex task
These tools have been designed with configuration management and automation of compute assets in mind or plain simple workflow management. They are not tailored for network automation but as they are generic and opened enough and are built in a modular approach and plugin based they have been enhanced with network modules, plugins and workflows in such a way that they are as much compute and network automation relevant now.
These tools can address complex architectures and workflow, can be event based thus implement self healing workflows, as they are opened or at least generic they are well suited for multi-vendor approach, they integrate all of the components necessary for humanless network automation. However they are not out-of-the-box, they require :
- tuning to the environment in which the are going to be used,
- scaling up to the different technologies and concepts they come with
- adapting to their constraints
- accepting they may not fill all the requirements for a particular business case
This approach can be the best in certain environments but may not be suited for all. It’s up to every network automation team to assess if this will suit their needs.
Integrate a generic automation tool approach
A google search on “network vendor automation tools” give a lot more answers now than a year or two ago. Network device manufacturer are shipping their product with tools to help automate their configuration, but there are also pure players who are building automation tools specific for network environment which are vendor agnostic. These tools propose to implement new configuration element into a network without using code to automate them but just a visual workflow tool. Some of them propose machine learning technology which analyses data coming from the network and take actions accordingly. Cisco has been leading this segment with Network Services Orchestrator and Crosswork but the other vendors have also their product like Juniper with Contrail and Northstar. The pure player propose the same set of functionalities with the added benefit that they are cross platform :
- Anuta Network automation
This approach could be suited for homogeneous network environment which are predictive enough and where there is a need to leverage the machine learning, auditing, policy enforcing functions that come with some of these products. And as it can be expected for most of the off the shelf products that target these specific and complex problem some particular but unavoidable scenarii would not be taken into account and it may be very difficult to integrate them seamlessly such that sometimes the benefits are totally wiped out.
This approach was not the most relevant for InterCloud, as we need to be able to operate in a heterogeneous multi-vendor environment, where any possible scenario can arise, standardized and automated. However this approach is the one followed by most of the largest carrier-providers as they have the capacity to build an integrated team around network equipment device vendors and their ecosystem.