Models as the Universal Abstraction Layer

Enterprises that adopt cloud technology often use private cloud technology as well as public cloud services, and in fact they frequently use public cloud services from multiple providers. Different cloud technologies tend to expose different Application Programming Interfaces (APIs), which makes it difficult for enterprise customers to develop truly portable cloud management tools. While enterprises desire a “write once—deploy everywhere” approach to cloud management, the reality is that management features often must be implemented multiple times, once for each supported cloud platform.

Similar challenges exist in the NFV space, where Virtual Network Functions (VNFs) from different vendors tend to expose proprietary APIs for life-cycle management as well as ongoing management. The ETSI NFV reference architecture deals with this by bundling each VNF with its own VNF manager to provide life-cycle management services. However, this architecture arguably complicates integration efforts between VNFs since it introduces the need for coordination between VNF Managers from different vendors.

A common solution to these types of portability and integration challenges is to introduce an abstraction layer that creates a standardized umbrella API on top of various disparate interfaces exposed by the different platforms. This abstraction layer translates calls made through the umbrella API into the corresponding vendor-specific API calls (and performing the necessary data adaptations in the process).

One example of such an umbrella API is OASIS CAMP which is an interface for self-service provisioning, monitoring, and control of cloud services. Similar API-based interoperability approaches have been suggested in the NFV space. For example, some architectures propose the introduction of a standardized life-cycle management API to be used between the VNF Manager and the VNFs themselves, which would allow the introduction of a single “generic” VNF Manager. While this approach would address the integration challenges between different VNF Managers, the standardization of such an API is likely to take some time, assuming it happens at all.

While API-based abstraction layers improve portability, they suffer from a number of shortcomings:

• Umbrella APIs tend to provide only lowest common denominator functionality, which means that developers won’t be able to take advantage of the more sophisticated features provided by each platform. To take advantage of those features, developers must go back to using proprietary interfaces, which negates a lot of the value that umbrella APIs bring to the table in the first place. 
• API-based abstraction layers tend to be complex to design, especially when APIs aren’t stateless. Complex logic might be necessary to accommodate different state machines assumed by different APIs.
• Adding support for a new type of managed elements requires the introduction of a new API that is tailored to that type of device. However, adding new APIs into an abstraction layer dynamically is challenging, which means that often a new version of the system is required when adding support for new types of elements. 

This begs the question whether unifying APIs is the best way to address portability and integration challenges. Perhaps leading with APIs is like putting the cart before the horse. APIs focus on actions, the operations that can be executed by the provider of an interface. What if we focused instead on the abstractions--the entities on which these operations execute? By trying to model these abstractions, we would enable a much simpler abstraction approach as follows:

• Define standard models for the various elements being managed. These models acts as vendor-independent representations of these elements and are decoupled from vendor-specific mechanisms for operating on those elements.
• Introduce simple management interfaces that operate on the models rather than on the managed entities directly. When management tools use these interfaces, they affect the state of the models only rather than the state of the managed elements.
• Synchronize the state of the managed elements with the state of the models at all times: when the state of any of the modeled entities is changed, use whatever vendor-specific APIs are available to reflect these changes into state changes of the managed elements. Similarly, whenever external events affect the state of the managed elements (e.g. because of a failure) make sure that the state of the managed element is reflected back into the state of the model. The result is that the state of the model is synchronized at all times with the state of the managed entities.

This approach gets us out of the business of trying to standardize APIs since each vendor can plug in whatever APIs are most convenient for their implementations. More importantly, this approach works at all layers in the stack, from infrastructure to network functions, application functions, and entire services, which results in a uniform abstraction methodology across the entire cloud service stack. Adding support for new types of elements does not require new APIs, only new models.

There is of course still the need to standardize the interface for operating on the models themselves, but this interface requires only a trivial set of primitive operations such as creating or deleting entities, changing properties of entities in the model, and changing relationships between entities in the model. As such, model-based abstraction layers offer benefits in simplicity similar to those provided by REST APIs.

Note that model-based abstraction is exactly the approach taken by the OpenDaylight (ODL) SDN Controller. While the original version of OpenDaylight was based on an API-Driven Service Abstraction Layer (AD-SAL), the AD-SAL has been abandoned in the most recent version of ODL in favor of a Model-Driven Service Abstraction Layer (MD-SAL) to avoid the complexities associated with trying to define umbrella API abstractions in an ever-evolving networking world.

Unfortunately, ODL uses YANG as the domain-specific language to describe the models managed by the controller, which makes it difficult to extend ODL beyond SDN and towards a broader cloud and NFV management mission. YANG was conceived as a configuration language for network devices, not as a general service modeling language.  It doesn’t support the necessary service life-cycle management operations, it doesn’t have a type system that allows for multiple interoperable implementations, and it doesn’t easily support composition.  Even as a configuration language, it is somewhat limited since it doesn’t support recursive hierarchies, which are typical in modern service architectures.

What’s needed instead to create a universal model-based abstraction layer is a modeling framework that is designed specifically for the life-cycle management and integration requirements of modern cloud-based services. Such a modeling framework needs to include the following:

• A mechanism for modeling services, the components that make up these services, and the relationships between these components.
• Support for recursive decomposition which allow services to be used as components of other services.
• A standardized life-cycle management interface that can be used to instantiate, start, stop, and remove services.
• A mechanism for modeling the integration aspects of services and their components in a standardized way. This is typically done using abstractions such as capabilities that model the features exposed by service components and requirements that model external features that are needed by service components. 
• A standardized type system for portability and interoperability

TOSCA might be a good starting point for such a modeling framework since it already includes many of the features outlined here.

Adopting a model-based abstraction layer addresses many of the shortcomings of API-based approaches:

• The abstraction layer can be easily extended at run-time, just by introducing new models into the framework.
• By leveraging vendor-specific APIs for synchronizing state, we don’t need to compromise on functionality.
• Complexity can be managed within individual models rather than across an entire API layer, with improved reliability as a result.

As an industry, let’s focus on abstractions—not actions—and take advantage of these benefits.