Chapter 25. Neutron - Quality of Service (QoS)

Quality of Service (QoS) is used to provide differentiated services (DS) on network traffic. This will allow policies to be created to filter and govern network traffic flow. When applied to a virtual network environment, this filtering and policing takes place at the Network Port level, specifically the port to which the virtual machine or container is connected to the virtual network. This effectively policies all traffic exiting the VM and entering the virtual network.

QoS is configured with standard Neutron operations via the MidoNet Neutron plugin or directly via the MidoNet CLI. Note that using the MidoNet CLI will bypass the Neutron operations and end up with policies and rules that are not tracked in the Neutron database. For this reason, it is highly suggested to use the standard Neutron QoS commands when the virtual environment is based on OpenStack (this doc), and MidoNet CLI (Chapter 23, MidoNet - Quality of Service (QoS)) when OpenStack is not present.

 Creating Quality of Service Policies

In order to use QoS policies, a policy must be created, rules added to it, and then that policy must be applied to a Port (or to all ports on a Network).

 Creating a QoS Policy object

QoS policies can be created via the Neutron command:

$ neutron qos-policy-create --description <desc> --shared <true|false> <name>

Where desc is an optional description, shared can be set to true (share this policy among all tenants) or false (make this policy available only to this tenant), and name will name this policy (the name is the only required field).

 Creating QoS Policy Rules for a Policy

Even though a QoS policy is created, it’s just an empty box with no rules set until some rules are created on the policy. Rules must be created directly on a specific policy, and the rule type must be decided ahead of time. Rules cannot be created standalone nor can they be shared between policies. Rules are only applicable to the policy on which they were created.

 Bandwidth Limit Rules

A Bandwidth Limit type rule will limit data throughput on a port. There are two meters that can be applied via a bandwidth limit rule: maximum data rate over a long period of time (referred to as "max_kbps"), and maximum data burst allowed with a very short window (referred to as "max_burst_kbps", which unfortunately implies a data rate, when in practice, this is a maximum limit on data sent at a particular time).

[Note]Note

How Linux Meters Traffic

The best way to contextualize this is to actually explain briefly how the mechanics of how Linux implements data rate limiting. Basically, picture a bucket with tokens. The max_kbps represents how many tokens put into the bucket each second (e.g. 1,000 kbps means 1,000 tokens are put into the bucket every second). The max_burst_kbps represents the maximum number of tokens allowed in the bucket. For every byte of data that passes through the port, a token is removed from the bucket. If there are not enough tokens in the bucket for the packet to take, the packet is dropped.

This means that the tokens can fill up over time up to a maximum of max_burst_kbps, which can then allow a large packet to pass through, however if the traffic is constant at max_kbps, this means tokens are being removed from the bucket as fast as they are being added, and larger packets will be dropped as there will never be enough tokens available.

qos token bucket 773x954

Creating a Bandwidth Limit Rule

Bandwidth limit rules can be created via the Neutron command:

$ neutron qos-bandwidth-limit-rule-create --max-kbps <max_kbps> --max-burst-kbps <max_burst_kbps> <policy_id>

Where policy ID is the UUID of the policy to add the rule to, max_kbps is the maximum sustained data rate allowed (rate to fill the token bucket in the example above), and max_burst_kbps is the maximum packet size allowed (maximum tokens allowed in the token bucket in the example above).

 DSCP Marking Rules

A QoS policy can have rules added to alter the IPv4 ToS field (now used for DSCP) of every packet that enters the cloud via a port. This will affect all IPv4 packets coming into the cloud on the port with the QoS policy set. Packets exiting the cloud on the port will not be affected. At this time, the DSCP marking rules will affect all packets, meaning the last DSCP marking rule will replace the ToS field with its marking value, overwriting any previous values set (even by other DSCP marking rules). This is to say that it really only makes sense to have a single DSCP marking rule on any QoS policy.

Creating a DSCP Marking Rule

DSCP marking rules can be created via the Neutron command:

$ neutron qos-dscp-marking-rule-create --dscp-mark <dscp_mark>

Where policy ID is the UUID of the policy to add the rule to and dscp_mark is the value that will be written to each IPv4 packet’s ToS field.

 Applying QoS Policies

QoS rules are not actually put into effect unless they are attached to a port on a virtual network that has ingress traffic from a VM or container. Once attached to a port, the policy will regulate traffic on that port.

There are two ways to apply the effects of a QoS policy: either apply it directly to a specific port, or apply it to an entire network, which will affect all ports on that network.

[Important]Important

When a QoS policy is specified on both a port AND that port’s network, the specific port’s policy will apply, overriding the network setting.

If a port does not have a specific policy, or if it’s specific policy is removed, traffic will then conform to the QoS policy on the network, if present. In this way, a default can be set for an entire network, with specific ports overriding the policy to their own individual needs.

If a QoS policy which is attached to a port or network has its rule set updated (either a rule is deleted or a rule is created), that change will take effect on that network or port immediately upon the update. A separate update of the port or network object is not necessary.

 Applying a QoS Policy to a Single Port

Policies are applied to a port at the QoS policy object-level. The individual rules on that policy object will be applied. It is not possible to only apply a subset of rules on a policy to a port. If a specific port needs special handling where only a subset of rules should be applied, then an entire new QoS policy object must be created with only the desired subset of rules added to it, and that new policy can then be applied to the special port.

Note that, unlike rules, the QoS policy object can be set on many ports or networks, in which case the policy’s rules will apply in the same way to each port with the policy.

A port can be updated to have a QoS policy via the Neutron command:

$ neutron port-update --qos-policy <policy_id> <port_id>

The policy ID is the UUID of the created QoS policy object, and the port ID is the UUID of the port to be updated.

Ports can also be created with the QoS policy set:

$ neutron port-create <...other params...> --qos-policy <policy_id> <network_id>

Where policy ID is the UUID of the policy object and network ID is the UUID of the network on which to create the port. Other parameters can be specified along with qos-policy.

 Applying a QoS Policy to All Ports on a Network

QoS Policies can also be applied to an entire network, in which case all VM/container ports on that network will have to rules applied. Again, if a particular port ever has a separate QoS policy assigned to it (via the above command, for example), that port’s specific policy will take effect instead of the network’s policy. The QoS policy that will be applied will be determined at any time by: a) If the port has a policy, use that, otherwise b) if the network has a policy, use that, otherwise c) use no QoS policy. Any updates to either the port’s or network’s QoS policy settings will take effect immediately.

A network can be updated to have a QoS policy via the Neutron command:

$ neutron net-update --qos-policy <policy_id> <port_id>

The policy ID is the UUID of the created QoS policy object, and the network ID is the UUID of the network to be updated.

Networks can also be created with the QoS policy set:

$ neutron net-create <...other params...> --qos-policy <policy_id> <name>

Where policy ID is the UUID of the policy object and name is the desired name of the network. Other parameters can be specified along with qos-policy.

 Listing Active Policies and their Rules

QoS policies and their rules can be listed, displaying their parameters.

 Listing All Policies

These commands will list all policy objects.

$ neutron qos-policy-list

This will list all policies along with their rules.

 Listing Rules on a Policy

These commands will list all rules for a given policy.

$ neutron qos-bandwidth-limit-rule-list <policy_id>

Where policy ID is the UUID of the policy object to list the rules from.

 Listing the Policy Active for a Port or Network

These commands show the configured QoS policy for a port or network.

$ neutron port-show <port_id>

Where port ID is the UUID of the port to show the configured QoS policy. This will display the port information. The UUID of the configured QoS policy will be set under the "qos_policy_id" field. If the field is absent or not set, then no QoS policy is set on the port.

For a network, the process is the same:

$ neutron net-show <network_id>

Where network ID is the UUID of the network to show the configured QoS policy. This will display the network information. The UUID of the configured QoS policy will be set under the "qos_policy_id" field. If the field is absent or not set, then no QoS policy is set on the network.

 Updating Policies

QoS policies can be updated to change the policy parameters and the parameters of the rules on the policy. A policy must have been created to update it, but it does not have to have active rules or be attached to any port or network in order to update it. To update rules on a policy, the rule must be updated directly, which means bulk updates of rules are not supported.

 Updating General Policy Information

General policy information includes the name of the policy, it’s description and whether or not it' shared across tenants.

To update a policy, simply issue the command:

$ neutron qos-policy-update --name <name> --description <desc> --shared <true|false> <policy_id>

Where name is the name of the policy, desc is an optional short description, shared specifies whether to allow other tenants to access this policy, and policy ID is the UUID of the QoS policy object to update. If a parameter is omitted, it will not be updated and will remain unchanged.

 Updating Rules on a Policy

QoS policies can also have updates to the set of rules that are set on that policy. This is done through an update to the rule on the policy. Rules can only be updated one-by-one. When a rule’s parameters are updated, the change will take effect immediately upon the update.

 Updating a Bandwidth Limit Rule

These commands update the Bandwidth Limit rules for a QoS policy.

$ neutron qos-bandwidth-limit-rule-update --max-kbps <max_kbps> --max-burst-kbps <max_burst_kbps> <rule_id> <policy_id>

Where rule ID is the UUID of the rule to update, policy ID is the UUID of the policy the rule belongs to, max_kbps is the maximum sustained data rate allowed (rate to fill the token bucket in the example above), and max_burst_kbps is the maximum packet size allowed (maximum tokens allowed in the token bucket in the example above).

 Updating a DSCP Marking Rule

These commands update the DSCP marking rules for a QoS policy.

$ neutron qos-dscp-marking-rule-update --dscp-mark <dscp_mark>

Where rule ID is the UUID of the rule to update, policy ID is the UUID of the policy the rule belongs to, and dscp_mark is the value to set in each IPv4 packet’s ToS field.

 Changing Rules on a Policy

Changing which rules are active on a QoS policy basically boils down to either creating or deleting rules directly. You cannot affect the rules on a policy by manipulating the policy object itself, it must be done by targeting the individual rule on a specific policy. This also means that bulk changes of rules on a policy (like deleting many rules at a time, clearing all rules, creating many rules at once) are not supported. Please see the section called “Creating QoS Policy Rules for a Policy” and the section called “Deleting a Rule on a Policy”.

 Deleting QoS Policies

QoS policy objects or individual rules can be deleted. Rules must be deleted individually from a specific QoS policy object. If a QoS policy object is deleted, all rules that belong to that object are also deleted. If a deleted QoS policy is attached to a port or network, that link is cleared and the qos-policy-id field of the port or network object is set to empty.

 Deleting a Rule on a Policy

A single rule on a QoS policy object can be deleted to remove its effects from the policy. Once a rule is deleted from a policy, traffic through any associated ports or networks will cease to be affected by the rule’s parameters immediately after it is deleted.

$ neutron qos-bandwidth-limit-rule-delete <rule_id> <policy_id>

Or:

$ neutron qos-dscp-marking-rule-delete <rule_id> <policy_id>

Where rule ID is the UUID of the rule to delete, policy ID is the UUID of the policy the rule belongs to.

 Deleting an Entire Policy

An entire QoS policy, and any attached rules, can be deleted at a single time with the following commands. Once a policy is deleted, it will immediately cease to have any effect on any network traffic. All rules created on the deleted policy will be cleaned up and removed, and all networks and ports attached to this policy will have their qos-policy-id fields cleared.

$ neutron qos-policy-delete <policy_id>

Where policy ID is the UUID of the policy to delete.

Questions? Discuss on Mailing Lists or Chat.
Found an error? Report a bug.


loading table of contents...