Chapter 27. Flow State Storage

Flow state is flow-scoped data that agents need to query when processing a packet to make decisions that depend on other previously seen packets belonging to the same connection.

These previous packets may belong to the same flow (have the same or equivalent flow match) or they may belong to traffic that flows in the opposite direction within the same connection.

We call this stateful packet processing. And we call these pieces of state state keys, because not only do they keep state of the flow, but are globally unique across all of MidoNet.

Some examples of such state keys are:

  • NAT mappings. Which keep record network/transport address translations that a virtual device may perform on a connection.
  • Connection tracking keys. Which tell MidoNet’s virtual devices whether a packet belongs to a "forward flow" (the direction of traffic that initiated a particular connection) or to a "return flow" (the opposite direction).

Because packets belonging to the same connection may be simulated at different hosts, MidoNet Agents need a mechanism to share this data. MidoNet Agents push this state to each other proactively over the underlay network but the details are out of scope of this document. We refer the reader to the development documentation for a more detailed explanation of how this flow state is shared among MidoNet Agents.

Even though this state keys are pushed among MidoNet Agents and kept in in-memory volatile tables, there are a couple of corner cases that require these keys to be pushed to durable storage: i) after an agent reboot, in which case the state keys will be lost; and ii) upon a port migration, in which case the new owner will be missing the state keys for connections that were active on that port before the migration occurred.

Starting in MidoNet v5.2.1, these flow state keys are stored on disk locally on each agent instead of storing them to Cassandra, which is now considered optional and not necessary for the normal operation of the MidoNet Agents.

Among other benefits, storing flow state locally simplifies the operation of MidoNet as operators don’t need to configure and monitor a separate Cassandra cluster.

Questions? Discuss on Mailing Lists or Chat.
Found an error? Report a bug.

loading table of contents...