Ask the data plane what it is doing#
The appliance has two views of forwarding. The control plane knows what FRR has learned and what it has programmed; the data plane holds the tables packets are actually forwarded from. They normally agree. This guide is for when you suspect they do not — traffic is being dropped and nothing explains it, a route is in the routing table but traffic does not follow it, or a routing session flaps under load.
Four commands read the data plane's own state. None of them reports what the control plane believes it programmed.
| Question | Command |
|---|---|
| Is the data plane dropping packets, and at which stage? | show dataplane drops |
| What would the data plane do with a packet for this address? | show dataplane route <address> [vrf <name>] |
| Is traffic addressed to the box being rate-limited? | show punt |
| What does the data plane do with this MPLS label? | show dataplane mpls <label> |
You need: a CLI login. The commands change nothing and add nothing to the packet path — each one is answered once, when you ask.
Where are packets being dropped?#
user@bng> show dataplane drops
Forwarding drops by reason (since the data plane started)
Name Packets Meaning
----------- ------- -----------------------------------------------------------------------------------------------------------
total 812 every forwarding-decision drop; the reasons below are subsets of it
urpf 40 source address failed the subscriber's reverse-path check (spoofed, or an address the session does not own)
rate-limit 21 over the subscriber's rate — QoS policing, expected when a subscriber fills their plan
no-session 702 no active session on the subscriber circuit, or (from the core) none owning the destination address
ttl-expired 31 TTL / hop limit ran out here; time-exceeded is returned where possible (traceroute does this)
too-big 6 larger than the egress MTU and not fragmentable (DF set, or IPv6); too-big is returned to the sender
other 12 total minus every reason above (and drops from before a data-plane restart) — the stages below say where
Forwarding drops by pipeline stage
Name Packets Meaning
------------------ ------- -------------------------------------------------------------------------------------------
subscriber-ingress 61 from subscribers toward the core: uRPF, policing, no route, blackhole, empty next-hop group
core-downstream 751 from the core toward subscribers, including traffic for an address no session owns
Outside the forwarding decision (not in total)
Name Packets Meaning
------------------------- ------- -----------------------------------------------------------------------------------
punt-rate 1290 a punt rule's own rate limit discarded traffic for the box (show punt has the rule)
port 1 (ens1f1) rx-missed 37 the NIC dropped on receive: its ring overflowed because the workers fell behind
Counters at zero are not shown; 'show dataplane drops all' lists every one.
All counters are cumulative since the data plane started, so run it twice a few seconds apart and compare — it is the counter that is moving that matters.
Read it in three steps:
- Reasons.
totalcounts every forwarding drop. The named reasons are subsets of it — a dropped packet is counted under one of them, never two — and most are the appliance doing its job:rate-limitis a subscriber at their plan's rate,urpfis a source address the subscriber does not own,no-sessionfrom the core is traffic for an address nobody holds. The table below says what each one means.otheris what no reason accounts for, and on a healthy box it is small. - Stages. The same packets, by where they were when they were dropped:
core-downstreamis traffic arriving from the core,subscriber-ingressis traffic from subscribers. A reason tells you why; the stage tells you which side to look at, and for the few drops that are stillotherit is the lead. - Outside the forwarding decision. These are not in
total.rx-missedon a port means the NIC dropped before the data plane saw the packet — the workers are saturated (GET /api/v1/system/statusreports worker occupancy).punt-rateis covered below.unresolved-next-hopmeans a next hop has no MAC address: check the neighbour and the link. The appliance asks for the address of every next-hop gateway on a core port as soon as a route uses it, and keeps asking while there is no answer, so a count that keeps rising means the gateway is not answering ARP or neighbour discovery; the log names it (FIB sync: next hop … is still unresolved).unresolved-to-kernelis not a drop: a packet whose next hop has no MAC address yet is handed to the kernel, which forwards it and resolves the neighbour. It moves briefly after a data-plane restart, when the kernel's neighbour table starts empty. If it keeps rising while traffic to a directly connected next hop is lost, the kernel is not resolving that neighbour — look atip neighfor the address, and ping it from the appliance.
Rows at zero are left out. show dataplane drops all lists every counter.
What each reason means#
| Reason | What happened | What to do |
|---|---|---|
urpf |
The source address is not one the subscriber's session owns | Usually a misconfigured or compromised CPE. Persistent from one subscriber: look at that session |
rate-limit |
Over the subscriber's rate | Nothing — this is the plan being enforced |
blocked |
The subscriber is hard-blocked | Nothing, if the block is intended |
destination-scope, source-routing |
A subscriber sent to a destination that cannot be routed off-link, or used source routing | Nothing |
mpls-ttl, tunnel-mtu, ip-checksum |
An expired label TTL; an L2TP frame larger than the uplink carries; a corrupt IPv4 header | tunnel-mtu rising: check the uplink MTU |
cgn-untranslatable |
CG-NAT cannot translate the packet: a TCP or UDP source port of 0, or a transport header cut short | The subscriber's device or a test tool is sending packets no NAT can carry. Nothing to fix on the appliance |
cgn-unsupported |
CG-NAT does not translate this kind of packet: a protocol other than TCP, UDP or ICMP (GRE, ESP, SCTP), or an ICMP error for a flow it does not have | Real traffic being lost. A subscriber who needs GRE or IPsec without NAT-T needs a public address, not CG-NAT |
cgn-no-binding |
The subscriber has no active CG-NAT binding, or its IP group is not active | show cgn bindings, show cgn groups — a full pool leaves new subscribers without a binding |
cgn-setup-rate |
The subscriber opened new flows faster than its limit | Expected under a scan or a misbehaving device; otherwise raise the IP group's setups-per-sec |
cgn-capacity |
No room for a new flow | show cgn summary says which: the subscriber's port block (port cap), the per-subscriber flow cap, or a full table |
cgn-hairpin |
Traffic to another subscriber's public address that could not be translated or delivered there | The destination subscriber has gone, or has no mapping for that port |
cgn-no-mapping |
Traffic from the core to a CG-NAT address nothing is mapped to | Unsolicited inbound — background noise on any public pool |
cgn-fragment |
A later fragment of a fragmented packet through CG-NAT, in either direction, whose first fragment the appliance did not translate in the two seconds before it | A few are normal: the first fragment was lost or arrived second. A steady count while customers report failed calls or large UDP transfers means fragments are arriving out of order or in very large numbers; ask the customer to use SIP over TCP or TLS, or to shorten the codec list, so the message fits in one packet |
no-session |
From a subscriber: a frame on a circuit with no active session. From the core: a packet for an address no session owns | From the core this is normally scanning, or a route still announced for a subscriber who has gone. From subscribers: a CPE sending before it has signed on |
malformed |
A header that does not parse, or a frame shorter than its headers say | A steady trickle is normal. A lot from one port: a faulty device or link |
encap-mismatch |
On a PPPoE session's circuit, a frame that is not PPPoE or carries another PPPoE session id | A CPE still using a session the appliance has torn down |
unhandled-protocol |
Not IP or PPPoE, and nothing on the appliance wants it | LLDP, spanning tree and similar from access equipment. Nothing |
ttl-expired |
The packet's TTL ran out here | Traceroute does this. A lot of it: a routing loop — check the route with the next command |
too-big |
Larger than the next link's MTU and not allowed to be fragmented | The sender is told (path-MTU discovery). Rising steadily: an MTU mismatch on the path |
no-route |
No route for the destination in the VRF, or a blackhole route | show dataplane route <address> vrf <name>, next section |
no-next-hop |
The route points at a next-hop group that is empty or unusable | The same command: it lists the group's members |
mpls-label |
A label the data plane has no entry for | show dataplane mpls <label> |
tx-failed |
The frame could not be built or queued for transmit | An egress port that is down or missing. tx-refused and the per-port counters, lower down, say more |
tunnel-peer |
L2TP data for a live session, from an address that is not that tunnel's | Somebody else's traffic, or a LAC that changed address without re-establishing |
subscriber-nd |
An IPv6 Router Advertisement or other Neighbour Discovery message from a customer, sent in a way that would have reached your core | A customer router advertising itself upstream, or something probing. The appliance never forwards these; a steady count from one circuit is worth a look at that customer's equipment |
The reasons from cgn-untranslatable down are kept by the data plane itself and asked for each
time you run the command. If it does not answer, a Note: under the table says so and those
drops are counted in other. They start again from zero when the data plane restarts — total
does not, across an upgrade that keeps subscribers up — so just after a restart other includes
the drops from before it.
CG-NAT is dropping a subscriber's traffic#
show cgn summary ends with the same six cgn- reasons a subscriber's own packets can be
refused for:
user@bng> show cgn summary
Enabled: yes
Running: no (activates at startup)
Active bindings: 0 / 8192
auto-learned: 0
Max flows: 2000000
IP groups: 1
Subscriber packets CG-NAT refused ('show dataplane drops' explains each):
cgn-untranslatable: 0
cgn-unsupported: 0
cgn-no-binding: 0
cgn-setup-rate: 0
cgn-capacity: 0
cgn-hairpin: 0
The counters above that block count capacity — a pool, a port block or a table that ran out.
A subscriber whose traffic is simply not translatable (cgn-untranslatable, cgn-unsupported)
moves none of them, which is why they are listed separately.
For one subscriber, open their session in the web UI (Sessions, then the session). A
translated subscriber's page has a CGNAT box — a subscriber who is not translated has no
such box — showing what show cgn bindings <inside-ip> shows, and a little more:
| Row | What it tells you |
|---|---|
| Inside IP, VRF | The address being translated, and the VRF it is in |
| Public IP | The address the subscriber's traffic leaves from |
| Port blocks | Every block the subscriber holds, each with when it was granted, then how many it holds of how many it may hold. A second or third block means the first ran out |
| Group | The CG-NAT IP group the public address came from |
| Source | session (made with the session) or auto-learned (learned from traffic) |
| Created | When the binding was made |
| New-flow limit | New flows per second the subscriber may open |
| Ports in use | Of the ports its blocks hold. Close to all of them, with every block granted: the subscriber is at its port cap |
| Active translations, Translations created | Flows now (and how many were opened from outside), and since the binding was made |
| Port-cap drops, Setup-rate drops | Packets refused for those two reasons — this subscriber's share of cgn-capacity and cgn-setup-rate |
The rows from Ports in use down come from the data plane and are missing when it is not
reporting. The same figures are in GET /api/v1/sessions/{id} as cgnBindings (null for a
subscriber that is not translated), and
show subscribers sessions <id> prints the binding in brief. The page's Addressing box lists, under
the session's own IPv4, IPv6 and delegated prefix, any Additional routes — prefixes routed
to the subscriber beyond its own addresses.
A route of a session is not installed when the configuration itself routes the same prefix in
the same table (a configured static, an announced pool or CG-NAT aggregate): the configured
route stands. show subscribers sessions <id> then has a Not installed: line under
Routes:, the session detail API lists the prefix in routesNotInstalled, and the log has
ROUTE_INJECT_SKIPPED. The subscriber is still forwarded to; only the route is absent.
What would the data plane do with this destination?#
user@bng> show dataplane route 100.64.12.37 vrf internet
Data-plane lookup: 100.64.12.37 in VRF internet (id 1)
Own address: no
Subscriber address: 100.64.12.37/32 -> session 40213 (slot 0, active), alice@example.net
Subscriber prefix: no
FIB: 0.0.0.0/0 -> next-hop group 12
next-hop group 12: 2 members, source MAC 02:00:00:00:00:01
0: via 10.0.0.1 port 1 (ens3f1) MAC 00:11:22:33:44:0a no labels weight 1
1: via 10.0.0.2 port 1 (ens3f1) MAC 00:11:22:33:44:0b labels 16001 (transport), 24005 (vpn) weight 1
From the core: delivered to subscriber session 40213
From a subscriber: forwarded through next-hop group 12 (one of 2 members per flow)
vrf takes a configured VRF name, default, or a data-plane VRF id; without it the lookup is in
the global table.
Each line is one of the data plane's tables answering for itself. The last two lines say what
those answers add up to, because the answer depends on where the packet came from. Traffic
arriving from the core is delivered to the subscriber session that owns the address; if none
does, what happens depends on the packet's source — a CG-NAT subscriber or a source-VRF rule
routes it onward, anything else is dropped — so a destination lookup can only say that no session
claims it. Traffic from a subscriber is forwarded by the FIB — except that a route which cannot
forward (no usable next hop) hands the packet to the subscriber session that owns the
destination in that VRF, if one does. That is how a subscriber in the global table reaches
another subscriber of the same appliance: a subscriber's own route there is a host route on the
access port with no next hop to resolve, and From a subscriber: reads delivered directly to
subscriber session N. In a named VRF a subscriber's route is not in the FIB at all, so the
packet follows the VRF's routes to the core, which sends it back — unless the route it follows
cannot forward that packet (an uplink down, a next hop not yet resolved), in which case it is
delivered directly there as well. For a next-hop group with some members usable and some not,
the line says what happens to the flows that hash to the unusable ones.
Compare it with what the control plane holds:
| The data plane says | Compare with | If they differ |
|---|---|---|
FIB: <prefix> -> next-hop group N |
show route vrf <name> — the same prefix should be the longest match there |
A route in the routing table but no route here was not programmed: check the FIB sync line of show system status, and fib-writes-refused under show dataplane drops all |
| A member's port, MAC and labels | The route's next hop in show route vrf <name>, and the neighbour's MAC |
UNRESOLVED means the neighbour has not answered ARP/ND. A label that differs from the one the next hop advertised is a disagreement to report |
Subscriber address: … -> session N |
show subscribers sessions |
NOT a session the control plane has is a data-plane entry left behind by a session that has gone. Note the address and session id and report it |
… (slot S, active, BLOCKED) |
The subscriber's status | The data plane is dropping this subscriber's traffic on purpose: the session is blocked |
From a subscriber: dropped: … has no members |
— | The route points at an empty next-hop group: the next hop was withdrawn and the route was not |
From a subscriber: delivered directly to subscriber session N |
show subscribers sessions |
Expected for a subscriber's own address in the global table. The receiving session's block, download rate and MTU apply, as they do to traffic from the core |
A destination that matches a blackhole route is dropped by design; the FIB line says so.
Is traffic for the box being policed?#
Packets addressed to the appliance itself — routing protocols, SSH, subscriber sign-on — are punted to the kernel or the control plane, by rule, under rate limits. A routing session that flaps under load, or subscribers that cannot sign on during a flood, is often a punt limit.
user@bng> show punt
Punt classes (counters since the data plane started)
Class Admitted Refused Rules Budget
---------- -------- ------- ----- ----------------------------------------------------------------------------------------------------------
routing 184166 0 3 20000 pps box-wide
management 9120 0 1 2000 pps box-wide
subscriber 730152 1290 2 per-circuit 100, per-port 5000, per-peer 2000, global 20000, aggregate 20000 pps, divided across 4 workers
default 12044 0 1 per-port 5000, aggregate 20000 pps, divided across 4 workers
Punt rules: 7 of 256 installed, in evaluation order (first match wins)
ID Prio Match To Class Limit Matched Punted Dropped Description
--- ---- ------------------------------------------------------- ------------- ---------- --------- ------- ------ ------- -----------------------
100 100 ipv4 udp dport 67 on subscriber control-plane subscriber 5000 pps 412907 411617 1290 DHCPv4 from subscribers
110 110 pppoe-discovery on subscriber control-plane subscriber 5000 pps 318535 318535 0 PPPoE discovery
690 690 ipv4 tcp dport 179 from 192.0.2.2/32 to-self on network kernel routing - 18211 18211 0 BGP from 192.0.2.2
700 700 ipv4 tcp dport 179 to-self on network drop routing - 37 0 37 BGP from anyone else
850 850 arp on network kernel routing 5000 pps 165955 165955 0 ARP (core)
880 880 ipv4 tcp dport 22 to-self on network kernel management 2000 pps 9120 9120 0 SSH
900 900 ipv4 to-self kernel default 10000 pps 12044 12044 0 IPv4 to self
Class is where a match is charged when it arrives on a network port; anything from a
subscriber port is charged to subscriber. Refused includes a rule's own limit.
- Classes each have their own budget, so a flood in one cannot starve another. A
Refusedcount that is rising onroutingmeans routing protocols are losing packets to policing — thedp.punt.routing.refusedalarm is raised for the same condition. Raise the budget withset dataplane control-punt-limits routing-pps, or find what is flooding the core side. - Rules are listed in the order the data plane evaluates them; the first match wins. A rule
whose
Droppedis rising has reached its ownLimit. A rule withTo=dropis a filter: its matches are discarded on purpose (for example BGP from an address that is not a configured peer). - A rule you expect and do not see is not installed: the data plane will not punt that traffic, whatever the configuration says.
MPLS labels#
user@bng> show dataplane mpls 24010
Data-plane label 24010: pop, then forward the inner packet in VRF business (id 1)
A label that is not installed is dropped. For an L3VPN the label FRR advertised for a VRF
(show route vrf <name> on the remote PE) must appear here as pop into that VRF.
If the command cannot ask#
| Message | Meaning |
|---|---|
the running data plane predates this command |
The control plane was upgraded and the data plane has not been restarted since. The data plane restarts with the next upgrade or systemctl restart bng-dataplane — plan it: a data-plane restart interrupts forwarding |
the data plane did not answer |
The data plane is not running, or is not servicing commands. Check systemctl status bng-dataplane on the host |
no data plane to ask |
A development build running the mock data plane |
From the API#
The same answers, as JSON: GET /api/v1/dataplane/punt, /drops, /route?destination=…&vrf=…
and /mpls/{label} — see the System section of the REST API reference.
At the CLI, append | display json to any of the commands above.