With everything becoming virtual, we get more tools, but that means more things to remember. These are my notes for a capture from an ESXi host.
First lets look capture at the uplink, we need to find which port:
esxcfg-nics -lWhich gives us something like this:
Name PCI Driver Link Speed Duplex MAC Address MTU Description
vmnic0 0000:02:00.0 r8168 Up 1000Mbps Full 00:01:2e:6e:1c:2c 9000 Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
vmnic1 0000:03:00.0 r8168 Down 0Mbps Half 00:01:2e:6e:1c:2d 9000 Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
Now we can see which nics we have and their links state, and here is a capture for both direction and display to the screen
pktcap-uw --uplink vmnic0 --dir 2I may want to do a single VM and save to a file, lets find which VM/switch ports are here:
net-stats -lWhich gives us something like this:
PortNum Type SubType SwitchName MACAddress ClientName
33554434 4 0 vSwitch0 00:01:2e:6e:1c:2c vmnic0
33554436 5 9 vSwitch0 00:0c:29:b9:83:b4 PAN1
33554438 5 9 vSwitch0 00:0c:29:ad:31:64 U1804
50331650 4 0 vSwitch1 00:01:2e:6e:1c:2d vmnic1
50331652 5 9 vSwitch1 00:0c:29:b9:83:be PAN1
50331654 5 9 vSwitch1 00:0c:29:ad:31:78 U1804
67108866 3 0 Internal 00:01:2e:6e:1c:2d vmk0
67108867 5 9 Internal 00:0c:29:b9:83:c8 PAN1
67108868 5 9 Internal 00:0c:29:b9:83:aa PAN1
67108869 5 9 Internal 00:0c:29:b9:83:a0 PAN1
67108870 5 9 Internal 00:0c:29:b9:83:96 PAN1
67108875 5 9 Internal 00:0c:29:ad:31:6e U1804
83886082 5 9 vwire1 00:0c:29:b9:83:dc PAN1
83886083 5 9 vwire1 00:0c:29:b9:83:d2 PAN1
100663298 5 9 vwire2 00:0c:29:b9:83:f0 PAN1
100663299 5 9 vwire2 00:0c:29:b9:83:e6 PAN1
Now lets get the packets coming out of a VM and auto stop at 100MB (could also do -c 1000 to stop at 1000 packets, I prefer to have something that keeps us from filling up the disk)
pktcap-uw --switchport 83886082 --dir 0 -o /vmfs/volumes/LocalSSD/PAN1-vwire1.pcap -C 100Now I can download it and look at it with Wireshark
And use –help to get extra info like using filters, but it may be easier to look at in a browser:
Packet Capture and Trace command usage:
== Create session to capture packets ==
pktcap-uw [--capture <capture point>[,capture point]...
| [--dir <0|input|1|output|2|inputAndOutput>]
[--stage <0|pre|1|post>] [-K|--kernelside]]
[--switchport <PortID> | --vmk <vmknic> | --uplink <vmnic> |
--fcport [fastpathSwitchID.]<PortID> |
--dvfilter <filter name> |
--overlay <overlay class name>]
[--lifID <lif id for vdr>]
[-f [module name.]<function name>[:offset]]
[-AFhP] [-p|--port <Socket PORT>]
[-c|--count <number>] [-s|--snapLen <length>]
[-G <seconds>]
[-C <file_size>]
[-o|--outfile <FILE>] [--console]
[Flow filter options]
== Create session to trace packets path ==
it can trace up to 32 level from pkt allocation and record up to 12 IOChain or portID
pktcap-uw --trace
[-AFhP] [-p|--port <Socket PORT>]
[-c|--count <number>] [-s|--snapLen <length>]
[-G <seconds>]
[-C <file_size>]
[-o|--outfile <FILE>] [--console]
[Flow filter options]
The command options:
-p, --port <Socket PORT>
Specify the port number of vsocket server.
-G, <seconds>
Specify the seconds to rotate the outfile.
-C, <file_size MB>
-o, --outfile <FILE>
Specify the file name to dump the packets. If unset,
output to console by default. If '-', then stdout is used.
-P, --ng (only working with '-o')
Using the pcapng format to dump into the file.
--console (by default if without '-o')
Output the captured packet info to console.
-s, --snaplen <length>
Only capture the first <length> packet buffer.
The minimum snap length is 24 bytes. However, setting
snaplen to 0 will capture entire packet.
-c, --count <NUMBER>
How many count packets to capture.
-h
Print this help.
-A, --availpoints
List all capture points supported.
-F
List all dynamic capture point functions supported.
-4
Capture only IPv4 Packet.
-6
Capture only IPv6 Packet.
--capture <capture point>
Specify the capture point. Use '-A' to get the list.
If not specified, will select the capture point
by --dir and --stage setting
The switch port options:
(for Port, Uplink and Etherswitch related capture points)
--switchport <port ID>
Specify the switch port by ID
--fcport [fastpathSwitchID.]<port ID>
Specify the fastpath port by ID
--lifID <lif ID>
Specify the logical interface id of VDR port
--vmk <vmk NIC>
Specify the switch port by vmk NIC
--uplink <vmnic>
Specify the switch port by vmnic
The capture point auto selection options without --capture:
--dir <0|input|1|output|2|inputAndOutput> (for --switchport, --vmk, --uplink, --fcport)
The direction of flow, with respect to the vswitch:
0- Input: to vswitch (Default), 1- Output: from vswitch, 2- Input and Output
--stage <0|pre|1|post> (for --switchport, --vmk, --uplink, --dvfilter, --overlay)
The stage at which to capture: 0- Pre: before, 1- Post: after
--kernelside (for --uplink)
The capture point is in kernel instead of in driver.
This option is always true and no longer required.
The capture point options
-f [module name.]<function name>[[:offset]|[:line number]]
The function name and the offset/line number in the function.
The default module name is 'vmkernel'.
The default offset into the function is 0 (the beginning of the function).
The line number must start with letter 'L' if it's file related
line number, or 'F' for function related line number.
(for 'Dynamic', 'IOChain' and 'TcpipDispatch' capture points)
--dvfilter <filter name>
Specify the dvfilter name for DVFilter related points
--overlay <overlay class name>
Specify the overlay class name for 'Overlay' capture point
--pkt-list-param-idx <num>
Specify the pktList index in function's parameter table, starting from 1
--pkt-param-idx <num>
Specify the pkt index in function's parameter table, starting from 1
--port-id-param-idx <num>
Specify the portId index in function's parameter table, starting from 1
--switchport option is required to filter packets by port ID
--mbuf-param-idx <num>
Specify the mbuf index in function's parameter table, starting from 1
--mbuf-array-param-idx <num>
Specify the mbuf array index in function's parameter table, starting from 1
--num-mbufs-param-idx <num>
Specify the mbuf array size index in function's parameter table, starting from 1
--vprobe-pkt-list <string>
If function offset is specified in '-f' argument, this is the vprobe
expression to access the pointer to pktList, e.g. "rax+18"
If file/function relative line number is specified in '-f' argument,
this is the C expresstion to access pktList, all variables must start
with '$' sign, e.g. "$pktListPtr"
--vprobe-pkt <string>
Vprobe/C expression to access the pointer to pkt, e.g. "rax+18"
or "$pkt", see argument --vprobe-pkt-list for details.
--vprobe-port-id <string>
Vprobe/C expression to access switch port id, e.g. "rax+18"
or "$portID", see argument --vprobe-pkt-list for details.
--vprobe-mbuf <string>
Vprobe expression to access mbuf, e.g. "rax+18"
or "$mbufPtr", see argument --vprobe-pkt-list for details.
--vprobe-mbuf-array <string>
Vprobe expression to access mbuf array, e.g. "rax+18"
or "$mbufArray", see argument --vprobe-pkt-list for details.
--vprobe-num-mbufs <string>
Vprobe expression to access number of mbufs in the array, e.g. "rax+18"
or "$numMbufs", see argument --vprobe-pkt-list for details.
--vprobe-symdbs <directory>
The parent directory of the symbol database.
--no-vprobe
Don't emit vprobe script to assist packet capture, the capture
function itself has dynamic capture point inserted
--compile-only
Print the vprobe script to standard output and exit
--script <FILE>
Start vprobe with the input .emt file to filter packets
Flow filter options, it will be applied when set:
--srcmac <xx:xx:xx:xx:xx>
The Ethernet source MAC address.
--dstmac <xx:xx:xx:xx:xx>
The Ethernet destination MAC address.
--mac <xx:xx:xx:xx:xx>
The Ethernet MAC address(src or dst).
--ethtype 0x<ETHTYPE>
The Ethernet type. HEX format.
--vlan <VLANID>
The Ethernet VLAN ID, one of 0-4095.
--srcip <x.x.x.x[/<range>]>
The source IPv4/IPv6 address.
--dstip <x.x.x.x[/<range>]>
The destination IPv4/IPv6 address.
--ip <x.x.x.x>
The IPv4/IPv6 address(src or dst).
--proto 0x<IPPROTYPE>
The IPv4/IPv6 protocol.
--srcport <SRCPORT>
The TCP source port.
--dstport <DSTPORT>
The TCP destination port.
--tcpport <PORT>
The TCP port(src or dst).
--srcudpport <SRCPORT>
The UDP source port.
--dstudpport <DSTPORT>
The UDP destination port.
--udpport <PORT>
The UDP port(src or dst).
--vni <vni>
The VNI of flow, one of 0-16777215.
--vxlan <vxlan id>
The vxlan id of flow. This option is depreciated, use 'vni' instead.and this is the list for –capture
Supported capture points:
1: Dynamic -- The dynamic inserted runtime capture point.
2: UplinkRcv -- Function to Rx packets from uplink at driver side (obsoleted).
3: UplinkSnd -- Function to Tx packets on uplink at driver side (obsoleted).
4: VnicTx -- Function in vnic backend to Tx packets from guest
5: VnicRx -- Function in vnic backend to Rx packets to guest
6: PortInput -- Port_Input function of any given port
7: IOChain -- The virtual switch port iochain capture point.
8: EtherswitchDispath -- Function that receives packets for switch
9: EtherswitchOutput -- Function that sends out packets, from switch
10: PortOutput -- Port_Output function of any given port
11: TcpipDispatch -- Tcpip Dispatch function
12: PreDVFilter -- Before DVFIlter capture point
13: PostDVFilter -- After DVFilter capture point
14: Drop -- Dropped Packets capture point
15: VdrRxLeaf -- The Leaf Rx IOChain for VDR
16: VdrTxLeaf -- The Leaf Tx IOChain for VDR
17: VdrRxTerminal -- Terminal Rx IOChain for VDR
18: VdrTxTerminal -- Terminal Tx IOChain for VDR
19: PktFree -- Packets freeing point
20: TcpipRx -- TcpipRX function
21: TcpipTx -- TcpipTX function
22: UplinkRcvKernel -- The function that receives packets from uplink dev at kernel side
23: UplinkSndKernel -- Function to Tx packets on uplink at kernel side
24: PreOverlayInput -- Before overlay input callback
25: PostOverlayInput -- After overlay input callback
26: PreOverlayOutput -- Before overlay output callback
27: PostOverlayOutput -- After overlay output callback
28: EnsPortReaderRx -- Read mbufs from an ENS port
29: EnsPortWriterTx -- Write mbufs to an ENS port
30: EnsPortWriterQueue -- Queue mbufs to an ENS port
31: EnsPortWriterFlush -- Flush mbufs on an ENS port