Extensions to Compress and Derive Fields in HTTP Datagrams

3.2.1. Templates

If the peer advertises the max-templates value greater than 0, the endpoint MAY create template contexts up to that limit using capsules defined in Section 4.¶

An endpoint MUST NOT create templates exceeding the peer's advertised max-template-segments limit when that parameter is present.¶

If the peer advertises an mtu limit, the sender MUST NOT transmit a datagram that would reconstruct into a packet larger than the advertised limit after all processing contexts have been applied.¶

This limit applies whether the datagram is carried in a QUIC DATAGRAM frame or a DATAGRAM Capsule, including when no processing context is used.¶

3.2.2. Derived Fields

An endpoint MAY create a derived context only if every operation in the capsule appears in the peer's derived list.¶

3.2.3. Checksum Offload

An endpoint MAY create a checksum offload context only if the peer advertised checksum=?1.¶

4.1.1. Processing Context Construction

Processing contexts are created using capsules that define a new unique non-zero Context ID encoded as a variable-length integer. A Context ID MUST NOT be reused, including one allocated by another mechanism on the same HTTP request. As specified in Section 4 of [CONNECT-UDP], even-numbered Context IDs are allocated by the client and odd-numbered by the proxy.¶

Each processing context references a next context using Next Context ID encoded as a variable-length integer. This MUST identify Context ID 0, a previously defined Processing Context, or a context already established by another mechanism. The referenced context MUST be available for use in the same sending direction. Forward references are not permitted. If Next Context ID identifies a Processing Context, the chain continues through that context. Otherwise, it identifies the Terminal Context and ends the processing chain.¶

A processing chain MUST contain at most one Template context (created by TEMPLATE_ASSIGN), at most one Derived Field context (created by DERIVED_ASSIGN), and at most one Checksum Offload context (created by CHECKSUM_ASSIGN). Other processing context chains can also have invalid combinations, if they include contradictory or mutually exclusive steps. Any capsule definition used for these contexts needs to define such cases if they are possible (such as for checksum offload contexts, Section 4.4). A receiver that detects such an invalid processing chain condition MUST treat the context as malformed and follow the error-handling procedure defined in Section 3.3 of [HTTP-DATAGRAMS].¶

A receiver of an *_ASSIGN capsule with an invalid Context ID or unknown Next Context ID MUST treat it as malformed and follow the error-handling procedure defined in Section 3.3 of [HTTP-DATAGRAMS].¶

4.1.2. Processing Context Acknowledgement

For each *_ASSIGN capsule received, the receiver MUST transmit the corresponding *_ACK capsule after successfully installing the context.¶

Endpoints MAY transmit datagrams referencing contexts prior to receiving the *_ACK. A receiver MAY buffer datagrams referencing unknown Context IDs but MUST bound buffering by time and memory.¶

A receiver of an *_ACK capsule with an unknown Context ID, a Context ID not assigned by the corresponding *_ASSIGN capsule, or any data after Context ID MUST treat it as malformed and follow the error-handling procedure defined in Section 3.3 of [HTTP-DATAGRAMS].¶

4.1.3. Processing Context Closure

Processing Contexts are retired by sending corresponding *_CLOSE capsule. Closing a context implicitly closes all contexts that reference it directly or transitively.¶

Closing a Terminal Context defined by another mechanism also closes all Processing Contexts that reference it directly or transitively. The closure and any retention of the Terminal Context are governed by the mechanism that defined it. Closing a Processing Context does not close its Terminal Context.¶

A receiver of a *_CLOSE capsule SHOULD retain the closed context and its descendants for a short period to allow in-flight datagrams to arrive, but MUST bound the retention time and memory usage.¶

*_CLOSE capsules with an unknown Context ID, a Context ID not assigned by the corresponding *_ASSIGN capsule, or any data after Context ID MUST be treated as malformed. Receiver of such capsules MUST follow the error-handling procedure defined in Section 3.3 of [HTTP-DATAGRAMS].¶

4.2.1. TEMPLATE_ASSIGN Capsule

TEMPLATE_ASSIGN Capsule {
  Type (i) = 0x3ee3143f,
  Length (i),
  Context ID (i),
  Next Context ID (i),
  Static Segment (..) ...
}

The TEMPLATE_ASSIGN capsule contains a sequence of one or more Static Segments.¶

Static Segment {
  Segment Offset (i),
  Segment Length (i),
  Segment Payload (..),
}

Each Static Segment contains following fields:¶

Segment Offset:

Byte offset from the start of the reconstructed packet, encoded as a variable-length integer¶

Segment Length:

Length of the Segement Payload field, encoded as a variable-length integer¶

Segment Payload:

Static bytes to insert at the Segment Offset¶

4.2.2. TEMPLATE_ACK Capsule

TEMPLATE_ACK Capsule {
  Type (i) = 0x3ee31440,
  Length (i),
  Context ID (i),
}

Processing of the TEMPLATE_ACK capsule is described in Section 4.1.2¶

4.2.3. TEMPLATE_CLOSE Capsule

TEMPLATE_CLOSE Capsule {
  Type (i) = 0x3ee31441,
  Length (i),
  Context ID (i),
}

Processing of the TEMPLATE_CLOSE capsule is described in Section 4.1.3¶

4.3.1. DERIVED_ASSIGN Capsule

DERIVED_ASSIGN Capsule {
  Type (i) = 0x3ee31442,
  Length (i),
  Context ID (i),
  Next Context ID (i),
  Derived Field Type (i) ...
}

The DERIVED_ASSIGN capsule defines a processing context that generates and inserts one or more derived fields into the reconstructed packet. The sender does not transmit these fields in the datagram payload.¶

The Derived Field Types defined by this document and their reconstruction procedures are specified in Section 6. The numeric code points for these field types are registered in Section 9.3.¶

4.3.2. DERIVED_ACK Capsule

DERIVED_ACK Capsule {
  Type (i) = 0x3ee31443,
  Length (i),
  Context ID (i),
}

Processing of the DERIVED_ACK capsule is described in Section 4.1.2¶

4.3.3. DERIVED_CLOSE Capsule

DERIVED_CLOSE Capsule {
  Type (i) = 0x3ee31444,
  Length (i),
  Context ID (i),
}

Processing of the DERIVED_CLOSE capsule is described in Section 4.1.3¶

4.4.1. CHECKSUM_ASSIGN Capsule

CHECKSUM_ASSIGN Capsule {
  Type (i) = 0x3ee31445,
  Length (i),
  Context ID (i),
  Next Context ID (i),
  Checksum Field Offset (i),
  Checksum Start Offset (i),
}

The CHECKSUM_ASSIGN capsule defines a processing context that completes an Internet checksum for the reconstructed packet using a sender-provided partial checksum.¶

In addition to Context ID and Next Context ID CHECKSUM_ASSIGN capsule contains following fields encoded as variable-length integers:¶

Checksum Field Offset:

Byte offset of the 16-bit Internet checksum field within the reconstructed packet¶

Checksum Start Offset:

Byte offset where checksum coverage begins. Coverage runs from this offset to the end of the reconstructed packet.¶

4.4.2. CHECKSUM_ACK Capsule

CHECKSUM_ACK Capsule {
  Type (i) = 0x3ee31446,
  Length (i),
  Context ID (i),
}

Processing of the CHECKSUM_ACK capsule is described in Section 4.1.2¶

4.4.3. CHECKSUM_CLOSE Capsule

CHECKSUM_CLOSE Capsule {
  Type (i) = 0x3ee31447,
  Length (i),
  Context ID (i),
}

Processing of the CHECKSUM_CLOSE capsule is described in Section 4.1.3¶

5.1.1. Template Contexts

If the selected context chain contains a Template context, the sender constructs the datagram payload as the concatenation of all variable byte regions not covered by static segments.¶

Variable regions are emitted in strictly increasing offset order starting at offset 0.¶

If the context chain contains no Template context, the payload MUST be the complete packet.¶

5.1.2. Derived Field Contexts

Derived fields are not transmitted by the sender. When a derived context is in use, the sender MUST remove the octets corresponding to derived fields from the datagram payload. These octets are supplied by the receiver during reconstruction.¶

The sender MUST construct the payload as if the derived field octets were not part of the variable regions. That is, the payload MUST contain only the remaining variable octets in strictly increasing offset order.¶

5.1.3. Checksum Offload Contexts

If the context chain contains a checksum offload context, the sender MUST place a precomputed partial Internet checksum value into the checksum field at Checksum Field Offset in the reconstructed packet image prior to transmission. This value is combined with the receiver computation as described in Section 5.2.¶

The partial value MUST be the 16-bit one's-complement sum of the checksum contributions not included in the receiver's checksum coverage, such as the pseudo-header. The sender MUST fold any carries into the low 16 bits without complementing the sum and store the value in network byte order.¶

Checksum offload reduces per-packet CPU cost by allowing the sender to preserve a packet's partial checksum instead of completing the checksum before encapsulation. The partial value includes the pseudo-header contribution and is carried in the variable portion of the datagram, with Checksum Start Offset pointing to the transport header. After any template reconstruction and derived field processing, the receiver can delegate checksum completion to a capable NIC [LINUX-CHECKSUM-OFFLOAD] or pass the reconstructed packet with checksum-offload metadata through an operating-system interface, such as Linux TUN/TAP [LINUX-TUN-TAP].¶

5.1.4. Context Selection

If a packet does not match any available Processing Context, the sender MUST use the appropriate Terminal Context ID and transmit the complete payload required by that context, subject to the size limits and oversized-packet handling described in Section 5.1.5.¶

To reduce variation in compression across packets, senders SHOULD define broad contexts that will be able to handle most or all expected traffic.¶

5.1.5. MTU Considerations

The advertised mtu bounds the reconstructed packet size. A packet that would not fit in a QUIC DATAGRAM frame without compression can still be sent if compression makes it fit. The transport size checks apply after compression: the resulting QUIC DATAGRAM frame MUST respect the peer's frame size limit, and the outer IP packet, including HTTP/3, QUIC, UDP, and IP overhead, MUST fit within the current path MTU. Even packets below the advertised mtu can exceed these transport limits when compression is insufficient or no matching context is available.¶

Compression can differ between packets of the same size, including between PMTU probes and other traffic. A successful probe therefore does not establish that all packets of that original size will fit. PMTU information can also be shared across flows that use different contexts; see the discussion of shared PLPMTU state in item 9 of Section 3 of [DPLPMTUD]. When choosing a tunnel MTU or reporting a PMTU to carried traffic, endpoints SHOULD account for the least compression available to the traffic sharing that MTU, including packets sent without compression, rather than assuming the savings of a particular context.¶

Oversized packets are handled according to the underlying HTTP Datagram protocol's MTU and error-signalling rules. In particular, Section 10.1 of [CONNECT-IP] and Section 6.1 of [CONNECT-UDP] recommend dropping oversized packets and sending ICMP Packet Too Big feedback instead of switching to DATAGRAM Capsules. Where permitted by the underlying protocol, DATAGRAM Capsules can carry larger datagrams across multiple transport packets, but introduce reliable delivery, head-of-line blocking, additional buffering and retransmission costs, and possible reordering relative to QUIC DATAGRAM frames. Such delivery can hide size limitations from PMTU discovery for the carried traffic; it does not increase the path MTU or relax the advertised mtu limit.¶

5.1.6. Conservative Template Composition

Reusable templates are most effective when they remain valid for many datagrams. This guidance applies to the packets using a particular template. A template can be used for a subset of the HTTP Datagrams associated with an HTTP request. Senders SHOULD construct templates conservatively by including only fields that are expected to remain stable across applicable packets for the lifetime of the Processing Context. Fields whose values can change across packets SHOULD remain in the variable portion of the datagram unless the sender has specific knowledge that those fields will remain constant or be particularly frequent for some period of time.¶

Fields that are commonly suitable for static segments include:¶

• Ethernet source and destination addresses, when stable for the frame sequence¶

• EtherType¶

• 802.1Q tag¶

• IP version¶

• Source and destination IP addresses¶

• IPv4 TTL / IPv6 Hop Limit¶

• IPv4 Protocol / IPv6 Next Header¶

• Source and destination transport ports¶

Fields that commonly vary between packets, or that can be modified by endpoints or network devices, are generally poor candidates for static segments. Examples include:¶

• IPv4 DSCP and ECN bits, and IPv6 Traffic Class¶

• IPv4 Identification¶

• IPv4 Flags and Fragment Offset, unless packets using the template are known not to be fragmented¶

• IPv6 Flow Label, unless the sender knows it has the same value in all packets using the template¶

• TCP Sequence Number¶

• TCP Acknowledgement Number¶

• TCP Flags¶

• TCP Window¶

• TCP Timestamp values¶

• Packet length fields¶

• Checksum fields¶

When a field can be represented as a Derived Field defined by this specification, implementations SHOULD prefer using the corresponding Derived Field instead of including that field in a static segment or transmitting it as part of the variable payload. Packet length fields and complete checksums are examples of fields that are generally better represented using Derived Fields when supported by the peer.¶

The suitability of a field for inclusion in a static segment depends on the deployment and the traffic being carried. For example, some deployments might preserve DSCP values while others rewrite them, and some applications might intentionally vary the IPv6 Flow Label. Implementations are encouraged to prefer templates that remain valid across many datagrams, even if those templates omit fewer bytes than a more aggressive template.¶

5.2.1. Template Reconstruction

If a Template context is present, the receiver reconstructs the packet as follows:¶

1. Allocate a buffer large enough to contain the reconstructed packet.¶

2. Insert static segment bytes at their specified offsets.¶

3. Fill all remaining gaps using bytes from the datagram payload in strictly increasing offset order.¶

If payload bytes are exhausted before all gaps have been filled the datagram MUST be dropped.¶

Packets larger than the advertised mtu in http-datagram-contexts MUST be dropped.¶

5.2.2. Derived Field Processing

For each derived field present in the context chain, the receiver computes the field value and inserts it into the reconstructed packet at the location defined by the derived field type.¶

Derived fields are inserted into the packet image and therefore increase the reconstructed packet size. The receiver MUST compute derived field values based on the final reconstructed packet size and structure.¶

Initial field definitions and reconstruction procedures are specified in Section 6.¶

If the required header cannot be located, the packet MUST be dropped.¶

5.2.3. Checksum Offload Processing

If a checksum offload context is present, the receiver completes the Internet checksum after all derived fields have been inserted.¶

The receiver MUST complete the checksum as follows, using one's-complement arithmetic as described in [INTERNET-CHECKSUM]:¶

1. Read and retain the 16-bit partial sum from the checksum field in network byte order.¶

2. Compute the one's-complement sum of the octets from Checksum Start Offset to the end of the reconstructed packet, treating the checksum field as zero for this computation. Pair octets into 16-bit words in network byte order; if the number of octets is odd, pad the final octet with a zero octet for the computation only.¶

3. Add the retained partial sum to this sum using end-around carry: repeatedly add any bits above the low 16 bits back into the low 16 bits until no carry remains.¶

4. Take the bitwise one's complement of the resulting 16-bit sum and write it to the checksum field in network byte order. For a UDP checksum, encode a zero result as 0xffff.¶

If any offset exceeds the reconstructed packet length, the packet MUST be dropped.¶

6.1.1. Sender Behavior

Before using a Derived Field Type, the sender MUST perform the applicable header checks in Section 6.2 and any additional checks in the field's definition. If a required header or field cannot be located, or an applicability check fails, the sender MUST NOT use that Derived Field Type for the packet.¶

The sender MUST verify that the original field value equals the value that the receiver would generate using the computation in the field's definition for the reconstructed packet. This comparison includes any specified encoding of special values, such as encoding a zero UDP checksum as 0xffff. If the values differ, the sender MUST NOT use that Derived Field Type for the packet.¶

When using a Derived Field Type, the sender removes the corresponding field octets from the datagram payload.¶

6.1.2. Receiver Behavior

The receiver MUST perform the applicable header checks in Section 6.2 and the field-specific checks below. If the packet is too short to contain a required header or field, a required header cannot be located, a validation check fails, or the computed value is outside the range specified for the field, the receiver MUST drop the packet.¶

The receiver computes and inserts length fields before checksum fields, using the final reconstructed packet size and structure as described in Section 5.2. In particular, IPv4 Total Length, IPv6 Payload Length, and UDP Length are computed before IPv4 header, TCP, or UDP checksums. The receiver writes each computed value at the location specified in the field's definition.¶

6.2.1. IP Headers

For CONNECT-IP, the IP header starts at offset 0. For CONNECT-ETHERNET, the IP- and transport-layer Derived Field Types apply to Ethernet II frames carrying IPv4 or IPv6, including frames with a single IEEE 802.1Q VLAN tag:¶

These offsets are measured from the beginning of the Ethernet frame. For the tagged form, the Tag Protocol Identifier (TPID) at offset 12 has value 0x8100. Frames with multiple VLAN tags or other link-layer encapsulations are not processed by these Derived Field Types unless a future specification defines how the relevant network-layer header is located.¶

The sender and receiver MUST locate the IP header and verify that its first four bits contain the version required by the Derived Field Type: 4 for IPv4 [IPv4] or 6 for IPv6 [IPv6]. For CONNECT-ETHERNET, the EtherType MUST also identify that version: 0x0800 for IPv4 or 0x86dd for IPv6.¶

For all IPv4 Derived Field Types, the receiver MUST verify that the Internet Header Length (IHL) field indicates a header length of at least 20 octets and no larger than the reconstructed IPv4 packet. The sender's IHL check for IPv4 Header Checksum is specified in Section 6.6.¶

6.2.2. Transport Headers

The following checks apply to UDP Length, TCP Checksum, and UDP Checksum. The required transport protocol and minimum header size are:¶

For an IPv4 variant, the IPv4 Protocol field MUST identify the required transport protocol. The packet MUST NOT be fragmented: the More Fragments flag and Fragment Offset field MUST both be zero. The transport header starts immediately after the IPv4 header, whose length is determined by IHL.¶

For an IPv6 variant, the sender and receiver MUST locate the required transport header by traversing the IPv6 Next Header chain from the fixed 40-octet IPv6 header. The traversal follows extension headers that identify the following header using a Next Header field and whose length can be determined from the packet contents. If traversal encounters an IPv6 Fragment header before the required transport header, or a header whose following header cannot be determined, the required header is considered not located.¶

The IP packet MUST contain at least the minimum header size at the resulting transport header offset. For TCP, the Data Offset field MUST indicate a header length of at least 20 octets and no larger than the TCP segment contained in the IP packet. The TCP header length is the Data Offset value multiplied by 4.¶

6.5.1. UDP Length over IPv4

The ipv4-udp-length type applies this procedure to UDP carried over IPv4.¶

6.5.2. UDP Length over IPv6

The ipv6-udp-length type applies this procedure to UDP carried over IPv6. Octets preceding the UDP header, including IPv6 extension headers, are not included in the derived length.¶

6.7.1. TCP Checksum over IPv4

The ipv4-tcp-checksum type applies this procedure to TCP carried over IPv4, using the IPv4 pseudo-header.¶

6.7.2. TCP Checksum over IPv6

The ipv6-tcp-checksum type applies this procedure to TCP carried over IPv6, using the IPv6 pseudo-header.¶

6.8.1. UDP Checksum over IPv4

The ipv4-udp-checksum type applies this procedure to UDP carried over IPv4, using the IPv4 pseudo-header. It always generates a UDP checksum. The sender MUST NOT use this Derived Field Type for IPv4 UDP packets that use a UDP Checksum field value of zero to indicate that no checksum is present.¶

6.8.2. UDP Checksum over IPv6

The ipv6-udp-checksum type applies this procedure to UDP carried over IPv6, using the IPv6 pseudo-header.¶