* WGs marked with an * asterisk has had at least one new draft made available during the last 5 days

Ticket #178 (closed design: fixed)

Opened 5 years ago

Last modified 3 years ago

Content-MD5 and partial responses

Reported by: mnot@pobox.com Owned by: julian.reschke@gmx.de
Priority: normal Milestone: 14
Component: p3-payload Severity: Active WG Document
Keywords: Cc:
Origin: http://www.w3.org/mid/D689800C-3AD4-44A8-B5C6-CB8DF6D786F9@mnot.net

Description

After a quick look, my reading is that a Content-MD5 header on a partial response reflects the bytes in that message, rather than the whole (non-partial) response:

The entity-header field "Content-MD5", as defined in [RFC1864], is an MD5 digest of the entity-body for the purpose of providing an end-to- end message integrity check (MIC) of the entity-body. (Note: a MIC is good for detecting accidental modification of the entity-body in transit, but is not proof against malicious attacks.)

Content-MD5 = "Content-MD5" ":" OWS Content-MD5-v Content-MD5-v = <base64 of 128 bit MD5 digest as per [RFC1864]>

The Content-MD5 header field MAY be generated by an origin server or client to function as an integrity check of the entity-body. Only origin servers or clients MAY generate the Content-MD5 header field; proxies and gateways MUST NOT generate it, as this would defeat its value as an end-to-end integrity check. Any recipient of the entity-body, including gateways and proxies, MAY check that the digest value in this header field matches that of the entity-body as received.

The MD5 digest is computed based on the content of the entity-body, including any content-coding that has been applied, but not including any transfer-encoding applied to the message-body. If the message is received with a transfer-encoding, that encoding MUST be removed prior to checking the Content-MD5 value against the received entity.

Note that a multipart message is allowed to have C-MD5 on individual parts; The entity-body for composite types MAY contain many body-parts, each with its own MIME and HTTP headers (including Content-MD5, Content-Transfer-Encoding, and Content-Encoding headers).

RFC3230 (n.b., - standards track) agrees:

HTTP/1.1 defines a Content-MD5 header that allows a server to include a digest of the response body. However, this is specifically defined to cover the body of the actual message, not the contents of the full file (which might be quite different, if the response is a Content- Range, or uses a delta encoding).

The most immediate problem here is that a cache that combines multiple ranges can't change Content-MD5, can't strip it, and if it forwards it, it will be incorrect.

Attachments

i178.diff (7.6 KB) - added by julian.reschke@gmx.de 4 years ago.
Proposed changes for part 3 (remove C-MD5, mention in Changes from 2616, ref 6151)

Change History

comment:1 Changed 5 years ago by henrik@henriknordstrom.net

Another problem here is that a cache that produces 206 responses are very likely to include the Content-MD5 from the 200 response literally with Content-MD5 defined as an entity-header and 206 specified to (sometimes) include all entity headers from 200. Squid is known to do this, and expected that some other proxy caches do the same.

jigsaw is an example of an implementation known to do content-MD5 on the 206 message body.

comment:2 Changed 5 years ago by henrik@henriknordstrom.net

Additionally it's only origin servers and clients who are allowed to compose Content-MD5. "proxies and gateways MUST NOT generate it"

comment:3 Changed 5 years ago by mnot@pobox.com

  • Owner set to mnot@pobox.com
  • Priority set to normal

Proposal:

both:

1) caches MUST strip the Content-MD5 header when combining 206 responses 2) origin servers MUST NOT send a Content-MD5 header on 206 responses

comment:4 Changed 5 years ago by mnot@pobox.com

  • Milestone changed from unassigned to 08

add to proposal: 3) clients MUST ignore Content-MD5 on 206 responses.

comment:5 Changed 5 years ago by julian.reschke@gmx.de

  • Milestone changed from 08 to 09

comment:6 Changed 5 years ago by julian.reschke@gmx.de

  • Milestone changed from 09 to 10

comment:7 Changed 5 years ago by ylafon@w3.org

  • Owner changed from mnot@pobox.com to ylafon@w3.org

comment:8 Changed 5 years ago by ylafon@w3.org

  • Status changed from new to assigned

comment:9 Changed 4 years ago by mnot@pobox.com

  • Owner ylafon@w3.org deleted
  • Status changed from assigned to new
  • Milestone changed from 10 to unassigned

comment:10 Changed 4 years ago by julian.reschke@gmx.de

There are changes related to this ticket in [874].

comment:11 Changed 4 years ago by julian.reschke@gmx.de

From [880]:

Note change in [874] relating to issue 178 (see #178)

comment:12 Changed 4 years ago by henrik@henriknordstrom.net

According to my notes we had consensus on the mailinglist for the proposed changes to make Content-MD5 undefined/ignored for 206 responses.

As already noted there is implementations on both variants.

Content-MD5 on the full resource (including content-encoding, but not transfer encoding).

  • Apache, if enabled using the ContentDigest? directive (default off)
  • Squid, indirectly when copying Content-MD5 from a 200 response when constructing a 206 response in response to a range request.

Content-MD5 on the partial representation

  • Jigsaw

comment:13 Changed 4 years ago by mnot@pobox.com

  • Owner set to mnot@pobox.com

Prague 2011 editors meeting proposal: deprecate Content-MD5

comment:14 Changed 4 years ago by julian.reschke@gmx.de

Deprecate means: "leave it in, but warn about inconsistent behavior"?

comment:15 Changed 4 years ago by julian.reschke@gmx.de

  • Owner changed from mnot@pobox.com to julian.reschke@gmx.de
  • Milestone changed from unassigned to 14

Plan:

  • remove the definition
  • record it in "Changes from RFC 2616", saying "because of inconsistent implementations and MD5 being deprecated"

Changed 4 years ago by julian.reschke@gmx.de

Proposed changes for part 3 (remove C-MD5, mention in Changes from 2616, ref 6151)

comment:16 Changed 4 years ago by julian.reschke@gmx.de

From [1267]:

Remove Content-MD5 (see #178)

comment:17 Changed 4 years ago by julian.reschke@gmx.de

  • Status changed from new to closed
  • Resolution set to incorporated

comment:18 Changed 4 years ago by mnot@pobox.com

  • Status changed from closed to reopened
  • Resolution incorporated deleted

comment:19 Changed 4 years ago by mnot@pobox.com

  • Status changed from reopened to closed
  • Resolution set to fixed

comment:20 Changed 3 years ago by julian.reschke@gmx.de

From [1367]:

remove mention of C-MD5 in HEAD (see #178)

Note: See TracTickets for help on using tickets.