Problems parsing simple W3C examples

[luchr]

There are some open problems/questions I have concerning this module:

1. Simple W3C-examples like https://www.w3.org/TR/html401/interact/forms.html

   Content-Type: multipart/form-data; boundary=AaB03x
   
   --AaB03x
   Content-Disposition: form-data; name="submit-name"
   
   Larry
   --AaB03x
   Content-Disposition: form-data; name="files"
   Content-Type: multipart/mixed; boundary=BbC04y
   
   --BbC04y
   Content-Disposition: file; filename="file1.txt"
   Content-Type: text/plain
   
   ... contents of file1.txt ...
   --BbC04y
   Content-Disposition: file; filename="file2.gif"
   Content-Type: image/gif
   Content-Transfer-Encoding: binary
   
   ...contents of file2.gif...
   --BbC04y--
   --AaB03x--

   are not parsed correctly. [Even FieldStorage can parse this.]

2. Support for encoded field names (RFC 2388, 3. and 5.4) by "the standard method described in RFC 2047"?
3. Support for embedded "multipart/mixed" (RFC 2388, 4.2)?
   • 100% test coverage? Of what? The W3C-examples? RFC 2388?
   • prevent DOS attacks? Do you count the bytes in the headers of the (sub-)parts?
   • Did you read RFC 2388? Ever thought why FieldStorage-objects can be nested? Ever thought about why the docu of FieldStorage says the name can be None (and this is intentional)? Having no "name": What does this tell you about the idea to save (sub-)parts in a dict?
   • Do you plan to note this "limitations" somewhere?

[defnull]

Correct me if I'm wrong, but as far as I know, 1) multipart/mixed is not used by modern browsers or HTTP client libraries. 2) browsers or HTTP client libraries send filenames as UTF-8 encoded unicode strings nowadays. 3) see 1.

Implementing support for unused or outdated parts of am RFC is not always the right thing to do. If you find a valid use-case, however, I'd be happy to accept your pull request.

As for your other questions:

• Test coverage usually refers to code coverage of the test suite. Features that are not implemented are not tested and therefore do not counted for or against coverage. Claiming 100% standard conformance, on the other hand, would be hard to measure or prove anyway. I don't thing that such a claim could be taken seriously. Then again, even 100% test coverage does not say the implemented parts are bug-free. It just indicates that ever single line of the implementation is touched by a test case at least once. Tests can be wrong, too.
• No, headers are not counted against the in-memory limit, but that would be easy to add. Actually, that's a very good idea.
• No I did not read the entire RFC word by word, nor do I think someone is required to do that in order to write a useful library. This library is meant to parse form uploads as they appear in real world and is not, and does not claim to be, a reference implementation of any sort.
  • Nested multipart messages is an unused feature of a 14 years old recommendation. If there is a real world use-case to support it, I will accept pull requests.
  • Yes, the name can be None and None is a perfectly fine value to use as a dictionary key. Fields without a name are parsed, and accessible through this library, as far as I remember.
  • The dictionary is returned by a convenience function, the actual parser returns an iterator. And even the dict stores lists of values. The only information lost is the exact order of the multipart elements, because elements with the same name are grouped together. If you need the exact order, use the parser directly. By the way, the parser happily returns parts with a content-type of multipart/mixed if it finds one. You can parse that recursively, if you want. The parse_form_data() function is not the only entry point into this library.
• Feel free to send a pull request. You can edit the readme directly via github in your browser. A list of RFC-Features that are not implemented by this library would actually be a constructive and useful addition to the documentation. Hooray for open source software!

[luchr]

I added a pull request, where I tried to describe the limitations mentioned above.

No I did not read the entire RFC word by word, nor do I think someone is required to do that in order to write a useful library

Thank you for this statement/clarification. Then, I have no further questions.