klange / toaruos

A completely-from-scratch hobby operating system: bootloader, kernel, drivers, C library, and userspace including a composited graphical UI, dynamic linker, syntax-highlighting text editor, network stack, etc.
https://toaruos.org/
University of Illinois/NCSA Open Source License
6.09k stars 477 forks source link

Improve `fetch` utility #209

Open klange opened 3 years ago

klange commented 3 years ago

The fetch utility has a long history in ToaruOS. It was originally written before the NIH policy was implemented and used a third-party HTTP parser. It was haphazardly adapted to do its own parsing, and more recently converted from using the devfs-based socket backend to the new BSD-style socket interface.

The fetch-tls version available from the repositories is still built on Joylent's HTTP parser, and was patched to include some features that were added to the upstream fetch, but it is still quite lacking.

Ideally, the core fetch utility to should be able to use mbedTLS as a backend transport seamlessly, without having to be built for it - possibly through a plugin architecture - to avoid having these two separate binaries and the potential differences in behavior that entails.

The utility also needs support for redirects, HTTP/1.1, better header generation, and general form submission. It would also be a welcome improvement to implement all of this as a reusable library, a la libcurl, so that fetching may be done directly from applications like the package manager, or in Kuroko scripts.