Support providing non-plaintext documents as Assistant contexts

[noamross]

Check for existing issues

• [X] Completed

Describe the feature

It would be very helpful to be able to add common document files (e.g., PDFs, Word documents) to the assistant pane with /file contexts, as often there is documentation or other relevant information in them.

This could be partially supported with #17600, as one could run commands to extract plain text for documents (e.g., /terminal extract_text.sh myfile1.pdf myfile2.docx)

[gwtaylor]

I was going to open the same feature request, and I'm glad you beat me to it! For some context: I am a professor, and while I do write some code, the majority of development I am doing inside Zed is writing-related tasks, e.g. as Markdown, LaTex, or Plain Text. Often the context I want to include in the assistant pane with file is a PDF document (e.g. scientific papers). Let me share the workarounds I use now in case it is helpful to others:

• If I want to include an arXiv paper as context, I can do a /fetch with the HTML (experimental) URL of the paper. For new-ish papers, the HTML link is available as a link under "Access Paper" from the paper's abstract page. If the paper is older, you can obtain an experimental HTML version by changing the 'x' in the URL to '5', e.g. https://arxiv.org/abs/2406.01416 becomes https://ar5iv.org/abs/2406.01416
• If the PDF is not an arXiv paper, I typically convert to text on the CLI using pdftotext. I agree with your point that this could be partially supported by #17600.
• I don't think text extraction is ideal as modern LLMs are capable of processing more than text in a PDF. That said, Anthropic claims that Claude will only extract text from pdfs and docx files. Their cookbook demonstrates the use of pypdf when working with the API.

[notpeter]

I was thinking about exactly this the other day, but you could implement this as a Slash Command Extension if you had a local endpoint that would take a https/file url as a parameter and return plaintext. It would be relatively trivial to whip up a Flask service which took a url as a parameter, fetched the file, spawned pdftotext and generated a response.

If you could do the file processing entirely in pure rust (no external commands, no filesystem, no C libs) it could compile to wasm so you could even skip the local server altogether. I did a poor mans version of this for cleaning up the plaintext of RFC text files for my RFC extension: notpeter/rfc-zed. It's nothing fancy, but it would a decent starting point if you wanted to build this. See rfc.rs.

[gwtaylor]

Thank you! Slash command extension is a great way to implement it and the RFC extension you shared is an excellent starting point.