Is there any way to send raw binary data as a file?

[tarcisioe]

I might be missing something, but by reading the documentation and the code I only see the possibility of sending a multipart/form-data by specifying a dict in the form {name: path}. I'm now facing the issue where I want to send raw binary data (e.g. from an already-opened file) as a file, and I see no way to do it in asks's api.

Sending other types in the same dictionary depends on anyio.aopen raising TypeError or FileNotFoundError, which doesn't work if I send a bytes object that has a null byte on it, for example (pretty common in binary data). Also, it assumes the object is a string and tries to convert it into bytes specifying an encoding, which also doesn't work (causes TypeError: encoding without a string argument).

Also, asks will attempt to open every argument, which is both error-prone and inefficient, in case a file exists with that name and that argument isn't actually expected to be a file.

As an example, urllib3 uses a filetuple to disambiguate different situations: https://github.com/urllib3/urllib3/blob/master/src/urllib3/fields.py#L159

I'm not proposing to change the files= argument radically if that is not desired, but maybe adopt another API similar to requests, which allows specifying a file-like object to read from (easy enough to adapt bytes into), the content type and custom headers: https://github.com/psf/requests/blob/9ed5db8ed28e816b597dafd328b342ec95466afa/requests/models.py#L110

I also volunteer for contributing code after this is discussed :)

Thanks for the great work on this library, by the way!

[theelous3]

Yeah this has been on the todo list for a while. More than happy to accept a pr for this.

On Fri 22 May 2020, 4:13 a.m. Tarcisio, notifications@github.com wrote:

I might be missing something, but by reading the documentation and the code I only see the possibility of sending a multipart/form-data by specificating a dict in the form {name: path}. I'm now facing the issue where I want to send raw binary data (e.g. from an already-opened file) as a file, and I see no way to do it in asks's api.

Sending other types in the same dictionary depends on anyio.aopen raising TypeError or FileNotFoundError, which doesn't work if I send a bytes object that has a null byte on it, for example (pretty common in binary data). Also, it assumes the object is a string and tries to convert it into bytes specifying an encoding, which also doesn't work (causes TypeError: encoding without a string argument).

Also, asks will attempt to open every argument, which is both error-prone and inefficient, in case a file exists with that name and that argument isn't actually expected to be a file.

As an example, urllib3 uses a filetuple to disambiguate different situations: https://github.com/urllib3/urllib3/blob/master/src/urllib3/fields.py#L159

I'm not proposing to change the files= argument radically if that is not desired, but maybe adopt another API similar to requests, which allows specifying a file-like object to read from (easy enough to adapt bytes into), the content type and custom headers: https://github.com/psf/requests/blob/9ed5db8ed28e816b597dafd328b342ec95466afa/requests/models.py#L110

I also volunteer for contributing code after this is discussed :)

Thanks for the great work on this library, by the way!

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/theelous3/asks/issues/164, or unsubscribe https://github.com/notifications/unsubscribe-auth/AETL2SUK24WZ3RPPLM7WYCTRSXUVJANCNFSM4NHN4PBQ .

[theelous3]

Wasn't able to reply fully earlier. I'll post my thoughts on this asap.

Shouldn't be too awkward. Likely some kind of duck-type check and pass it off to a streaming coro, off the top of my head anyway. Would be nice to keep the api roughly the same, or see if we can improve on the alternatives.

[tarcisioe]

Well, here goes my proposal. I based my API design on requests/urllib3 "file-tuples", but with a real type to represent it. Should now only try to open files that the user intends to send, and support sending in-memory data as files as well, fairly easily. The tests I included should show usage fairly well.

Of course I'm open on changing any details if needed, and contributing to the documentation if my proposal is accepted.