theelous3 / asks

Async requests-like httplib for python.
MIT License
508 stars 63 forks source link

why callback is bytearray,not response #104

Closed spritecn closed 5 years ago

spritecn commented 5 years ago

if response,any thing can bo done with callback

theelous3 commented 5 years ago

https://asks.readthedocs.io/en/latest/idioms.html#handling-response-body-content-downloads-etc

Callbacks are used to process byte chunks as they come in from the stream. Your callback must take the chunk as an arg, and can do whatever it wants with it.

I don't know your use case, but I would probably recommend setting stream=True and using the stream response instead :)

spritecn commented 5 years ago

like this await g.spawn(searchDataPost,s,proSearchUrl,searchRequestData,cookiesDict,i) we must make a func searchDataPost to save response,if can use callback to do it,we can use 'await g.spawn(s.post(url,data,cookies,callback=saveResponse))'

theelous3 commented 5 years ago

Just use stream responses:

https://asks.readthedocs.io/en/latest/overview-of-funcs-and-args.html#streaming-response-data

spritecn commented 5 years ago
async def searchDataPost(s,url,data,cookies,page):
    _r = await s.post(url,data=data,cookies=cookies)
    idList.append(_r.json()['pIdArr'])

just want to get json,no stream

theelous3 commented 5 years ago

Can you explain your problem more completely?

On Sun 10 Feb 2019, 4:05 a.m. spritecn, notifications@github.com wrote:

async def searchDataPost(s,url,data,cookies,page): _r = await s.post(url,data=data,cookies=cookies) propertyIdPageList.append(_r.json()['pIdArr'])

just want to get json,no stream

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/theelous3/asks/issues/104#issuecomment-462101893, or mute the thread https://github.com/notifications/unsubscribe-auth/ASa9SoKfmxvi4u0eSjTd0m7PjEcmhtm5ks5vL5qjgaJpZM4ay2uZ .

spritecn commented 5 years ago

thanks,sorry a bout my poor english, look,i

async def saveResponse(response):
             idList.append(_r.json()['pIdArr'])

i try to callback it await g.spawn(s.post(url,data,cookies,callback=saveResponse))

so,its not wokrs,tell me callback arg is bytearray not response so...

theelous3 commented 5 years ago

If that's all you want to do, you don't need a callback or a stream response. You just need to make normal calls.

https://asks.readthedocs.io/en/latest/idioms.html

You see that first example? Like that, but use a task group.

We have a worker function. Its job is only to act as a coroutine we can spawn with our task group so it can make a request for us.

We can use that same approach, except return r or r.json() and append that to a list. We don't need to handle the body in any kind of streaming or callback way. Just simple, like you would with requests.

I don't know if you're using curio or trio, but you can iterate their task groups and extract results from their tasks. This would be the best way of doing it.