ClericPy / ichrome

Chrome controller for Humans, based on Chrome Devtools Protocol(CDP) and python3.7+.
https://pypi.org/project/ichrome/
MIT License
227 stars 29 forks source link

怎么获取 tab 的 headers.尤其是authorization #120

Closed ipdear closed 1 year ago

ipdear commented 1 year ago

怎么获取 tab 的 headers.尤其是authorization

ClericPy commented 1 year ago

https://clericpy.github.io/ichrome/reference/AsyncTab/#ichrome.async_utils.AsyncTab.iter_fetch 抓流量可以吧 或者用 JS 去拿?

ipdear commented 1 year ago

能提供一下dome么,我尝试适应iter_fetch无法获取authorization,只能获取COOKIE之类的一般headers

ClericPy commented 1 year ago

authorization 是 HTTP Basic auth 吗, 那个会转 base64 藏 headers 里吧?

ClericPy commented 1 year ago

@ipdear 首先, 不要暴露你这个 headers 出来啊, 直接暴露你密码了. 那个 Authorization 在 headers 里拿不到吗

iter_fetch 在上面文档里就有例子, 其实就是把请求发出去之前截获了, handleAuthRequests=True 设置了吗. 虽然设置了只是 handle 那个要帐号密码的请求, 然后在 header 里配置上这个 bear. 具体我也没试过, 都是跟着 chrome cdp 走的

ipdear commented 1 year ago

谢谢您的指导,我再尝试一下 我主要目的在于,通过Ichrome模拟登陆或获取 tab下的cookies和authorization ,cookies很容易活动,但authorization 没能找到正确的活动方式。

ClericPy commented 1 year ago

所以我想问的是: 你是想模拟登录, 还是手动登录拿到登录后的 Authorization...

纯粹为了填入 auth 帐号密码的话, 可以参考 https://clericpy.github.io/ichrome/reference/AsyncTab/#ichrome.async_utils.AsyncTab.pass_auth_proxy

        async with self.iter_fetch(handleAuthRequests=True) as f:
            try:
                task = asyncio.create_task(self.goto(test_url, timeout=1))
                for _ in range(iter_count):
                    if ok:
                        break
                    event: dict = await f
                    if event['method'] == 'Fetch.requestPaused':
                        await f.continueRequest(event)
                    elif event['method'] == 'Fetch.authRequired':
                        if callback:
                            ok = await ensure_awaitable(callback(event))
                        else:
                            await f.continueWithAuth(
                                event,
                                'ProvideCredentials',
                                user,
                                password,
                            )
                            ok = True
            finally:
                await task
                return ok

这部分是用来通过 proxy 里的 auth 的. 普通网页类似道理

ipdear commented 1 year ago

通过手动登录后获取 Authorization, 因为网站会随点击操作变化Authorization,所以手动登陆后需要监听Authorization

ClericPy commented 1 year ago

你 iter_fetch 代码是什么啊 或者有试过 https://username:password@www.example.com/ 么...

ClericPy commented 1 year ago

找到了, 信息藏在 Network.requestWillBeSentExtraInfo 事件里面

demo1: 自动填写 auth 密码

import asyncio

from ichrome import AsyncChromeDaemon

async def main():
    async with AsyncChromeDaemon(clear_after_shutdown=True,
                                 headless=False) as cd:
        async with cd.connect_tab(0, auto_close=True) as tab:
            async with tab.iter_fetch(handleAuthRequests=True) as f:
                await tab.goto('https://httpbin.org/basic-auth/user/password',
                               timeout=0.1)
                async for event in f:
                    print(event, flush=True)
                    if event['method'] == 'Fetch.requestPaused':
                        await f.continueRequest(event)
                    elif event['method'] == 'Fetch.authRequired':
                        await f.continueWithAuth(
                            event,
                            'ProvideCredentials',
                            'user',
                            'password',
                        )
                        await tab.wait_loading(3)
                        print('HTML:', await tab.html)

if __name__ == "__main__":
    asyncio.run(main())

demo2: 监控流量获取手动填写的 headers

import asyncio

from ichrome import AsyncChromeDaemon

async def main():
    async with AsyncChromeDaemon(clear_after_shutdown=True,
                                 headless=False) as cd:
        async with cd.connect_tab(0, auto_close=True) as tab:
            async with tab.iter_events(['Network.requestWillBeSentExtraInfo']) as f:
                await tab.goto('https://httpbin.org/basic-auth/user/password',
                               timeout=0.1)
                async for event in f:
                    if 'authorization' in str(event).lower():
                        print(event)
                        break

if __name__ == "__main__":
    asyncio.run(main())
ipdear commented 1 year ago

感谢🙏,已解决 authorization 的获取问题!

ClericPy commented 1 year ago

好的, 那我把问题关闭了