Open NghiaTranUIT opened 4 years ago
Any progress on this?
@Saklad5 Sorry, but at the moment, I'm working on Protobuf feature.
When it's done, I will kick start the HTTP/2 since it's crucial to support gRPC feature too ๐
Glad to see the Protobuf feature is done: itโs not something Iโm interested in, but it means you might work on this soon. Once Proxyman can run without downgrading traffic, I think Iโll purchase it. Until then, I canโt really justify running it for long periods of time, which limits its utility.
I would like to say that HTTP/2 and HTTP/3 are must-have features of Proxyman, and the progress is really good at the moment ๐
Protobuf, the first blocker is done, it's time for HTTP/2.
Stay tuned ๐ฎ
HTTP/3 isnโt formally accepted yet, so Iโd say thatโs less of a must-have than HTTP/2. I generally donโt expect things to support draft standards, and I understand it is pretty different.
By the way, does Proxyman do any other downgrading at the moment? Besides forcing HTTP/1, does simply running it affect outgoing traffic at all?
At the moment, only HTTPS domains, which enable SSL-Proxying (To see the HTTP Response Content) is downgraded to HTTP/1.
Otherwise, HTTPS (no SSL-Proxying) and HTTP are not, since it's just a proxy server, no interception. Thus, it would keep HTTP/2 or other protocols without changing it.
As there are no reliable HTTP/2 servers on macOS (Swift/Objc) yet, so it's the reason why Proxyman doesn't support HTTP/2 at the first implementation. However, Swift-NIO from Apple is fully supported recently, so I'm planning to rewrite the Proxyman Core with NIO stack.
When 1.23.0 build is out this weekend, I will push HTTP/2 forward ๐
Looking forward for gRPC support. Could be a killer feature, as nothing yet so handy support it.๐
@NghiaTranUIT sounds like you've made a ton of progress towards this. Could you share where you're at with HTTP/2, specifically to enable gRPC?
@bbuckland We're migrating the codebase to HTTP/2 but it hasn't finished yet. I will push it forward soon after the Multiple Filters and Diffing is done ๐
To workaround and see the gRPC data, you might consider using Atlantis from Proxyman (https://github.com/ProxymanApp/atlantis#2-my-app-use-grpc)
Would love to know if there are any updates on this, our backend microservices only talk over HTTP/2 and I would love to use Proxyman + the beta of the reverse proxy feature to improve my backend workflow.
That being said I switched over from Charles this week and it has been amazing for testing iOS things!
Thanks for the upvote @inickt. Since the Reverse Proxy is almost done (we're going to release it in this week), so our team is working on the HTTP/2 ๐
We will send a Beta build here when it's available ๐
Hey, I was curious if there have been any updates on this? Still been loving the app so thanks for all of your work and the great updates!
Hey @inickt Thanks for your shoutout.
To be honest, I tried to support HTTP/2 months ago but it doesn't work well since it's a breaking change in our codebase ๐ฟ . The problem is not just supporting HTTP/2, it also includes HTTP/1.1 as backward compatible.
At the moment, I'm focusing on the Folder Rule (where you can organize the tool rules in folders). When it's done, I will back to HTTP/2 ๐
Hello @NghiaTranUIT ๐๐ป; could we expect http2 support any time soon or not likely to happen soon? Thanks ๐ค
Sorry @NicolasCombe5555 This ticket is implemented but it's delayed due to technical problems. Basically, in order to support HTTP/2, we have to redesign the ProxymanCore, which is primarily designed for HTTP/1.1. Hence, we decide to postpone this ticket until we found a better solution.
Can you elaborate on why you need HTTP/2? Does it important for your work? @NicolasCombe5555 ๐ค
@NghiaTranUIT Well part of my project has http/2 in our tech stack; so I cannot see that traffic when using Proxyman. But I understand if we are having issues with supporting it here; thanks for the quick response tho ๐๐ป
I've read this thread and understand the complexity surrounding adding HTTP/2 support, but I just wanted to more or less throw a "+1" onto this issue as someone who would like to see it supported.
In the meantime, I am just using mitmproxy to view any traffic from my application where HTTP/2 is required.
@NghiaTranUIT to give you some context about the HTTP/2 need - we're starting to use more gRPC across my teams, and since that's HTTP/2, we're not able to proxy those requests using Proxyman. In fact, having Proxyman open when those requests go out essentially breaks them. (I work on the iOS apps, and am debugging that app's traffic to our servers)
Thanks for your work on such a great tool!
Even Proxyman can support HTTP/2 and Protobuf, but there is no guarantee that it can capture and properly decrypt the gRPC package because we don't support it yet.
To workaround, @eseay you can use Atlantis, which allows you to capture gPRC traffic ๐ https://github.com/ProxymanApp/atlantis#2-my-app-use-grpc
@NghiaTranUIT well I'll clarify that the back-end is serving a gRPC hybrid called Connect, so we're actually just using POST
requests with normal request bodies, but those requests require HTTP/2.
In either case - I am curious what I may be doing wrong in Proxyman to not be seeing HTTP/2 traffic.
When I run mitmproxy, I see that all of my requests (both gRPC-Connect and classic REST) are being conducted via HTTP/2; I see the same thing when proxying through Charles.
In Proxyman, when I go to inspect my requests, they are all showing as using HTTP/1.1. Is there something I may have configured incorrectly in Proxyman that's preventing the HTTP/2 transmissions?
I see the same thing. Is there any way to prevent Proxyman from breaking my gRPC services in app? I have tried Atlantis with no luck.
@j-j-m can you use this Atlantis code to capture gRPC traffic? https://github.com/ProxymanApp/atlantis#2-my-app-use-grpc
and open Tools Menu -> SSL Proxying List -> Remove the gRPC domain in the Include List. It will fix the SSL Error, and allows the Atlantis to work properly ๐
Good news ๐ ๐
I've supported the HTTP/2 protocol (Beta) for the Proxyman app ๐
Beta build: https://proxyman.s3.us-east-2.amazonaws.com/beta/Proxyman_4.1.0_Support_HTTP_2.dmg
HTTP/1.1
or HTTP/2
. It doesn't matter since Proxyman will use the correct HTTP Protocol behind the scenes => It means, you can use your current Map Local Rule/ Breakpoint Raw Message without any problems โ
If you guys find any bugs, please let me know, I'm all ears to fix it ๐
Just a friendly tag: @j-j-m @eseay @NicolasCombe5555 @inickt @bbuckland @Saklad5, @Ashraf-Ali-aa, @julasamer, @wasder, @ildar-gilfanov, @NSMyself, @heyzooi, @inickt, @nxtSwitch, @basecde, water-a
Tested it out, and it looks like it works to me! I was able to request an image where, previously, the connection was downgraded to 1.1. Thank you!
Thank you. There is a bug though. HTTP2 does not have the host header. So this RAW is incorrect. HTTP2 heading should look something like this.
:method: GET :authority: www.google.com :scheme: https :path: /
@NghiaTranUIT
Thanks @brr53. It's just a UI bug. Behind the sense, Proxyman automatically translates the HTTP/1.1 Raw format into the HTTP/2 format, so it's still working.
I'm going to fix the Raw Tab for HTTP/2 now ๐
A user reports that TLS/Extension for the HTTP/2 doesn't meet the standard from Google Chrome / Charles Proxy. We should fix it.
on https://tls.peet.ws/api/all
this is proxyman: 3:100,6:16384|00|0|p,m,s,a
this is chrome: 1:65536,2:0,3:1000,4:6291456,6:262144|15663105|0|m,a,s,p
this is charles: "1:65536,2:0,3:1000,4:6291456,6:262144|15663105|0|m,a,s,p",
@NghiaTranUIT What you have listed aren't related to TLS by the way. I think these are HTTP2 components.
WS doesn't work with HTTP/2
@brr53 just wondering: Why the akamai_fingerprint
is important to you ๐ค
I've investigated: The reason is the difference between the SentFrame Setting.
"HEADER_TABLE_SIZE = 65536",
"ENABLE_PUSH = 0",
"MAX_CONCURRENT_STREAMS = 1000",
"INITIAL_WINDOW_SIZE = 6291456",
"MAX_HEADER_LIST_SIZE = 262144"
"MAX_CONCURRENT_STREAMS = 100",
"MAX_HEADER_LIST_SIZE = 16384"
You can check out the Section 4: Passive HTTP/2 Fingerprint โ Suggested Format at https://www.blackhat.com/docs/eu-17/materials/eu-17-Shuster-Passive-Fingerprinting-Of-HTTP2-Clients-wp.pdf
Hi @NghiaTranUIT, thank you for your effort! It's working great, but I found a crash when connected to VPN before Proxyman is launched and trying to intercept some gRPC traffic. Not sure if there are any logs I can give you to track down the issue.
SentFrame
setting issueI went don't a fun rabbit hole related to why my service was falling back to http/1.1 (and breaking) and eventually landed here. Killing the proxy resolved my problem.
Any comment on if/when Proxyman will be able to allow http/3 as well?
@lundstrj Not soon yet since the HTTP/2 is still developing (it's 80% now).
Any updates or ETA on HTTP/2 support @NghiaTranUIT? ๐
Bump @NghiaTranUIT ๐
@djtarazona Sorry for the late reply. It's still in BETA.
Here is the v10 Update: https://download.proxyman.io/beta/Proxyman_4.9.0_HTTP_2_v10.dmg
@NghiaTranUIT Hello! Any update or ETA on HTTP/2 support? I'm still seeing connections be downgraded to HTTP/1.1 when proxied through Proxyman.
The update above indicates that it's still in beta as of Jul 30, but I can't find any mention of it after. Is there a way to access Beta builds of Proxyman?
Same here, please make HTTP/2 available to the public
Sorry guys @ekscrypto @lukeadamson , HTTP/2 is still in Beta because there is some technical issue that I haven't found a solution yet. It doesn't satisfy my standards so I decided not to bring it to the public until all issues are resolved.
For example:
Thanks for the context. Appreciate you holding a quality bar.
I don't need WS/WSS, so these betas have been working good for me. I'd appreciate an option to download a build with this HTTP/2 beta for each stable Proxyman release (not sure if the beta build generation is automatic). Or perhaps consider including this feature in stable releases but only opt-in from some preferences pane for beta features.
Thanks a bunch for your work @NghiaTranUIT!
@NghiaTranUIT any chance to get an up-to-date beta with HTTP2 support? thanks in advance!
@NghiaTranUIT I'm a bit confused by the HTTP2 support in the builds provided here - does HTTP2 support in Proxyman mean that Proxyman itself will use HTTP2 when connecting to the target server or does it mean that support HTTP2 when receiving incoming traffic, i.e. listening on localhost:9090? or both?
does HTTP2 support in Proxyman mean that Proxyman itself will use HTTP2 when connecting to the target server or does it mean that support HTTP2 when receiving incoming traffic
@Saklad5 yes, it's. Proxyman Beta prefers using HTTP/2, and fallback to HTTP/1 if your server doesn't talk HTTP2.
Okay, but from the client's perspective, the client connecting to proxyman that is, is the connection http2 or http1.1?
๐ถ Brief
At the moment, all intercepted HTTP/HTTPS through Proxyman will be downgraded to HTTP/1.1. We should support HTTP/2 too and implement the HTTP/2 parser correctly.
๐ Criteria
Update 1 (5 Feb 2023): Beta build for HTTP/2 is ready โ
Comment: https://github.com/ProxymanApp/Proxyman/issues/416#issuecomment-1417959460
Update 2 (14 Feb 2023): New Update
Comment: https://github.com/ProxymanApp/Proxyman/issues/416#issuecomment-1429048662
Update 3 (22 Feb 2023):
https://github.com/ProxymanApp/Proxyman/issues/416#issuecomment-1439878308