When hitting a rule that points to a PAC file, foxy tries to access it directly

valleedelisle commented 7 months ago

Happening on Chrome Version 119.0.6045.199 (Official Build) (arm64)
Same issue with v8.2 tag (Checked it out a couple of minutes ago)
Same issue on the main branch
If I specify the PAC proxy directly, it works.
All other SOCK5 proxies are working.
Happens even if the PAC proxy is the first on the list and it has a catchall pattern. Not sure how to pull some logs under chrome

erosman commented 7 months ago

Are you setting the mode to "Proxy by Patterns"? Are you trying to pass some requests through normal proxies (http/socks) and some through PAC?

If that is the case, it wont work. The options are:

Use a PAC for all
Use normal proxy for all
User patterns for normal proxy (non-PAC)

valleedelisle commented 7 months ago

I am indeed setting "Proxy by patterns". I believe that's a regression because, in the previous version, I could use SOCKS for some pattern and PAC for the rest? Why this limitation?

erosman commented 7 months ago

PAC (Proxy Auto-Configuration file) is a file that sets the proxy configuration which would handle all proxying. There is no option to use a PAC for some requests and use a proxy server for others.

Please note that Patterns are not applicable in PAC type since all configurations will be handled by the Proxy Auto-Configuration file.

valleedelisle commented 7 months ago

Normally, PAC are company wide configuration files. It's not a one-size-fits-all kind of configuration unfortunately and a lot of people needs to override some of this configuration. This was working before and now it's not anymore, so I believe this is a regression.

erosman commented 7 months ago

The current Chrome proxy doesn't allow for a PAC URL and other proxies to run at the same time in the same scope. The options are to set a PAC URL, OR PAC script, OR Individual Proxy.

It is not possible to override PAC URL configuration. I will have to check the FoxyProxy 3 code to see what it was doing and get back to you..

erosman commented 7 months ago

Apparently, Chrome FP3, grabs the remote PAC, and compiles a new PAC with the patterns, and sets it as an inline PAC. I will see, what can be done about it.

valleedelisle commented 7 months ago

That would be awesome! Thanks!

erosman commented 7 months ago

Remote PAC Rules (draft)

Remote PAC

The standard operation for remote PAC (Proxy Auto-Configuration file) is:

PAC URL is set in browser network settings
Browser then fetches the PAC file and processes all of the connections settings according to the PAC rules

Proposal: Store Locally

Add a feature to fetch the remote PAC, and use its rules alongside other rules

Browser Coverage

While it is possible to add the feature to both Chrome & Firefox, it can conflict with the SOCKS5 authentication support in Firefox. Therefore, the feature is considered for Chrome implementation at the moment.

Implementation

Added UI (e.g. checkbox) store PAC locally
Locally stored PACs are updated every time options are saved
Added a PAC View button to fetch and display the remote PAC content
Individual Proxy
- All PACs will be set as remote PAC, regardless of the checkbox state
Proxy by Patterns
- Locally stored PACs will be processed in order, after processing other proxy patterns

Caveats

Please note that are multiple limits involved:

Limit on PAC length
Limit on Sync length
Limit on Storage length

Therefore ...

Option should only be used for altering remote PAC rules
Storing large or many PACs locally is not recommended and can prevent browser sync
There are PACs that are over 10mb
There are limits on the overall PAC length when set locally
- Chrome seems to have a limit of 1mb (Issue 678022)
- Firefox limit will posted

bocobeware commented 6 months ago

I have the exact same issue. I've sent an email with screenshots at:
support@getfoxyproxy.org but never got any answer...

erosman commented 6 months ago

@bocobeware The issue is being discussed here.

Use Case 1

Happens even if the PAC proxy is the first on the list and it has a catchall pattern.

Catchall means everything to go through PAC proxy. In this case, all other proxies and their patterns would be ignored. Selecting "Proxy by Patterns" would be meaningless in this situation. The logical choice would be to set the PAC proxy as the Individual Proxy and then everything would go through it.

Use Case 2

Normally, PAC are company wide configuration files. It's not a one-size-fits-all kind of configuration unfortunately and a lot of people needs to override some of this configuration.

For this purpose, new feature is added in v8.3 as per https://github.com/foxyproxy/browser-extension/issues/46#issuecomment-1845843040.

Scenario

There is PAC that proxies netflix.com, youtube,com, yahoo.com, example.com, etc User wants to use the PAC but prefers to send netflix.com through a different proxy (or no proxy).

Settings

Enter PAC in the FoxyProxy proxies (if not already added)
Select "Store Locally"
Add a new proxy server (if not already added)
Add a pattern for netflix.com
This proxy can be a proxy server (send it through this proxy) OR direct (send it directly)
SAVE
Select "Proxy by Patterns"

Now, FoxyProxy checks the patterns FIRST and if it doesn't find a matching matching pattern, processes the PAC rules.

This way, users are able to override the PAC rule for netflix.com.

Other Use Cases

Let me know if the scenario is not covered by the feature and I will update it accordingly.

ericjung commented 6 months ago

I think the local editing of PAC files has been conflated with the ability to use PAC files with "proxy by patterns and order" mode. Users who ask about this feature want this (forget about editing remote PAC files for the moment):

Suppose I have these settings in this order:

HTTP proxy and include pattern netflix.com/*
SOCKS5 proxy pattern and include patter hulu.com/*
PAC defined at https://foo.com/bar.pac and include pattern * (all URLs)

If mode is set to "proxy patterns and order" then:

If I navigate to netflix.com, HTTP proxy #1 above is used
If I navigate to hulu.com, SOCKS5 proxy #2 above is used
If I navigate to erosman.com, then the PAC file at https://foo.com/bar.pac is used to determine the proxy server (if any) that is used to load erosman.com

A separate feature request by just one user was that he is able to edit the #3 PAC file locally before it is processed. But many many more users have requested the first feature -- to add PAC-defined proxy settings to the "proxy by patterns and order" mode.

erosman commented 6 months ago

A separate feature request by just one user was that he is able to edit the 3 PAC file locally before it is processed. But many many more users have requested the first feature -- to add PAC-defined proxy settings to the "proxy by patterns and order" mode.

Editing PAC is something for the advanced users.

An advance user can simply ...

Download the PAC
Edit
Put the edited PAC somewhere accessible (e.g. Github)
Enter the edited PAC URL in the FoxyProxy settings as a remote PAC

valleedelisle commented 6 months ago

Editing PAC is something for the advanced users.

An advance user can simply ...
* Download the PAC

* Edit

* Put the edited PAC somewhere accessible (e.g. Github)

* Enter the edited PAC URL in the FoxyProxy settings as a remote PAC

I don't think it's just about editing the PAC, but more like maintaining it. If corporate decides to add rules in there, then I need to update my local copy as well when I notice it's been modified. I'll probably realize it's my PAC that is not updated when I'll try to reach a website and it's not working, and I'll probably troubleshoot for hours before I realize it's the PAC file.

erosman commented 6 months ago

@valleedelisle Didn't the feature outline in https://github.com/foxyproxy/browser-extension/issues/46#issuecomment-1845843040 cover your use cases?

I don't think it's just about editing the PAC, but more like maintaining it. If corporate decides to add rules in there, then I need to update my local copy as well when I notice it's been modified. I'll probably realize it's my PAC that is not updated when I'll try to reach a website and it's not working, and I'll probably troubleshoot for hours before I realize it's the PAC file.

That would be the same if FoxyProxy stores it. Maintenance is the same, regardless of where it is stored.

valleedelisle commented 6 months ago

That would be the same if FoxyProxy stores it. Maintenance is the same, regardless of where it is stored.

I believe the previous version was updating it automagically when there was a change? It probably stored the checksum and regenerates it when it changes.

erosman commented 6 months ago

I believe the previous version was updating it automagically when there was a change? It probably stored the checksum and regenerates it when it changes.

I am not sure if I understand your use case.

Are you going to use the PAC verbatim and update it as the original updates?
Are you going to edit it? Then, you have to re-edit every time the original updates.
What is the purpose of having a copy of the original PAC?

If your use case is not covered in the proposed feature, please explain the use case and the desired outcome.

valleedelisle commented 6 months ago

My expectations, based on the previous version of this extension is something like this:

Trigger: Browser starts, or PAC proxy added

Flow:

Download PAC and compare checksum with what we already have
If we had no original PAC file, or the checksum changed, proceed with parsing, otherwise keep everything in place.
If we have to parse, add pattern based rules from the PAC at the position where the PAC is located in the list of proxies.

erosman commented 6 months ago

Did FoxyProxy 3 allow you to compare checksums? Did FoxyProxy 3 allow you to edit the PAC?

valleedelisle commented 6 months ago

Did FoxyProxy 3 allow you to compare checksums? Did FoxyProxy 3 allow you to edit the PAC?

I don't know what it did in the background, I just know that it was working as expected :)

ericjung commented 6 months ago

Did FoxyProxy 3 allow you to compare checksums?

No, but it had an auto-refresh feature that retrieved the PAC periodically. It did not compare anything.

Did FoxyProxy 3 allow you to edit the PAC?

definitely not. It had a viewer and that it. the use case of editing a PAC is confusing, very very rarely needed, and I don’t think FP should support it.

foxyproxy / browser-extension