403 Forbidden on new sessions

[seth-rah]

Summary

So I tend to pull data every week from pixiv and this week it seems to have started giving me a 403 error which seems like an access issue

On my end I recreated my fanbox session and provided it with a new session ID, but the new sessions seems to be returning the same

Any chance fanbox put measures in place to stop pulling data, or might this be an isolated case?

Output

[fanbox-dl] 2024/06/27 17:54:52 [INFO] Launching Pixiv FANBOX Downloader!
2024/06/27 17:54:52 ===== ERROR LOG =====
2024/06/27 17:54:52 fanbox-dl error: resolve creator IDs: list all creator IDs: list supporintg plans: status is 403 Forbidden
2024/06/27 17:54:52 =====================
2024/06/27 17:54:52 The error log seems a bug, please open an issue on GitHub: https://github.com/hareku/fanbox-dl/issues.

[4piu]

Similar error here:

[fanbox-dl] 2024/06/28 02:58:31 [INFO] Launching Pixiv FANBOX Downloader!
[fanbox-dl] 2024/06/28 02:58:31 [INFO] Use input creator IDs: [artistfoo]
[fanbox-dl] 2024/06/28 02:58:31 [INFO] Started downloading of "artistfoo".
2024/06/28 02:58:31 ===== ERROR LOG =====
2024/06/28 02:58:31 fanbox-dl error: failed downloading of "artistfoo": list posts of "artistfoo": status is 403 Forbidden
2024/06/28 02:58:31 =====================
2024/06/28 02:58:31 The error log seems a bug, please open an issue on GitHub: https://github.com/hareku/fanbox-dl/issues.

[Ecollections]

I'm seeing the same error. I got a cloudflare validation when I refreshed my pixiv/fanbox login I had never seen before when mine started failing.

Updated my session ID, logged out and back in, and restarted my PC as sanity checks. Still getting the error.

[fanbox-dl] 2024/06/26 08:10:10 [INFO] Launching Pixiv FANBOX Downloader!
2024/06/26 08:10:10 ===== ERROR LOG =====
2024/06/26 08:10:10 fanbox-dl failed to run: failed to resolve creator IDs: failed to list supporintg plans: status is 403 Forbidden 2024/06/26 08:10:10 =====================
2024/06/26 08:10:10 The error log seems a bug, please open an issue on GitHub: https://github.com/hareku/fanbox-dl/issues.

[seth-rah]

Same situation on my end with signing into fanbox and seeing the cloudflare challenge

[hareku]

The error does not occur with my JP ip address. fanbox may prohibit suspicious access (HTTP requests from non-browsers) from some countries.

[Ecollections]

I was able to confirm with a VPN. When I set it to an endpoint in Japan I was able to query and download posts again. Turning the VPN off after it completed, it produced the same error as we saw before.

[jcoveney]

for what it's worth, I am physically located in japan (and not using a VPN etc) and still got the 403

[hareku]

Yes, fanbox seems to be restricting from JP IPs as of today. I am trying to find a solution.

[crskycode]

Yes, fanbox seems to be restricting from JP IPs as of today. I am trying to find a solution.

Is it possible to send cookies ?

[asdfghjqwerty]

It looks to me like Fanbox now expects the user agent as well as the full cookie you get from your validated browser. If anyone wants to run it themselves, I was able to fix this by hard coding the User-Agent header and the full Cookie header in ‎OfficialAPIClient.Request().

func (c *OfficialAPIClient) Request(ctx context.Context, method string, url string) (*http.Response, error) {
    req, err := retryablehttp.NewRequest(method, url, nil)
    if err != nil {
        return nil, fmt.Errorf("http request building error: %w", err)
    }

    req = req.WithContext(ctx)

    req.Header.Set("Cookie", fmt.Sprintf("FANBOXSESSID=%s", c.SessionID)) // Replace with your cookie here, or pass everything after FANBOXSESSID= using the sessid flag
    req.Header.Set("Origin", "https://www.fanbox.cc") // If Origin header is not set, FANBOX returns HTTP 400 error.
    req.Header.Set("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:127.0) Gecko/20100101 Firefox/127.0") // Use your real User-Agent
    return c.HTTPClient.Do(req)
}

[Cheeseball-Ninja]

It looks to me like Fanbox now expects the user agent as well as the full cookie you get from your validated browser. If anyone wants to run it themselves, I was able to fix this by hard coding the User-Agent header and the full Cookie header in ‎OfficialAPIClient.Request().

func (c *OfficialAPIClient) Request(ctx context.Context, method string, url string) (*http.Response, error) {
  req, err := retryablehttp.NewRequest(method, url, nil)
  if err != nil {
      return nil, fmt.Errorf("http request building error: %w", err)
  }

  req = req.WithContext(ctx)

  req.Header.Set("Cookie", fmt.Sprintf("FANBOXSESSID=%s", c.SessionID)) // Replace with your cookie here, or pass everything after FANBOXSESSID= using the sessid flag
  req.Header.Set("Origin", "https://www.fanbox.cc") // If Origin header is not set, FANBOX returns HTTP 400 error.
  req.Header.Set("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:127.0) Gecko/20100101 Firefox/127.0") // Use your real User-Agent
  return c.HTTPClient.Do(req)
}

I tried re-compiling (Windows/x64) with these changes in official_api_client.go, still 403 for me. This is with using a VPN and without as well.

EDIT: To add, same result with my browser's User-Agent or with your example above. I made sure the latest valid session ID is also referenced as well.

[asdfghjqwerty]

I tried re-compiling (Windows/x64) with these changes in official_api_client.go, still 403 for me. This is with using a VPN and without as well.

EDIT: To add, same result with my browser's User-Agent or with your example above. I made sure the latest valid session ID is also referenced as well.

Does your Cookie header looks something like this now? The new header looks like a representation of all the fanbox cookies in your browser, you can check by opening the Network tab looking at the request headers for any api.fanbox.cc request. FANBOXSESSID=0 ; p_ab_id=0; p_ab_id_2=0; p_ab_d_id=0; privacy_policy_agreement=0; privacy_policy_notification=0; cf_clearance=0 ; __cf_bm=0 ;

[Cheeseball-Ninja]

Does your Cookie header looks something like this now? The new header looks like a representation of all the fanbox cookies in your browser, you can check by opening the Network tab looking at the request headers for any api.fanbox.cc request. FANBOXSESSID=0 ; p_ab_id=0; p_ab_id_2=0; p_ab_d_id=0; privacy_policy_agreement=0; privacy_policy_notification=0; cf_clearance=0 ; __cf_bm=0 ;

Gotcha. It should look like:

req.Header.Set("Cookie", "p_ab_id=x; p_ab_id_2=x; p_ab_d_id=xxxxxxxx; privacy_policy_agreement=x; privacy_policy_notification=x; agreement_master_terms_of_use_202408=x; cf_clearance=xxxxxxxxxxxxxxxxxxxxx; FANBOXSESSID=xxxxxxxxxxxxxxxx; __cf_bm=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx")

EDIT: Literally the entire cookie header network request from your browser (not from the cookie itself!).

Thanks so much!

[remiliacn]

Add to this, cf_clearance is unique per IP address, I have to turn off VPN on my browser to get the cookie working

[hareku]

https://github.com/hareku/fanbox-dl/releases/tag/v0.21.1 The latest version v0.21.1 sets the user-agent and I no longer get 403s from my IP. Please note that some IPs still get the error.

[crskycode]

I think cf cookies are required to send the request.

[seth-rah]

I'm able to bypass with a JP VPN

But yeah, doesn't look like the user-agent solved the issue for me when not on the VPN

[hareku]

https://github.com/hareku/fanbox-dl/releases/tag/v0.22.1 v0.22.1 supports --cookie and --user-agent flags. Thank you all for the information.