search

zhuzemin / pixiv_sort_by_popularity

pixiv_sort_by_popularity.user.js
12 stars 1 forks source link

脚本再次失效 #3

Open CCdsm opened 4 years ago

CCdsm commented 4 years ago

RT。依然和两个月前的情况一致。因为P站又更新了页面。希望作者能看到。

lisonge commented 4 years ago

@CCdsm cloudflare workers return error 1020

While visiting a site protected by Cloudflare, error 1020 “Access denied (Error: 1020)” indicates you've violated a Firewall Rule. ... If you're the site owner, you can find the requests that was blocked by the Firewall on the Firewall page under the Events menu.

maybe pixiv apply for protection to cloudflare

zhuzemin commented 4 years ago

@CCdsm cloudflare workers return error 1020

While visiting a site protected by Cloudflare, error 1020 “Access denied (Error: 1020)” indicates you've violated a Firewall Rule. ... If you're the site owner, you can find the requests that was blocked by the Firewall on the Firewall page under the Events menu.

maybe pixiv apply for protection to cloudflare

thanks for report! you are right, pixiv blocked access from cloudflare worker, so~~ I build another forward server behind cloudflare. XD

RayneSx commented 4 years ago

Soooo is there a way to fix it? let me know when new version of the script will be out! Thanks!!

CCdsm commented 4 years ago

情况应该是，P站更新了页面的样式，但是并非所有人都收到了更新，所以对于我这种收到了更新的用户来说，脚本失效了。等着全量更新时候再更新脚本吧。@lisonge

lisonge commented 4 years ago

情况应该是，P站更新了页面的样式，但是并非所有人都收到了更新，所以对于我这种收到了更新的用户来说，脚本失效了。等着全量更新时候再更新脚本吧。@lisonge

@CCdsm i do not use this plugin and The reason it can't be used is that pixiv blocked access from cloudflare worker. This has nothing to do with the style of the page. i just use the url api image

CCdsm commented 4 years ago

情况应该是，P站更新了页面的样式，但是并非所有人都收到了更新，所以对于我这种收到了更新的用户来说，脚本失效了。等着全量更新时候再更新脚本吧。@lisonge

@CCdsm i do not use this plugin and The reason it can't be used is that pixiv blocked access from cloudflare worker. This has nothing to do with the style of the page. image

好的，我明白了，谢谢你，我尝试修复一下

lisonge commented 4 years ago

@zhuzemin the plugin can use https://reqbin.com/ or other proxy api it can work well image

zhuzemin commented 4 years ago

@RayneSx @lisonge @CCdsm

so far with tested my self, script still working.

and I don't realize any style change between last two month.

CCdsm commented 4 years ago

@RayneSx @lisonge @CCdsm

so far with tested my self, script still working.

and I don't realize any style change between last two month.

Because this style update is not for everyone. But the script not working should be my own problem, not broken.

lisonge commented 4 years ago

@RayneSx @lisonge @CCdsm

so far with tested my self, script still working.

and I don't realize any style change between last two month.

@zhuzemin it does not work，did I miss something? image

RayneSx commented 4 years ago

Cattura

This is what I get, the page without any img at all... Herp!

Duscreen commented 4 years ago

It seems to have something to do with whether your research is written in japanese or roman alphabet.

RayneSx commented 4 years ago

It seems to have something to do with whether your research is written in japanese or roman alphabet.

Oh you are right... Thats the issue! And thats kinda bad, since using roman alphabet don't let you get most of the art! Can someone please fix it?

lisonge commented 4 years ago

It seems to have something to do with whether your research is written in japanese or roman alphabet.

Oh you are right... Thats the issue! And thats kinda bad, since using roman alphabet don't let you get most of the art! Can someone please fix it?

thank them! @zhuzemin Got the problem when search api url include non-ascii characters, it will work error ok url: https://proud-surf-e590.zhuzemin.workers.dev/ajax/https://www.pixiv.net/ajax/search/artworks/FGO?word=FGO&order=popular_d&mode=all&p=1&s_mode=s_tag_full&type=all&lang=zh error url: https://proud-surf-e590.zhuzemin.workers.dev/ajax/https://www.pixiv.net/ajax/search/artworks/FGO?word=FGO&order=popular_d&mode=all&p=1&s_mode=s_tag_full&type=all&lang=zh&z=草 error url: https://proud-surf-e590.zhuzemin.workers.dev/ajax/https://www.pixiv.net/ajax/search/artworks/キャル?word=キャル&order=popular_d&mode=all&p=1&s_mode=s_tag_full&type=all&lang=zh

the more detailed error is the following: error url: http://forwarder.eastus.cloudapp.azure.com/?k=xzSlE&t=30&order=popular_d&word=爱宕&mode=all&p=1&s_mode=s_tag_full&type=all&lang=zh&u=https://www.pixiv.net/ajax/search/artworks/爱宕 ok url: http://forwarder.eastus.cloudapp.azure.com/?k=xzSlE&t=30&order=popular_d&word=爱宕&mode=all&p=1&s_mode=s_tag_full&type=all&lang=zh&u=https://www.pixiv.net/ajax/search/artworks/FGO the reason of error is the parameter of key 'u' include non-ascii characters

RayneSx commented 4 years ago

Sooo, is there a way to update the script? Also why it was working before, and now stopped to work?

lisonge commented 4 years ago

Sooo, is there a way to update the script? Also why it was working before, and now stopped to work?

1.nothing to do if just update the script 2.it adds a layer of proxies with azure cloudapp 3.just wait for author

RayneSx commented 4 years ago

Let's wait then! Thanks

zhuzemin commented 4 years ago

It seems to have something to do with whether your research is written in japanese or roman alphabet.

Oh you are right... Thats the issue! And thats kinda bad, since using roman alphabet don't let you get most of the art! Can someone please fix it?

thank them! @zhuzemin Got the problem when search api url include non-ascii characters, it will work error ok url: https://proud-surf-e590.zhuzemin.workers.dev/ajax/https://www.pixiv.net/ajax/search/artworks/FGO?word=FGO&order=popular_d&mode=all&p=1&s_mode=s_tag_full&type=all&lang=zh error url: https://proud-surf-e590.zhuzemin.workers.dev/ajax/https://www.pixiv.net/ajax/search/artworks/FGO?word=FGO&order=popular_d&mode=all&p=1&s_mode=s_tag_full&type=all&lang=zh&z=草 error url: https://proud-surf-e590.zhuzemin.workers.dev/ajax/https://www.pixiv.net/ajax/search/artworks/キャル?word=キャル&order=popular_d&mode=all&p=1&s_mode=s_tag_full&type=all&lang=zh

the more detailed error is the following: error url: http://forwarder.eastus.cloudapp.azure.com/?k=xzSlE&t=30&order=popular_d&word=爱宕&mode=all&p=1&s_mode=s_tag_full&type=all&lang=zh&u=https://www.pixiv.net/ajax/search/artworks/爱宕 ok url: http://forwarder.eastus.cloudapp.azure.com/?k=xzSlE&t=30&order=popular_d&word=爱宕&mode=all&p=1&s_mode=s_tag_full&type=all&lang=zh&u=https://www.pixiv.net/ajax/search/artworks/FGO the reason of error is the parameter of key 'u' include non-ascii characters

tremendous help again!! thank you! ! , and thanks all of you for be patient of my slooowest response! and sorry, I forget handle non-ascii character. @RayneSx @lisonge @CCdsm @Duscreen

RayneSx commented 4 years ago

@zhuzemin don't worry! And thanks for your work, I love your script! Will wait for an updated version ♥

zhuzemin commented 4 years ago

It seems to have something to do with whether your research is written in japanese or roman alphabet.

Oh you are right... Thats the issue! And thats kinda bad, since using roman alphabet don't let you get most of the art! Can someone please fix it?

thank them! @zhuzemin Got the problem when search api url include non-ascii characters, it will work error ok url: https://proud-surf-e590.zhuzemin.workers.dev/ajax/https://www.pixiv.net/ajax/search/artworks/FGO?word=FGO&order=popular_d&mode=all&p=1&s_mode=s_tag_full&type=all&lang=zh error url: https://proud-surf-e590.zhuzemin.workers.dev/ajax/https://www.pixiv.net/ajax/search/artworks/FGO?word=FGO&order=popular_d&mode=all&p=1&s_mode=s_tag_full&type=all&lang=zh&z=草 error url: https://proud-surf-e590.zhuzemin.workers.dev/ajax/https://www.pixiv.net/ajax/search/artworks/キャル?word=キャル&order=popular_d&mode=all&p=1&s_mode=s_tag_full&type=all&lang=zh

the more detailed error is the following: error url: http://forwarder.eastus.cloudapp.azure.com/?k=xzSlE&t=30&order=popular_d&word=爱宕&mode=all&p=1&s_mode=s_tag_full&type=all&lang=zh&u=https://www.pixiv.net/ajax/search/artworks/爱宕 ok url: http://forwarder.eastus.cloudapp.azure.com/?k=xzSlE&t=30&order=popular_d&word=爱宕&mode=all&p=1&s_mode=s_tag_full&type=all&lang=zh&u=https://www.pixiv.net/ajax/search/artworks/FGO the reason of error is the parameter of key 'u' include non-ascii characters

I just realize you can see the second forward server(http://forwarder.eastus.cloudapp.azure.com) how do you do that?!

lisonge commented 4 years ago

@zhuzemin past picture image

lisonge commented 4 years ago

@zhuzemin it does not work now image

RayneSx commented 4 years ago

Is working again! Thanks ♥

proudtoberetarded commented 4 years ago

The second forward server is having connectivity issues.

proudtoberetarded commented 4 years ago

The second forward server is having connectivity issues. image

lisonge commented 4 years ago

hello everyone, the app-api.pixiv.net does not block cloudflare workers, so we can get popular artworks by app-api.pixiv.net

cloudflare-new workers

addEventListener('fetch', event => {
  event.respondWith(handleRequest(event.request))
})
const baseProxyUrl = 'https://proud-surf-e590.zhuzemin.workers.dev/ajax'
const pixivSearchByAppUrl = 'https://app-api.pixiv.net/v1/search/illust?search_target=partial_match_for_tags&sort=date_asc&word=fgo'
const backupUrl = 'https://www.pixiv.net/ajax/search/artworks/%E5%B7%A8%E4%B9%B3?word=%E5%B7%A8%E4%B9%B3&order=date_d&mode=all&p=1&s_mode=s_tag_full&type=all&lang=zh'
const typeObj = {
  illust: 0,
  manga: 1,
  ugoira: 2,
}
function getWebAppDataTemplate() {
  return ({
    "error": false,
    "body": {
      "novel": {
        "data": []
      },
      "relatedTags": [],
      "tagTranslation": {
        "\u5de8\u4e73": {
          "zh": "large breasts"
        },
        "\u7206\u4e73": {
          "zh": "huge breasts"
        },
        "\u30d1\u30a4\u30ba\u30ea": {
          "zh": "\u4e73\u4ea4"
        },
        "\u304a\u306d\u30b7\u30e7\u30bf": {
          "zh": "\u5fa1\u59ca\u6b63\u592a"
        },
        "\u6f2b\u753b": {
          "zh": "manga"
        },
        "\u3080\u3061\u3080\u3061": {
          "zh": "\u4e30\u6ee1"
        },
        "\u540c\u4eba": {
          "zh": "doujin"
        },
        "\u5c11\u5973": {
          "zh": "young girl"
        },
        "\u767e\u5408": {
          "zh": "yuri"
        },
        "\u53ef\u611b\u3044": {
          "zh": "\u53ef\u7231"
        },
        "\u30ad\u30e3\u30c3\u30c8\u30d5\u30a1\u30a4\u30c8": {
          "zh": "Cat Fight"
        },
        "\u30ec\u30ba\u30d0\u30c8\u30eb": {
          "zh": "\u767e\u5408\u683c\u9b25"
        },
        "NTW-20(\u30c9\u30fc\u30eb\u30ba\u30d5\u30ed\u30f3\u30c8\u30e9\u30a4\u30f3)": {
          "zh": "NTW-20\uff08\u5c11\u5973\u524d\u7ebf\uff09"
        },
        "\uc18c\ub140\uc804\uc120": {
          "zh": "\u5c11\u5973\u524d\u7ebf"
        },
        "\u304a\u3063\u3071\u3044": {
          "zh": "\u6b27\u6d3e"
        }
      },
      "zoneConfig": {
        "header": {
          "url": "https:\/\/pixon.ads-pixiv.net\/show?zone_id=header\u0026format=js\u0026s=2\u0026up=0\u0026a=29\u0026ng=g\u0026l=zh\u0026uri=%2Fajax%2Fsearch%2Ftop%2F_PARAM_\u0026is_spa=1\u0026K=4e70daf0399ba\u0026ab_test_digits_first=47\u0026yuid=JxYngmU\u0026suid=Pgfwek44gpid6xsyp\u0026num=5eeb02bc802"
        },
        "footer": {
          "url": "https:\/\/pixon.ads-pixiv.net\/show?zone_id=footer\u0026format=js\u0026s=2\u0026up=0\u0026a=29\u0026ng=g\u0026l=zh\u0026uri=%2Fajax%2Fsearch%2Ftop%2F_PARAM_\u0026is_spa=1\u0026K=4e70daf0399ba\u0026ab_test_digits_first=47\u0026yuid=JxYngmU\u0026suid=Pgfwek44gtqd71t6f\u0026num=5eeb02bc303"
        },
        "infeed": {
          "url": "https:\/\/pixon.ads-pixiv.net\/show?zone_id=illust_search_grid\u0026format=js\u0026s=2\u0026up=0\u0026a=29\u0026ng=g\u0026l=zh\u0026uri=%2Fajax%2Fsearch%2Ftop%2F_PARAM_\u0026is_spa=1\u0026K=4e70daf0399ba\u0026ab_test_digits_first=47\u0026yuid=JxYngmU\u0026suid=Pgfwek44gvlw7jeu1\u0026num=5eeb02bc123"
        }
      },
      "extraData": {
        "meta": {
          "title": "#\u6b27\u6d3e\u306e\u4eba\u6c17\u30a4\u30e9\u30b9\u30c8\u3084\u30de\u30f3\u30ac(\u6295\u7a3f\u8d85\u8fc750\u4e07\u4ef6\uff09 - pixiv",
          "description": "#\u6b27\u6d3e\u7684\u63d2\u753b\u6216\u6f2b\u753b\u4f5c\u54c1\u5171\u6709597749\u4ef6\uff0c#\u6b27\u6d3e\u7684\u5c0f\u8bf4\u6216\u5fae\u5c0f\u8bf4\u4f5c\u54c1\u5171\u67092346\u4ef6\u3002\u4e0e#\u6b27\u6d3e\u4e00\u8d77\u88ab\u6dfb\u52a0\u7684\u6807\u7b7e\u4e3b\u8981\u8fd8\u6709#large breasts\u3001#huge breasts\u3001#\u4e73\u4ea4\u3001#\u5fa1\u59ca\u6b63\u592a\u3001#manga\u3001#\u4e30\u6ee1\u3001#doujin\u3001#young girl\u3001#yuri\u3001#\u53ef\u7231\u3001#Cat Fight\u3001#\u767e\u5408\u683c\u9b25\u3001#NTW-20\uff08\u5c11\u5973\u524d\u7ebf\uff09\u3001#\u5c11\u5973\u524d\u7ebf\u7b49\u3002",
          "canonical": "https:\/\/www.pixiv.net\/tags\/%E3%81%8A%E3%81%A3%E3%81%B1%E3%81%84",
          "alternateLanguages": {
            "ja": "https:\/\/www.pixiv.net\/tags\/%E3%81%8A%E3%81%A3%E3%81%B1%E3%81%84",
            "en": "https:\/\/www.pixiv.net\/en\/tags\/%E3%81%8A%E3%81%A3%E3%81%B1%E3%81%84"
          },
          "descriptionHeader": "\u003Ca href=\u0022\/tags\/%E3%81%8A%E3%81%A3%E3%81%B1%E3%81%84\u0022\u003E#\u6b27\u6d3e\u003C\/a\u003E\u7684\u63d2\u753b\u6216\u6f2b\u753b\u4f5c\u54c1\u5171\u6709597749\u4ef6\uff0c\u003Ca href=\u0022\/tags\/%E3%81%8A%E3%81%A3%E3%81%B1%E3%81%84\u0022\u003E#\u6b27\u6d3e\u003C\/a\u003E\u7684\u5c0f\u8bf4\u6216\u5fae\u5c0f\u8bf4\u4f5c\u54c1\u5171\u67092346\u4ef6\u3002\u4e0e\u003Ca href=\u0022\/tags\/%E3%81%8A%E3%81%A3%E3%81%B1%E3%81%84\u0022\u003E#\u6b27\u6d3e\u003C\/a\u003E\u4e00\u8d77\u88ab\u6dfb\u52a0\u7684\u6807\u7b7e\u4e3b\u8981\u8fd8\u6709\u003Ca href=\u0022\/tags\/%E5%B7%A8%E4%B9%B3\u0022\u003E#large breasts\u003C\/a\u003E\u3001\u003Ca href=\u0022\/tags\/%E7%88%86%E4%B9%B3\u0022\u003E#huge breasts\u003C\/a\u003E\u3001\u003Ca href=\u0022\/tags\/%E3%83%91%E3%82%A4%E3%82%BA%E3%83%AA\u0022\u003E#\u4e73\u4ea4\u003C\/a\u003E\u3001\u003Ca href=\u0022\/tags\/%E3%81%8A%E3%81%AD%E3%82%B7%E3%83%A7%E3%82%BF\u0022\u003E#\u5fa1\u59ca\u6b63\u592a\u003C\/a\u003E\u3001\u003Ca href=\u0022\/tags\/%E6%BC%AB%E7%94%BB\u0022\u003E#manga\u003C\/a\u003E\u3001\u003Ca href=\u0022\/tags\/%E3%82%80%E3%81%A1%E3%82%80%E3%81%A1\u0022\u003E#\u4e30\u6ee1\u003C\/a\u003E\u3001\u003Ca href=\u0022\/tags\/%E5%90%8C%E4%BA%BA\u0022\u003E#doujin\u003C\/a\u003E\u3001\u003Ca href=\u0022\/tags\/%E5%B0%91%E5%A5%B3\u0022\u003E#young girl\u003C\/a\u003E\u3001\u003Ca href=\u0022\/tags\/%E7%99%BE%E5%90%88\u0022\u003E#yuri\u003C\/a\u003E\u3001\u003Ca href=\u0022\/tags\/%E5%8F%AF%E6%84%9B%E3%81%84\u0022\u003E#\u53ef\u7231\u003C\/a\u003E\u3001\u003Ca href=\u0022\/tags\/%E3%82%AD%E3%83%A3%E3%83%83%E3%83%88%E3%83%95%E3%82%A1%E3%82%A4%E3%83%88\u0022\u003E#Cat Fight\u003C\/a\u003E\u3001\u003Ca href=\u0022\/tags\/%E3%83%AC%E3%82%BA%E3%83%90%E3%83%88%E3%83%AB\u0022\u003E#\u767e\u5408\u683c\u9b25\u003C\/a\u003E\u3001\u003Ca href=\u0022\/tags\/NTW-20%28%E3%83%89%E3%83%BC%E3%83%AB%E3%82%BA%E3%83%95%E3%83%AD%E3%83%B3%E3%83%88%E3%83%A9%E3%82%A4%E3%83%B3%29\u0022\u003E#NTW-20\uff08\u5c11\u5973\u524d\u7ebf\uff09\u003C\/a\u003E\u3001\u003Ca href=\u0022\/tags\/%EC%86%8C%EB%85%80%EC%A0%84%EC%84%A0\u0022\u003E#\u5c11\u5973\u524d\u7ebf\u003C\/a\u003E\u7b49\u3002"
        }
      },
      "illustManga": {
        "data": [],
        "total": 999999999
      }
    }
  })
}
function switchData(illusts) {
  return illusts.map(illust => {
    const illustNew =
    {
      "illustId": illust.id + "",
      "illustTitle": illust.title,
      "id": illust.id + "",
      "title": illust.title,
      "illustType": illust.type,
      "xRestrict": illust.x_restrict,
      "restrict": 0,
      "sl": illust.sanity_level,
      "url": illust.image_urls.square_medium,
      "description": illust.caption,
      "tags": illust.tags.map(tag => tag.name),
      "userId": illust.user.id + "",
      "userName": illust.user.name,
      "width": illust.width,
      "height": illust.height,
      "pageCount": illust.page_count,
      "isBookmarkable": true,
      "bookmarkData": null,
      "alt": "",
      "isAdContainer": false,
      "titleCaptionTranslation": {
        "workTitle": null,
        "workCaption": null
      },
      "createDate": illust.create_date,
      "updateDate": illust.create_date,
      "profileImageUrl": illust.user.profile_image_urls.medium
    }
    return illustNew
  })
}

const modeObj = {
  s_tag: 'partial_match_for_tags',
  s_tag_full: 'exact_match_for_tags',
  s_tc: 'title_and_caption',
}

/**
 * Respond to the request
 * @param {Request} request
 */
async function handleRequest(request) {
  if (request.method == 'OPTIONS') {
    let h = 'x-user-id'
    if (request.headers.has('access-control-request-headers')) {
      h = request.headers.get('access-control-request-headers')
    }
    return new Response(
      '',
      {
        status: 200,
        headers: {
          'access-control-allow-origin': '*',
          'Access-Control-Allow-Headers': h,
        }
      }
    )
  }
  let webUrl = request.url.substring(new URL(request.url).origin.length + 1)
  if (webUrl.length < 10) {
    webUrl = backupUrl
  }
  const pixivSearchUrl = new URL(webUrl)
  const proxyUrl = new URL(pixivSearchByAppUrl)

  if (pixivSearchUrl.searchParams.has('s_mode')) {
    const sMode = pixivSearchUrl.searchParams.get('s_mode')
    if (sMode in modeObj) {
      proxyUrl.searchParams.set('search_target', modeObj[sMode])
    }
  }

  if (pixivSearchUrl.searchParams.has('scd')) {
    proxyUrl.searchParams.set('start_date', pixivSearchUrl.searchParams.get('scd'))
  }
  if (pixivSearchUrl.searchParams.has('ecd')) {
    proxyUrl.searchParams.set('end_date', pixivSearchUrl.searchParams.get('ecd'))
  }

  const word = pixivSearchUrl.searchParams.get('word')

  let p = 1
  if (pixivSearchUrl.searchParams.has('p')) {
    try {
      p = +pixivSearchUrl.searchParams.get('p')
    } catch{ }
  }

  let offset = (p - 1) * 60
  // error mseeage>>>{"offset":["Offset must be no more than 5000"]}
  if (offset > 4970) {
    offset = 4970
  }

  proxyUrl.searchParams.set('word', word)

  const urls = [0, 30].map(num => {
    proxyUrl.searchParams.set('offset', String(offset + num))
    return `${baseProxyUrl}/${proxyUrl.href}`
  })

  const webData = getWebAppDataTemplate()
  const appDatas = await Promise.all(urls.map(url => fetch(url).then(resp => resp.json())))

  appDatas.forEach(appData => {
    // console.log(appData)
    const illustsNew = switchData(appData.illusts)
    webData.body.illustManga.data.push(...illustsNew)
  })
  return new Response(
    JSON.stringify(webData),
    {
      status: 200,
      headers: {
        'access-control-allow-origin': '*',
        // 'Access-Control-Allow-Headers': 'x-user-id',
      }
    }
  )
}

and the test url is https://pixiv.lisonge.workers.dev/https://www.pixiv.net/ajax/search/artworks/アズールレーン?word=アズールレーン&order=date_d&mode=all&scd=2020-01-01&ecd=2020-06-26&p=1&s_mode=s_tag&type=all&lang=zh

this test url has been deprecated, you can also make a worker

otherwise i suggest you use chrome plugin which redirect url instead of tampermonkey.

preview image

image

zhuzemin commented 4 years ago

hello everyone, the app-api.pixiv.net does not block cloudflare workers, so we can get popular artworks by app-api.pixiv.net

cloudflare-new workers

addEventListener('fetch', event => {
  event.respondWith(handleRequest(event.request))
})
const baseProxyUrl = 'https://proud-surf-e590.zhuzemin.workers.dev/ajax'
const pixivSearchByAppUrl = 'https://app-api.pixiv.net/v1/search/illust?search_target=partial_match_for_tags&sort=date_asc&word=fgo'
const backupUrl = 'https://www.pixiv.net/ajax/search/artworks/%E5%B7%A8%E4%B9%B3?word=%E5%B7%A8%E4%B9%B3&order=date_d&mode=all&p=1&s_mode=s_tag_full&type=all&lang=zh'
const typeObj = {
  illust: 0,
  manga: 1,
  ugoira: 2,
}
function getWebAppDataTemplate() {
  return ({
    "error": false,
    "body": {
      "novel": {
        "data": []
      },
      "relatedTags": [],
      "tagTranslation": {
        "\u5de8\u4e73": {
          "zh": "large breasts"
        },
        "\u7206\u4e73": {
          "zh": "huge breasts"
        },
        "\u30d1\u30a4\u30ba\u30ea": {
          "zh": "\u4e73\u4ea4"
        },
        "\u304a\u306d\u30b7\u30e7\u30bf": {
          "zh": "\u5fa1\u59ca\u6b63\u592a"
        },
        "\u6f2b\u753b": {
          "zh": "manga"
        },
        "\u3080\u3061\u3080\u3061": {
          "zh": "\u4e30\u6ee1"
        },
        "\u540c\u4eba": {
          "zh": "doujin"
        },
        "\u5c11\u5973": {
          "zh": "young girl"
        },
        "\u767e\u5408": {
          "zh": "yuri"
        },
        "\u53ef\u611b\u3044": {
          "zh": "\u53ef\u7231"
        },
        "\u30ad\u30e3\u30c3\u30c8\u30d5\u30a1\u30a4\u30c8": {
          "zh": "Cat Fight"
        },
        "\u30ec\u30ba\u30d0\u30c8\u30eb": {
          "zh": "\u767e\u5408\u683c\u9b25"
        },
        "NTW-20(\u30c9\u30fc\u30eb\u30ba\u30d5\u30ed\u30f3\u30c8\u30e9\u30a4\u30f3)": {
          "zh": "NTW-20\uff08\u5c11\u5973\u524d\u7ebf\uff09"
        },
        "\uc18c\ub140\uc804\uc120": {
          "zh": "\u5c11\u5973\u524d\u7ebf"
        },
        "\u304a\u3063\u3071\u3044": {
          "zh": "\u6b27\u6d3e"
        }
      },
      "zoneConfig": {
        "header": {
          "url": "https:\/\/pixon.ads-pixiv.net\/show?zone_id=header\u0026format=js\u0026s=2\u0026up=0\u0026a=29\u0026ng=g\u0026l=zh\u0026uri=%2Fajax%2Fsearch%2Ftop%2F_PARAM_\u0026is_spa=1\u0026K=4e70daf0399ba\u0026ab_test_digits_first=47\u0026yuid=JxYngmU\u0026suid=Pgfwek44gpid6xsyp\u0026num=5eeb02bc802"
        },
        "footer": {
          "url": "https:\/\/pixon.ads-pixiv.net\/show?zone_id=footer\u0026format=js\u0026s=2\u0026up=0\u0026a=29\u0026ng=g\u0026l=zh\u0026uri=%2Fajax%2Fsearch%2Ftop%2F_PARAM_\u0026is_spa=1\u0026K=4e70daf0399ba\u0026ab_test_digits_first=47\u0026yuid=JxYngmU\u0026suid=Pgfwek44gtqd71t6f\u0026num=5eeb02bc303"
        },
        "infeed": {
          "url": "https:\/\/pixon.ads-pixiv.net\/show?zone_id=illust_search_grid\u0026format=js\u0026s=2\u0026up=0\u0026a=29\u0026ng=g\u0026l=zh\u0026uri=%2Fajax%2Fsearch%2Ftop%2F_PARAM_\u0026is_spa=1\u0026K=4e70daf0399ba\u0026ab_test_digits_first=47\u0026yuid=JxYngmU\u0026suid=Pgfwek44gvlw7jeu1\u0026num=5eeb02bc123"
        }
      },
      "extraData": {
        "meta": {
          "title": "#\u6b27\u6d3e\u306e\u4eba\u6c17\u30a4\u30e9\u30b9\u30c8\u3084\u30de\u30f3\u30ac(\u6295\u7a3f\u8d85\u8fc750\u4e07\u4ef6\uff09 - pixiv",
          "description": "#\u6b27\u6d3e\u7684\u63d2\u753b\u6216\u6f2b\u753b\u4f5c\u54c1\u5171\u6709597749\u4ef6\uff0c#\u6b27\u6d3e\u7684\u5c0f\u8bf4\u6216\u5fae\u5c0f\u8bf4\u4f5c\u54c1\u5171\u67092346\u4ef6\u3002\u4e0e#\u6b27\u6d3e\u4e00\u8d77\u88ab\u6dfb\u52a0\u7684\u6807\u7b7e\u4e3b\u8981\u8fd8\u6709#large breasts\u3001#huge breasts\u3001#\u4e73\u4ea4\u3001#\u5fa1\u59ca\u6b63\u592a\u3001#manga\u3001#\u4e30\u6ee1\u3001#doujin\u3001#young girl\u3001#yuri\u3001#\u53ef\u7231\u3001#Cat Fight\u3001#\u767e\u5408\u683c\u9b25\u3001#NTW-20\uff08\u5c11\u5973\u524d\u7ebf\uff09\u3001#\u5c11\u5973\u524d\u7ebf\u7b49\u3002",
          "canonical": "https:\/\/www.pixiv.net\/tags\/%E3%81%8A%E3%81%A3%E3%81%B1%E3%81%84",
          "alternateLanguages": {
            "ja": "https:\/\/www.pixiv.net\/tags\/%E3%81%8A%E3%81%A3%E3%81%B1%E3%81%84",
            "en": "https:\/\/www.pixiv.net\/en\/tags\/%E3%81%8A%E3%81%A3%E3%81%B1%E3%81%84"
          },
          "descriptionHeader": "\u003Ca href=\u0022\/tags\/%E3%81%8A%E3%81%A3%E3%81%B1%E3%81%84\u0022\u003E#\u6b27\u6d3e\u003C\/a\u003E\u7684\u63d2\u753b\u6216\u6f2b\u753b\u4f5c\u54c1\u5171\u6709597749\u4ef6\uff0c\u003Ca href=\u0022\/tags\/%E3%81%8A%E3%81%A3%E3%81%B1%E3%81%84\u0022\u003E#\u6b27\u6d3e\u003C\/a\u003E\u7684\u5c0f\u8bf4\u6216\u5fae\u5c0f\u8bf4\u4f5c\u54c1\u5171\u67092346\u4ef6\u3002\u4e0e\u003Ca href=\u0022\/tags\/%E3%81%8A%E3%81%A3%E3%81%B1%E3%81%84\u0022\u003E#\u6b27\u6d3e\u003C\/a\u003E\u4e00\u8d77\u88ab\u6dfb\u52a0\u7684\u6807\u7b7e\u4e3b\u8981\u8fd8\u6709\u003Ca href=\u0022\/tags\/%E5%B7%A8%E4%B9%B3\u0022\u003E#large breasts\u003C\/a\u003E\u3001\u003Ca href=\u0022\/tags\/%E7%88%86%E4%B9%B3\u0022\u003E#huge breasts\u003C\/a\u003E\u3001\u003Ca href=\u0022\/tags\/%E3%83%91%E3%82%A4%E3%82%BA%E3%83%AA\u0022\u003E#\u4e73\u4ea4\u003C\/a\u003E\u3001\u003Ca href=\u0022\/tags\/%E3%81%8A%E3%81%AD%E3%82%B7%E3%83%A7%E3%82%BF\u0022\u003E#\u5fa1\u59ca\u6b63\u592a\u003C\/a\u003E\u3001\u003Ca href=\u0022\/tags\/%E6%BC%AB%E7%94%BB\u0022\u003E#manga\u003C\/a\u003E\u3001\u003Ca href=\u0022\/tags\/%E3%82%80%E3%81%A1%E3%82%80%E3%81%A1\u0022\u003E#\u4e30\u6ee1\u003C\/a\u003E\u3001\u003Ca href=\u0022\/tags\/%E5%90%8C%E4%BA%BA\u0022\u003E#doujin\u003C\/a\u003E\u3001\u003Ca href=\u0022\/tags\/%E5%B0%91%E5%A5%B3\u0022\u003E#young girl\u003C\/a\u003E\u3001\u003Ca href=\u0022\/tags\/%E7%99%BE%E5%90%88\u0022\u003E#yuri\u003C\/a\u003E\u3001\u003Ca href=\u0022\/tags\/%E5%8F%AF%E6%84%9B%E3%81%84\u0022\u003E#\u53ef\u7231\u003C\/a\u003E\u3001\u003Ca href=\u0022\/tags\/%E3%82%AD%E3%83%A3%E3%83%83%E3%83%88%E3%83%95%E3%82%A1%E3%82%A4%E3%83%88\u0022\u003E#Cat Fight\u003C\/a\u003E\u3001\u003Ca href=\u0022\/tags\/%E3%83%AC%E3%82%BA%E3%83%90%E3%83%88%E3%83%AB\u0022\u003E#\u767e\u5408\u683c\u9b25\u003C\/a\u003E\u3001\u003Ca href=\u0022\/tags\/NTW-20%28%E3%83%89%E3%83%BC%E3%83%AB%E3%82%BA%E3%83%95%E3%83%AD%E3%83%B3%E3%83%88%E3%83%A9%E3%82%A4%E3%83%B3%29\u0022\u003E#NTW-20\uff08\u5c11\u5973\u524d\u7ebf\uff09\u003C\/a\u003E\u3001\u003Ca href=\u0022\/tags\/%EC%86%8C%EB%85%80%EC%A0%84%EC%84%A0\u0022\u003E#\u5c11\u5973\u524d\u7ebf\u003C\/a\u003E\u7b49\u3002"
        }
      },
      "illustManga": {
        "data": [],
        "total": 999999999
      }
    }
  })
}
function switchData(illusts) {
  return illusts.map(illust => {
    const illustNew =
    {
      "illustId": illust.id + "",
      "illustTitle": illust.title,
      "id": illust.id + "",
      "title": illust.title,
      "illustType": illust.type,
      "xRestrict": illust.x_restrict,
      "restrict": 0,
      "sl": illust.sanity_level,
      "url": illust.image_urls.square_medium,
      "description": illust.caption,
      "tags": illust.tags.map(tag => tag.name),
      "userId": illust.user.id + "",
      "userName": illust.user.name,
      "width": illust.width,
      "height": illust.height,
      "pageCount": illust.page_count,
      "isBookmarkable": true,
      "bookmarkData": null,
      "alt": "",
      "isAdContainer": false,
      "titleCaptionTranslation": {
        "workTitle": null,
        "workCaption": null
      },
      "createDate": illust.create_date,
      "updateDate": illust.create_date,
      "profileImageUrl": illust.user.profile_image_urls.medium
    }
    return illustNew
  })
}

const modeObj = {
  s_tag: 'partial_match_for_tags',
  s_tag_full: 'exact_match_for_tags',
  s_tc: 'title_and_caption',
}

/**
 * Respond to the request
 * @param {Request} request
 */
async function handleRequest(request) {
  if (request.method == 'OPTIONS') {
    let h = 'x-user-id'
    if (request.headers.has('access-control-request-headers')) {
      h = request.headers.get('access-control-request-headers')
    }
    return new Response(
      '',
      {
        status: 200,
        headers: {
          'access-control-allow-origin': '*',
          'Access-Control-Allow-Headers': h,
        }
      }
    )
  }
  let webUrl = request.url.substring(new URL(request.url).origin.length + 1)
  if (webUrl.length < 10) {
    webUrl = backupUrl
  }
  const pixivSearchUrl = new URL(webUrl)
  const proxyUrl = new URL(pixivSearchByAppUrl)

  if (pixivSearchUrl.searchParams.has('s_mode')) {
    const sMode = pixivSearchUrl.searchParams.get('s_mode')
    if (sMode in modeObj) {
      proxyUrl.searchParams.set('search_target', modeObj[sMode])
    }
  }

  if (pixivSearchUrl.searchParams.has('scd')) {
    proxyUrl.searchParams.set('start_date', pixivSearchUrl.searchParams.get('scd'))
  }
  if (pixivSearchUrl.searchParams.has('ecd')) {
    proxyUrl.searchParams.set('end_date', pixivSearchUrl.searchParams.get('ecd'))
  }

  const word = pixivSearchUrl.searchParams.get('word')

  let p = 1
  if (pixivSearchUrl.searchParams.has('p')) {
    try {
      p = +pixivSearchUrl.searchParams.get('p')
    } catch{ }
  }

  let offset = (p - 1) * 60
  // error mseeage>>>{"offset":["Offset must be no more than 5000"]}
  if (offset > 4970) {
    offset = 4970
  }

  proxyUrl.searchParams.set('word', word)

  const urls = [0, 30].map(num => {
    proxyUrl.searchParams.set('offset', String(offset + num))
    return `${baseProxyUrl}/${proxyUrl.href}`
  })

  const webData = getWebAppDataTemplate()
  const appDatas = await Promise.all(urls.map(url => fetch(url).then(resp => resp.json())))

  appDatas.forEach(appData => {
    // console.log(appData)
    const illustsNew = switchData(appData.illusts)
    webData.body.illustManga.data.push(...illustsNew)
  })
  return new Response(
    JSON.stringify(webData),
    {
      status: 200,
      headers: {
        'access-control-allow-origin': '*',
        // 'Access-Control-Allow-Headers': 'x-user-id',
      }
    }
  )
}

and the test url is https://pixiv.lisonge.workers.dev/https://www.pixiv.net/ajax/search/artworks/アズールレーン?word=アズールレーン&order=date_d&mode=all&scd=2020-01-01&ecd=2020-06-26&p=1&s_mode=s_tag&type=all&lang=zh

otherwise i suggest you use chrome plugin which redirect url instead of tampermonkey.

preview image

image

amazing works!!

I fixed second forward server, but your solution seems way more efficient!

lisonge commented 4 years ago

@zhuzemin it still has shortcomings, this solution just carry a few query parameters so the second forward server is better

proudtoberetarded commented 4 years ago

The script not working again?

proudtoberetarded commented 4 years ago

I downgraded to version 1.10 and the script works as intended, however the latest version of the script does not work for me.

zhuzemin commented 4 years ago

@proudtoberetarded

thank for your feedback,

I was added new feature in version 1.20 : support bookmark hint in search result.

but forget handle some case, cause script crashed,

now fixed.

roumi87 commented 4 years ago

1.21 and 1.20 are still not working.

1.10 works, but only for non japanese character search queries.

zhuzemin commented 4 years ago

@roumi87

thank for your feedback,

this time I tested script in several different situation: non japanese ui, non japanese keyword, search without other parameter,

script indeed not working in some case,

I fixed bugs of those situation;

if script not working happens again,

please post url of cause problem happen.

thanks

lisonge commented 4 years ago

@zhuzemin

I suggest your api url should be like this, it may deal unicode or url better original url: https://www.pixiv.net/ajax/search/illustrations/おっぱい proxy url: https://www-pixiv-net.zhuzemin.workers.dev/ajax/search/illustrations/おっぱい you can add second worker to test it you just only assert url.pathname.indexOf('/ajax/search/')==0 and change the request url's host and add your cookies

proudtoberetarded commented 4 years ago

the script not working again; tried it out on different search queries:

https://www.pixiv.net/en/tags/チェン(アークナイツ)/artworks?order=date&s_mode=s_tag https://www.pixiv.net/en/tags/Fate%2FGrandOrder/artworks?order=date&s_mode=s_tag

japanese search queries only sort by date; non-japanese search queries lead to a blank page.

proudtoberetarded commented 4 years ago

the script not working again; tried it out on different search queries:

https://www.pixiv.net/en/tags/チェン(アークナイツ)/artworks?order=date&s_mode=s_tag https://www.pixiv.net/en/tags/Fate%2FGrandOrder/artworks?order=date&s_mode=s_tag

japanese search queries only sort by date; non-japanese search queries lead to a blank page.

I forgot to mention that I was using the latest version when this happened.

WitherOfMc commented 4 years ago

Yes I have the same problem. The script doesn't work correctly.

The search queries only sort by date instead of popularity.