评分对比助手 is a great script, but some bugs need fixing

Kamikadashi commented 11 months ago

First of all, thank you for your 评分对比助手 script, it's great. Unfortunately, it has some problems.

The extraction of the VNDB score appears to be broken. But it can be fixed by replacing const s = vote.match(/average (\d+(\.\d+))/); with const s = vote.match(/(\d+(\.\d+)?)(?= average)/);

Additionally, the search logic requires enhancement as it is currently unable to retrieve game pages on EGS and VNDB when there are discrepancies such as between ＝ and =, ２ and 2, Ⅲ and III, - and －, etc. I managed to slightly improve results by modifying normalizeQuery function:

      let newQuery = query
          .replace(/([^～]*～[^～]*～[^～]*)/g, function(match) { return match.replace(/~|～/g, ' '); })
          .replace(/＝|=/g, ' ')
          .replace(/０/g, '0')
          .replace(/１/g, '1')
          .replace(/２/g, '2')
          .replace(/３/g, '3')
          .replace(/４/g, '4')
          .replace(/５/g, '5')
          .replace(/６/g, '6')
          .replace(/７/g, '7')
          .replace(/８/g, '8')
          .replace(/９/g, '9')
          .replace(/Ⅰ/g, 'I')
          .replace(/Ⅱ/g, 'II')
          .replace(/Ⅲ/g, 'III')
          .replace(/Ⅳ/g, 'IV')
          .replace(/Ⅴ/g, 'V')
          .replace(/Ⅵ/g, 'VI')
          .replace(/Ⅶ/g, 'VII')
          .replace(/Ⅷ/g, 'VIII')
          .replace(/Ⅸ/g, 'IX')
          .replace(/Ⅹ/g, 'X')
          .replace(/－|-/g, ' ')
          .replace(/\s{2,}/g, ' ')
          .replace(/～/g, '～')

          .trim();
      return newQuery;
  }

In addition to this, sometimes when fetching scores for some visual novels from EGS, the script fetches them from a random port of said visual novel that has a significantly smaller number of votes. The expected behavior should be either to fetch the score from the entry with the most number of votes or not to fetch scores automatically at all and prompt a user to press ‘Click to search’ instead.

See: https://vndb.org/v7302 https://vndb.org/v18498 https://vndb.org/v22783 https://vndb.org/v15473 https://vndb.org/v751 https://vndb.org/v57 https://vndb.org/v12849 https://vndb.org/v5154 https://vndb.org/v19684 https://vndb.org/v20424 https://vndb.org/v13666 https://vndb.org/v17 https://vndb.org/v7723 https://vndb.org/v29444

Another problem I encountered is also with fetching scores from EGS. It keeps fetching the 中央値 even when score: (_b = (_a = $q('#average > td')) === null || _a === void 0 ? void 0 : _a.textContent.trim()) !== null && _b !== void 0 ? _b : 0, is set.

Upon further investigation, it seems the script fetches a score number from the search page instead of the game page, and the search page only has the 中央値 value. When a user opens the game page on their own, the score number on other sites successfully updates to the average value. So, as I see it, the problem is that the score fetching is happening in the getSearchItem$1 function instead of the getSearchResult$1 function.

zhifengle commented 11 months ago

function normalizeQuery has updated. The second problem is caused by the lack of a release date. When multiple search results occurs, it would use title and release date to find out subject. https://github.com/zhifengle/gm_scripts/blob/37253d7a71c015900d5b2a351a697e8c61d1adfd/src/sites/common.ts#L30 VNDB has multiple release date, previous version does not use it. Now I try to use the date which has icon-rtcomplete mark.

The last one is not solved. In order to get 平均値, it would perform additional step which is to request game page, it increases the burden on the server. If this feature is necessary, I can add it later.

Kamikadashi commented 11 months ago

Thank you for the quick response!

At first glance, it appears that the second issue has been largely resolved. However, for some reason, the scores for 凍京NECRO＜トウキョウ・ネクロ＞ SUICIDE MISSION, which only has 4 votes, are still being loaded on [this page]. I suppose it's not the only one.

Regarding the final issue, I believe it would be beneficial if one could select which score to use. The score randomly changing after you open the page isn’t ideal either. Therefore, I think it’s worth making an additional request.

P.S. I’m not sure if it’s just my imagination or perhaps just a coincidence, but I feel that the script is loading results more slowly than before.

zhifengle commented 11 months ago

The game's title is 凍京NECRO＜トウキョウ・ネクロ＞ in VNDB. But search result in EGS only has 凍京NECRO＜トウキョウ・ネクロ＞ SUICIDE MISSION. To fix it, remove ＜トウキョウ・ネクロ＞ from search string.

The problem of 平均値 has been solved.

凍京NECRO This page is slow when loading results. It also has been solved. (It would not find subject by release month in bgm.tv. This step would take long time)

Kamikadashi commented 11 months ago

Thanks! I’ll test it and try to further improve normalizeQuery. I think it would probably be better to have two of them, one for VNDB and one for EGS, as the replacements that need to be done differ significantly.

By the way, perhaps it would be worthwhile to add a check for the release date when fetching VNDB scores as well? Currently, it fails to fetch anything in cases such as these, for example: link.

It seems the problem is similar to the one with EGS. If VNDB automatically returns a game page, then the score gets fetched. However, if it returns a list of games instead, then nothing gets fetched at all. The score appears, though, if the user manually loads the game page.

Anyway, thanks again for your hard work!

zhifengle commented 11 months ago

VNDB has been redesigned, parse function is broken. It is fixed.

No search result still occurs In VNDB. for example: ランス10 The game title is Rance Ⅹ -決戦- in VNDB. but it's ランス10 in EGS. Search feature is based on title and the search result of target website. I have no idea how to fix it.

Thanks for the feedback.

Kamikadashi commented 11 months ago

Great, thanks! I’ve tried it and most of the problems I had seem to be fixed indeed. However, from time to time, I get a NaN error for VNDB scores. chrome_UfsntCXLOH

For cases like the one with ランス10 (there are a number of them), one could probably try to retrieve the name written in Japanese from the ‘Aliases’ section, if one is available. Additionally, one might consider retrying a couple of times if the search result page is empty, etc.

I’ve added an additional replacement function specifically for EGS and will try to find more ways to improve the results. However, this needs more testing for now. Currently, my filters look something like this:


  function normalizeEGS(query) {
      let newQuery = query.replace(/([Ａ-Ｚａ-ｚ０-９])([Ａ-Ｚ])/g, '$1 $2');
      newQuery = newQuery.replace(/[Ａ-Ｚａ-ｚ０-９]/g, function(s) {
          return String.fromCharCode(s.charCodeAt(0) - 65248);
      });

      newQuery = newQuery
          .replace(/^(.*?～)(.*)(～[^～]*)$/, function (_, p1, p2, p3) {
              return p1.replace(/～/g, " ") + p2 + p3.replace(/～/g, " ");
          })
          .replace(/＝|=/g, " ")
          .replace(/　/g, " ")
          .replace(/０/g, "0")
          .replace(/１/g, "1")
          .replace(/２/g, "2")
          .replace(/３/g, "3")
          .replace(/４/g, "4")
          .replace(/５/g, "5")
          .replace(/６/g, "6")
          .replace(/７/g, "7")
          .replace(/８/g, "8")
          .replace(/９/g, "9")
          .replace(/Ⅰ/g, "I")
          .replace(/Ⅱ/g, "II")
          .replace(/Ⅲ/g, "III")
          .replace(/Ⅳ/g, "IV")
          .replace(/Ⅴ/g, "V")
          .replace(/Ⅵ/g, "VI")
          .replace(/Ⅶ/g, "VII")
          .replace(/Ⅷ/g, "VIII")
          .replace(/Ⅸ/g, "IX")
          .replace(/Ⅹ/g, "X")
          .replace(/－|-/g, " ")
          .replace(/―/g, " ")
          .replace(/\s{2,}/g, " ")
          .replace(/～/g, " ")
          .replace(/〜/g, " ")
          .replace(/━/g, " ")
          .replace(/\[/g, " ")
          .replace(/]/g, " ")
          .replace(/『/g, " ")
          .replace(/』/g, " ")
          .replace(/~/g, " ")
          .replace(/'/g, " ")
          .replace(/…/g, " ")
          .replace(/！/g, " ")
          .replace(/？/g, " ")
          .replace(/。/g, " ")
          .replace(/♥/g, " ")
          .replace(/❤/g, " ")
          .replace(/☆/g, " ")
          .replace(/\//g, " ")
          .replace(/♡/g, " ")
          .replace(/★/g, " ")
          .replace(/‥/g, " ")
          .replace(/○/g, " ")
          .replace(/⁉/g, " ")
          .replace(/,/g, " ")
          .replace(/\./g, " ")
          .replace(/-/g, " ")
          .replace(/【/g, " ")
          .replace(/】/g, " ")
          .replace(/◆/g, " ")
          .replace(/●/g, " ")
          .replace(/∽/g, " ")
          .replace(/＋/g, " ")
          .replace(/‼/g, " ")
          .replace(/＿/g, " ")
          .replace(/◯/g, " ")
          .replace(/※/g, " ")
          .replace(/♠/g, " ")
          .replace(/×/g, " ")
          .replace(/▼/g, " ")
          .replace(/％/g, " ")
          .replace(/#/g, " ")
          .replace(/∞/g, " ")
          .replace(/’/g, " ")
          .replace(/&/g, " ")
          .replace(/!/g, " ")
          .replace(/:/g, " ")
          .replace(/＇/g, " ")
          .replace(/"/g, " ")
          .replace(/？/g, " ")
          .replace(/＊/g, " ")
          .replace(/\*/g, " ")
          .replace(/＆/g, " ")
          .replace(/［/g, " ")
          .replace(/］/g, " ")
          .replace(/</g, " ")
          .replace(/>/g, " ")
          .replace(/＜/g, " ")
          .replace(/＞/g, " ")
          .replace(/`/g, " ")
          .replace(/_/g, " ")
          .replace(/「/g, " ")
          .replace(/」/g, " ")
          .replace(/¨/g, " ")
          .replace(/／/g, " ")
          .replace(/◇/g, " ")
          .replace(/：/g, " ")
          .replace(/♪/g, " ")
          .replace(/･/g, " ")
          .replace(/@/g, " ")
          .replace(/＠/g, " ")
          .replace(/°C/g, "℃")
          .replace(/、/g, " ")
          .replace(/，/g, " ")
          .replace(/△/g, " ")
          .replace(/《/g, " ")
          .replace(/》/g, " ")
          .replace(/†/g, " ")
          .replace(/〇/g, " ")
          .replace(/\//g, " ")
          .replace(/·/g, " ")
          .replace(/;/g, " ")
          .replace(/^/g, " ")
          .replace(/’/g, " ")
          .replace(/“/g, " ")
          .replace(/”/g, " ")
          .replace(/√/g, " ")
          .replace(/≪/g, " ")
          .replace(/≫/g, " ")
          .replace(/＃/g, " ")
          .replace(/→/g, " ")
          .replace(/♂/g, " ")
          .replace(/\?/g, " ")
          .replace(/%/g, " ")
          .replace(/~/g, " ")
          .replace(/■■/g, " ")
          .replace(/■/g, " ")
          .replace(/‘/g, " ")
          .replace(/〈/g, " ")
          .replace(/〉/g, " ")
          .replace(/Ω/g, " ")
          .replace(/♀/g, " ")
          .replace(/⇒/g, " ")
          .replace(/≒/g, " ")
          .replace(/§/g, " ")
          .replace(/■/g, " ")
          .replace(/♀/g, " ")
          .replace(/⇒/g, " ")
          .replace(/←/g, " ")
          .replace(/∬/g, " ")
          .replace(/🕊/g, " ")
          .replace(/¡/g, " ")
          .replace(/Ι/g, " ")
          .replace(/≠/g, " ")
          .replace(/±/g, " ")
          .replace(/『/g, " ")
          .replace(/』/g, " ")
          .replace(/♨/g, " ")
          .replace(/❄/g, " ")
          .replace(/—/g, " ")
          .replace(/~/g, " ")
          .replace(/Σ/g, " ")
          .replace(/⇔/g, " ")
          .replace(/↑/g, " ")
          .replace(/↓/g, " ")
          .replace(/‡/g, " ")
          .replace(/▽/g, " ")
          .replace(/□/g, " ")
          .replace(/』/g, " ")
          .replace(/〈/g, " ")
          .replace(/〉/g, " ")
          .replace(/＾/g, " ")
          .replace(/[①②③④⑤⑥⑦⑧⑨]/g, " ")
          .replace(/[¹²³⁴⁵⁶⁷⁸⁹⁰]/g, " ")
          .replace(/‐.*?‐/g, " ")
          .replace(/─/g, " ")
          .replace(/\|/g, " ")
          .replace(/\+/g, " ")
          .replace(/\(.*?\)/g, " ")
          .replace(/\（.*?\）/g, " ")
          .replace(/\.\.\./g, " ")
          .replace(/．/g, ' ')
          .replace(/([Ａ-Ｚａ-ｚ０-９])([Ａ-Ｚ])/g, '$1 $2')
          .replace(/～っ.*/, '')
          .replace(/・/g, " ")
          .replace(/\(.*?\)/g, '')
          .replace(/\（.*?\）/g, ' ')
          .replace(/\s+/g, " ")
          .replace(/淫獣学園EX La Blue Girl/g, "淫獣学園")
          .replace(/Rance X/g, "ランス10")
          .replace(/ランス5 D/g, "Rance5D")
          .trim();

          let parts = newQuery.split(' ');
          let englishWordCount = 0;
          let nonEnglishDetected = false;
          let japaneseWordCount = 0;
          let isJapaneseWord = false;

          for(let i = 0; i < parts.length; i++) {

                              let isEnglishWord = /^[a-zA-Z]+$/.test(parts[i]);

                              if(isEnglishWord) {
                                                  englishWordCount++;
                              } else {

                                                  isJapaneseWord = /[\p{Script=Hiragana}\p{Script=Katakana}\p{Script=Han}ーa-zA-Z0-9ａ-ｚＡ-Ｚ０-９々〆〤]/u.test(parts[i]);
                                                  if(isJapaneseWord) {
                                                                      nonEnglishDetected = true;
                                                                      japaneseWordCount++;
                                                  }
                              }

                              if(nonEnglishDetected && englishWordCount > 0) {
                                                  parts = parts.slice(0, i);
                                                  break;
                              }

                              if(isEnglishWord && englishWordCount == 2) {
                                                  parts = parts.slice(0, i + 1);
                                                  break;
                              }

                              if(isJapaneseWord && japaneseWordCount == 2) {
                                                  for(let j = 0; j <= i; j++) {
                                                                      if(parts[j].length <= 1 && j < i) {
                                                                                          continue;
                                                                      } else {
                                                                                          parts = parts.slice(0, j + 1);
                                                                                          break;
                                                                      }
                                                  }
                                                  break;
                              }
          }

          newQuery = parts.join(' ');

          return newQuery;

      }

  function normalizeQuery(query) {
      let newQuery = query
      .replace(/^(.*?～)(.*)(～[^～]*)$/, function (_, p1, p2, p3) {
          return p1.replace(/～/g, " ") + p2 + p3.replace(/～/g, " ");
      })
          .replace(/＝|=/g, " ")
          .replace(/　/g, " ")
          .replace(/０/g, "0")
          .replace(/１/g, "1")
          .replace(/２/g, "2")
          .replace(/３/g, "3")
          .replace(/４/g, "4")
          .replace(/５/g, "5")
          .replace(/６/g, "6")
          .replace(/７/g, "7")
          .replace(/８/g, "8")
          .replace(/９/g, "9")
          .replace(/Ⅰ/g, "I")
          .replace(/Ⅱ/g, "II")
          .replace(/Ⅲ/g, "III")
          .replace(/Ⅳ/g, "IV")
          .replace(/Ⅴ/g, "V")
          .replace(/Ⅵ/g, "VI")
          .replace(/Ⅶ/g, "VII")
          .replace(/Ⅷ/g, "VIII")
          .replace(/Ⅸ/g, "IX")
          .replace(/Ⅹ/g, "X")
          .replace(/－|-/g, " ")
          .replace(/―/g, " ")
          .replace(/\s{2,}/g, " ")
          .replace(/～/g, " ")
          .replace(/〜/g, " ")
          .replace(/━/g, " ")
          .replace(/\[/g, " ")
          .replace(/]/g, " ")
          .replace(/『/g, " ")
          .replace(/』/g, " ")
          .replace(/~/g, " ")
          .replace(/'/g, " ")
          .replace(/…/g, " ")
          .replace(/！/g, " ")
          .replace(/？/g, " ")
          .replace(/。/g, " ")
          .replace(/♥/g, " ")
          .replace(/☆/g, " ")
          .replace(/\//g, " ")
          .replace(/♡/g, " ")
          .replace(/★/g, " ")
          .replace(/‥/g, " ")
          .replace(/○/g, " ")
          .replace(/,/g, " ")
          .replace(/\./g, " ")
          .replace(/-/g, " ")
          .replace(/【/g, " ")
          .replace(/】/g, " ")
          .replace(/◆/g, " ")
          .replace(/×/g, " ")
          .replace(/▼/g, " ")
          .replace(/’/g, " ")
          .replace(/&/g, " ")
          .replace(/＇/g, " ")
          .replace(/"/g, " ")
          .replace(/？/g, " ")
          //.replace(/ー/g, " ")
          .replace(/\.\.\./g, " ")
          .replace(/．/g, ' ')
          .replace(/～っ.*/, '')
          .replace(/・/g, " ")
          .replace(/\(.*?\)/g, '')
          .replace(/\（.*?\）/g, ' ')
          .replace(/\s+/g, " ")
          .trim();
      return newQuery;
  }

  function genScoreRowInfo(title, page, info) {
      const favicon = getFavicon(page);
      const name = page.name.split('-')[0];
      let score = '0.00';
      let count = NO_MATCH_DATA;
      let searchUrl;

      if (page.searchApi === 'https://erogamescape.org/~ap2/ero/toukei_kaiseki/kensaku.php?category=game&word_category=name&word={kw}&mode=normal') {
          searchUrl = page.searchApi.replace('{kw}', encodeURIComponent(normalizeEGS(title)));
      } else {
          searchUrl = page.searchApi.replace('{kw}', encodeURIComponent(normalizeQuery(title)));
      }

      let url = searchUrl;
      if (info && info.url) {
          if (!isNaN(Number(info.score))) {
              score = Number(info.score || 0).toFixed(2);
          }
          else {
              score = '0.00';
          }
          count = (info.count || 0) + ' votes';
          url = info.url;
      }
      return { favicon, count, score, url, searchUrl, name };
  }

zhifengle commented 11 months ago

fixed NaN. Search Rance10 in EGS by alias ランス10－決戦－ or ランス10 決戦, no result. So I don't think using alias has improvement.

I think use unicode range to replace special char is better https://github.com/zhifengle/gm_scripts/blob/2f0a294481ac0a0f91bc94b5dc7c7e5fb2922b9c/src/utils/utils.ts#L114-L155

Then, add unit test. https://github.com/zhifengle/gm_scripts/blob/2f0a294481ac0a0f91bc94b5dc7c7e5fb2922b9c/src/utils/utils.test.ts#L23-L32

Kamikadashi commented 11 months ago

My regex would remove －決戦－ in such a case, so the page would be found. However, it’s hard to predict which alias to use because naming doesn’t follow any particular convention. That’s why it would be beneficial to have multiple attempts if the result is not found on the first try.

In general, shortening names on top of the search by release date you've added allows the script to find many more games, as long as the search request is no less than two Japanese symbols or at least one full English word.

The current filter does not replace all the characters that mine does ((for example ・,etc.) all the symbols I’ve included appear in names on vndb) and it breaks full-width text like this: ＲａｇｎａｒｏｋＩｘｃａ.

P.S. I've updated the filters above.

zhifengle commented 11 months ago

The filters have updated. It needs more test. https://github.com/zhifengle/gm_scripts/blob/65c7d46adcb7383fe4552021e0c681177c23e8b0/src/sites/erogamescape.ts#L38-L91

https://github.com/zhifengle/gm_scripts/blob/65c7d46adcb7383fe4552021e0c681177c23e8b0/src/utils/utils.ts#L128-L169

When staying in VNDB, It would search EGS with multiple try.

https://github.com/zhifengle/gm_scripts/blob/65c7d46adcb7383fe4552021e0c681177c23e8b0/src/scripts/score_comparation_helper/pages/erogamescape.ts#L43-L45

Kamikadashi commented 11 months ago

I tried installing the latest version without making any changes, but had to modify genScoreRowInfo to make specialized replacements work for EGS. I will try testing it more when I have time. Could you please explain how multiple attempts are supposed to work on VNDB?

P.S. I found some instances where the script fetched information for the wrong game. This happened even though the correct game was listed on the search page with a release date identical to that on VNDB. Interestingly, the game from which the information was fetched had a different release date. Links: the game in question this one has a slightly different problem another one and this page on EGS showed 0.0 for vndb score even though there are two votes

zhifengle commented 11 months ago

fixed v20719, v8716. EGS shows 0.0 and two votes, beacause the game has no score in VNDB's seach page. I guess votes amount is not enough. fixed it by trying to extract score from game page in VNDB.

https://github.com/zhifengle/gm_scripts/blob/b7024b9b669e415ccb7234e84993e54fac186313/src/sites/common.ts#L177-L196 The function searchDataByNames receives another function and iter over call that function.

  // try multiple query
  getSearchResult: (info: SearchResult) =>
    searchDataByNames(info, searchGameSubject),

Kamikadashi commented 11 months ago

Some of the errors in game detection that I’ve encountered on the newest version: 0 1 2 3 4 5 6 7 8

In cases like 0 and 7, I believe it should have tried ‘Si-Nis-Kanto’ first, then ‘至上のソラ’, and only then ‘シニシカント’ if all the previous attempts were unsuccessful.

Also, ＲａｇｎａｒｏｋＩｘｃａ broke because the script doesn’t add a space now. However, I believe this can be fixed by adding the rule to the reviseTitle function, as there don’t seem to be any other games with a similar problem: 'ＲａｇｎａｒｏｋＩｘｃａ': 'Ragnarok Ixca',

zhifengle commented 11 months ago

fixed: v10733, v7472, v8095,v1919,v295. Change function searchGameSubject in EGS.

Game v12775 does not have search result in EGS.

Add global variables: VNDB_REVISE_TITLE_DICT and EGS_REVISE_TITLE_DICT for user to revise title.

Kamikadashi commented 11 months ago

Here are a couple of links that don’t work as they should: https://vndb.org/v10825 https://vndb.org/v3091 https://vndb.org/v11683 https://vndb.org/v43470 https://vndb.org/v5315 v3091 and v10825 were fine before the last update. For games with the names like PARTS ─パーツ─, it’s probably worth treating PARTS パーツ, PARTS, and パーツ as aliases and search for the item that has the same year and month of release among them.

zhifengle commented 11 months ago

Fixed v10825, v3091. When browsing v11683, v43470 or v5315, There is no search result in EGS. It would try to use shorter query string. For example, convert "The Murder of Sonic the Hedgehog" to "The Murder" for searching. All of the search results are wrong, but fuse.js can filter out a result by using "The Murder". Skipping the fuse search step and comparing release date can solve it.

Spliting game's name to aliases for searching has been applied in EGS. Some websites restrict the search frequency of users. I do not recommend searching frequently.

Kamikadashi commented 11 months ago

Thank you! I've checked with the list of problematic games I've accumulated so far, and this version performs the best. Now, visual novels that are on EGS but have a slightly different release date from the first complete release on VNDB don't automatically show the score. However, I guess this is preferable to constantly fetching scores for incorrect games. Here are some examples: - VNDB v1919 - VNDB v7320 - VNDB v8733

In such cases, it's probably possible to set a threshold for name equivalence with the rawName and fetch them anyway if it's higher than a certain percentage/higher than that of all the other search results. However, I'm still unsure about the details of such an implementation, and this is a relatively minor issue so far.

The script still has problems with the following games that are on EGS with correct release dates though: - MUSICUS fetches scores for the 非18禁 version with fewer votes but the same release date. - VNDB v2671 - VNDB v6725 - VNDB v4421 - VNDB v8006 - VNDB v6836 - VNDB v865 - VNDB v7452 - VNDB v31044 - VNDB v2040 - VNDB v1637

When attempting to search with the long normalized query, I guess it's possible to search not with 'すてぃーる My はぁと Rhapsody of moonlight', but 'すてぃ?ght' for example. Or 'まじか?ュート' instead of 'まじかるサマー壊決天使エグゼキュート', etc. This may help fix the issue when the game cannot be found due to a difference of only one or two characters.

P.S. 2dfan icon seems to be broken.

zhifengle commented 11 months ago

All of the above issues have been fixed. The feature of using ? in query string is been added.

Kamikadashi commented 11 months ago

Thanks! From what I can see, most problems seem to have been fixed, but the issue of fetching scores for incorrect games has returned. Here are the problematic links: - VNDB v3436 - VNDB v13666 - VNDB v11683 - VNDB v43470 - VNDB v37797 - VNDB v36914 - VNDB v21704 - VNDB v18641 - VNDB v8687 - VNDB v28343 - VNDB v4102 - VNDB v6723 (In cases like this, it should probably fetch the score of the entry with the most scores.) - VNDB v9308 - EGS - EGS - EGS - EGS - EGS - EGS - EGS - EGS - EGS - EGS - EGS - EGS - EGS - EGS - EGS - EGS - EGS - EGS - EGS - EGS - EGS - EGS - EGS - EGS - EGS - EGS - EGS - EGS

Also, fetching from Bangumi and 2DFan appears to be broken. Can this be fixed? I’ll test the script more. I’ve noticed that EGS sometimes fetches average scores and at other times, it fetches weighted scores from VNDB.

Kamikadashi commented 11 months ago

I didn’t see that the script was updated to version 0.21. Unfortunately, the newest version doesn’t display any scores in the two browsers I tried.

zhifengle commented 11 months ago

Can you open this link: https://registry.npmmirror.com/tiny-segmenter/0.2.0/files/dist/tiny-segmenter-0.2.0.js New version do not bundle tiny-segmenter. I have tested script, it's ok.

The games in EGS do not been fixed, I will deal that later.

Kamikadashi commented 11 months ago

The link works fine. However, the script doesn’t run when the line // @require https://registry.npmmirror.com/tiny-segmenter/0.2.0/files/dist/tiny-segmenter-0.2.0.js is present. When I removed this line and copied the contents directly into the script, it started working properly. Just copying the contents didn’t work. I had to delete the line itself to make it work.

Uncaught (in promise) TypeError: input.split is not a function at userscript.html?name=%25E8%25AF%2584%25E5%2588%2586%25E5%25AF%25B9%25E6%25AF%2594%25E5%258A%25A9%25E6%2589%258B.user.js&id=afc406ec-ad46-4d36-ae09-2f4522e15bf6:100:19 at Proxy.<anonymous> (userscript.html?name=%25E8%25AF%2584%25E5%2588%2586%25E5%25AF%25B9%25E6%25AF%2594%25E5%258A%25A9%25E6%2589%258B.user.js&id=afc406ec-ad46-4d36-ae09-2f4522e15bf6:221:1) at Gt (<anonymous>:9:89) at userscript.html?name=%25E8%25AF%2584%25E5%2588%2586%25E5%25AF%25B9%25E6%25AF%2594%25E5%258A%25A9%25E6%2589%258B.user.js&id=afc406ec-ad46-4d36-ae09-2f4522e15bf6:1:87 at window.__f__lq78tn8n.4y (userscript.html?name=%25E8%25AF%2584%25E5%2588%2586%25E5%25AF%25B9%25E6%25AF%2594%25E5%258A%25A9%25E6%2589%258B.user.js&id=afc406ec-ad46-4d36-ae09-2f4522e15bf6:1:490) at Gt (<anonymous>:9:89) at o (<anonymous>:78:23) at <anonymous>:80:411 at f (<anonymous>:74:452) I'm using Chrome.

zhifengle commented 11 months ago

The link works fine. However, the script doesn’t run when the line // @require https://registry.npmmirror.com/tiny-segmenter/0.2.0/files/dist/tiny-segmenter-0.2.0.js is present. When I removed this line and copied the contents directly into the script, it started working properly. Just copying the contents didn’t work. I had to delete the line itself to make it work.

Uncaught (in promise) TypeError: input.split is not a function at userscript.html?name=%25E8%25AF%2584%25E5%2588%2586%25E5%25AF%25B9%25E6%25AF%2594%25E5%258A%25A9%25E6%2589%258B.user.js&id=afc406ec-ad46-4d36-ae09-2f4522e15bf6:100:19 at Proxy.<anonymous> (userscript.html?name=%25E8%25AF%2584%25E5%2588%2586%25E5%25AF%25B9%25E6%25AF%2594%25E5%258A%25A9%25E6%2589%258B.user.js&id=afc406ec-ad46-4d36-ae09-2f4522e15bf6:221:1) at Gt (<anonymous>:9:89) at userscript.html?name=%25E8%25AF%2584%25E5%2588%2586%25E5%25AF%25B9%25E6%25AF%2594%25E5%258A%25A9%25E6%2589%258B.user.js&id=afc406ec-ad46-4d36-ae09-2f4522e15bf6:1:87 at window.__f__lq78tn8n.4y (userscript.html?name=%25E8%25AF%2584%25E5%2588%2586%25E5%25AF%25B9%25E6%25AF%2594%25E5%258A%25A9%25E6%2589%258B.user.js&id=afc406ec-ad46-4d36-ae09-2f4522e15bf6:1:490) at Gt (<anonymous>:9:89) at o (<anonymous>:78:23) at <anonymous>:80:411 at f (<anonymous>:74:452) I'm using Chrome.

Problematic links: - VNDB v4612(this one was working with the previous update 0.20) - EGS(this one was working with the previous update 0.20) - VNDB v865(this one was working with the previous update 0.20) - VNDB v6836(this one was working with the previous update 0.20) - VNDB v5315(this one was working with the previous update 0.20) - VNDB v3091(this one was working with the previous update 0.20) - VNDB v10825(this one was working with the previous update 0.20) - VNDB v7484(this one was working with the previous update 0.20)

We need to weigh whether to rely on the search results of the target website or the results of fuzzy filtering.

v4612 search results is correct, but it's name is different between VNDB and EGS, the one is ラレンティア～熱情の都～ and the other is LARENTIA ～熱情の都～. It make fuzzy results is wrong. The version 0.1.20 try to compare their release date. Two subject have same date.

However, the game v21704, its the search results of 月影の鎖 is wrong in EGS. The release date of 月影の鎖 -狂爛モラトリアム- is 2016-12-21 which is same month as 2016-12-29.

Version 0.1.22 would rollback search steps: compare date, then if there is no result, then try to fuzzy filtering.

TODO: game=3527

no result

game=8841#ad

no result

v21704

confusing result

Kamikadashi commented 11 months ago

The last update (0.22) seems to have broken [Rewrite]. Other than that, it works fine. However, I need to test it more thoroughly first.

Some more problematic links: - VNDB v17375 - VNDB v747 - VNDB v3591 - VNDB v15917 - VNDB v21905 - VNDB v15027 - VNDB v33536 - VNDB v6270 - VNDB v10897 - EGS - EGS - EGS - EGS - EGS - EGS - EGS - EGS

zhifengle commented 11 months ago

Fix some of above. for example: 痕 -きずあと- 、銀色、ななついろ★ドロップス

The search result of Lost Colors in EGS is wrong, but its name can pass fuzzy filtering. Some search results have same releate date, but their names can not pass fuzzy filtering. Some search results have different releate date, and their names can not pass fuzzy filtering.

Using shorter query string can get more results, but it would easily get wrong game URL. Some game's name is short, it's also confusing.

This is why fixing a problem can create new problems in what was previously working fine.

Kamikadashi commented 10 months ago

The last update broke the following:

- EGS - EGS - EGS - EGS - EGS - EGS - EGS - EGS - EGS - EGS

From what I’ve seen during the short test, the previous update was more successful in fetching scores more often.

P.S. I didn’t notice there were more updates because the version number didn’t change. However, some of the links above still don’t work, even on the latest version, though they were working fine on version 0.1.22. For some of them, a VNDB link is available. Maybe we could try checking if it’s present, and if so, fetch from it?

zhifengle commented 10 months ago

I have forgotten to build 0.1.23

Bangumi has restricted the search frequency. Sometimes, there is no search result.

Kamikadashi commented 10 months ago

Problematic links (v0.1.25): - VNDB v33264 - EGS - EGS - VNDB v20493 - VNDB v17375 - EGS - VNDB v41644

zhifengle commented 10 months ago

No search result: VNDB v33264 VNDB v20493

Fixed VNDB v41644 D.C.P.C. ～ダ・カーポプラスコミュニケーション～ LOVE FOREVER 1 Progress by revising title.

No search result: EGS Fixed VNDB v17375

v0.1.26 adds some global dictionary Objects for revising query and title

  window.VNDB_REVISE_TITLE_DICT = {
  // your config
  };
  window.EGS_REVISE_TITLE_DICT = {
  // your config
  };
  window.VNDB_REVISE_QUERY_DICT = {
  // for example: skip search with 'does not exist query'
  // 'does not exist query': 'SCH_SKIP_SEARCH',
  'D.C.P.C. ～ダ・カーポ プラスコミュニケーション～': 'D.C.～ダ・カーポ～',
  };
  window.EGS_REVISE_QUERY_DICT = {
  // revsise query of `まじかるサマー壊決天使エグゼキュート`
  // 'まじかるサマー壊決天使エグゼキュート':   'まじかるサマー壊決天使 エグゼキュート',
  // This way is also ok.
  // 'まじかるサマー壊決天使エグゼキュート':  'まじか?ュート'
  };

The game which has many releases is one page in VNDB, but it has many pages in EGS. Sometimes, removing version can fix it. https://github.com/zhifengle/gm_scripts/blob/d6562199ea3f5f512b5ad5aa1fd05508d77ce7ae/src/sites/utils/str.ts#L41-L49 D.C.P.C. ～ダ・カーポプラスコミュニケーション～ is a different release in VNDB. The original name is D.C.～ダ・カーポ～. To fix it, need to revise query. https://github.com/zhifengle/gm_scripts/blob/d6562199ea3f5f512b5ad5aa1fd05508d77ce7ae/src/sites/vndb.ts#L20-L24 If you want to skip search some game, just revise its name to SCH_SKIP_SEARCH.

Kamikadashi commented 6 months ago

The latest VNDB update appears to have broken the script; it doesn’t display any scores.

zhifengle commented 6 months ago

The latest VNDB update appears to have broken the script; it doesn’t display any scores.

Fixed it. https://github.com/zhifengle/gm_scripts/blob/6f0944febc45ef37c7bf78857fc33b351dfb05a1/src/scripts/score_comparation_helper/pages/vndb.ts#L17-L21

The infoSelector is used to locate the insertion point after an element.

zhifengle / gm_scripts

评分对比助手 is a great script, but some bugs need fixing #5