search

keymanapp / api.keyman.com

https://api.keyman.com/ source
3 stars 4 forks source link

bug: updated keyboards are not being reflected on api.keyman.com #240

Closed mcdurdin closed 8 months ago

mcdurdin commented 8 months ago

e.g.

https://downloads.keyman.com/keyboards/itrans_roman/ is now on 1.2.0:

[DIR]   [1.0.0/](https://downloads.keyman.com/keyboards/itrans_roman/1.0.0/)    2021-04-22 20:14    -    
[DIR]   [1.1.0/](https://downloads.keyman.com/keyboards/itrans_roman/1.1.0/)    2021-04-22 20:14    -    
[DIR]   [1.1.1/](https://downloads.keyman.com/keyboards/itrans_roman/1.1.1/)    2024-02-14 17:20    -    
[DIR]   [1.2.0/](https://downloads.keyman.com/keyboards/itrans_roman/1.2.0/)    2024-02-14 23:42    -

However as of time of writing this issue, https://api.keyman.com/keyboard/itrans_roman still says 1.1.1:

{
    "license": "mit",
    "description": "A phonetic (ITRANS) keyboard for typing Indic languages in Roman script with accents or diacritics.",
    "id": "itrans_roman",
    "languages": {
        "sa-latn": {
            "displayName": "Sanskrit (Latin)",
            "languageName": "Sanskrit",
            "scriptName": "Latin"
        },
        "hi-latn": {
            "displayName": "Hindi (Latin)",
            "languageName": "Hindi",
            "scriptName": "Latin"
        },
        "ta-latn": {
            "displayName": "Tamil (Latin)",
            "languageName": "Tamil",
            "scriptName": "Latin"
        }
    },
    "name": "Indic Romanization from ITRANS",
    "authorName": "Shree Devi Kumar",
    "lastModifiedDate": "2023-08-11T07:10:37Z",
    "sourcePath": "release/itrans/itrans_roman",
    "version": "1.1.1",
    "packageFilename": "itrans_roman.kmp",
    "jsFilename": "itrans_roman.js",
    "encodings": [
        "unicode"
    ],
    "jsFileSize": 15510,
    "packageFileSize": 11984,
    "packageIncludes": [
        "documentation",
        "welcome"
    ],
    "minKeymanVersion": "10.0",
    "helpLink": "https://help.keyman.com/keyboard/itrans_roman",
    "platformSupport": {
        "windows": "full",
        "macos": "full",
        "linux": "full",
        "android": "full",
        "ios": "full",
        "desktopWeb": "full",
        "mobileWeb": "full"
    }
}

I am not sure where the disconnect is, yet.

mcdurdin commented 8 months ago 
api.keyman.com
host    api.keyman.com
tier    TIER_PRODUCTION
schema  k0
database build status   Ready
last database build completed   Wed, 14 Feb 2024 23:53:04 +0000

Shows last database build was after keyboard was published

mcdurdin commented 8 months ago

https://github.com/keymanapp/keyboards/issues/2539#issuecomment-1945217287

mcdurdin commented 8 months ago

This build deployed the changes:

https://build.palaso.org/buildConfiguration/Keyboards_Upload/443010?buildTab=changes&showFiles=false&showGraph=true&showBuilds=true&expandRevisionsSection=true

mcdurdin commented 8 months ago

This build notified the sites about the update, at 1:29am UTC+7 https://build.palaso.org/buildConfiguration/Keyboards_NotifyApiKeymanComOfUpdate/443022?buildTab=log&focusLine=0&logView=flowAware&linesState=39.50

mcdurdin commented 8 months ago

Most recent database build on the container has this log:

[/var/www/html] build.sh parameters: <configure>
[/var/www/html] ## configure starting...
[bootstrap] Bootstrap starting
[bootstrap]   Downloading bootstrap.inc.sh
[bootstrap] Downloading _common files
[bootstrap]   Downloading _common/builder.inc.sh
[bootstrap]   Downloading _common/docker.inc.sh
[bootstrap]   Downloading _common/keyman-local-ports.inc.sh
[bootstrap]   Downloading _common/JsonApiFailure.php
[bootstrap]   Downloading _common/KeymanHosts.php
[bootstrap]   Downloading _common/KeymanSentry.php
[bootstrap]   Downloading _common/MarkdownHost.php
[bootstrap] All _common files downloaded
[bootstrap] Bootstrap complete
[/var/www/html] ## configure completed successfully
Serialzing DB rebuild
Rebuild requested
---- Sleep 15 Before Generating DB ----
Building database for k0
Attempting to wake database server at sqlsrv:Server=api-db-svc,1433;TrustServerCertificate=true;Encrypt=false;Database=keyboards (attempt 1/15)
Timestamp: 0.02
Timestamp: 0.08
Timestamp: 0.08
Timestamp: 0.06
Timestamp: 0.05
Timestamp: 0.02
Timestamp: 0.42
Timestamp: 15.40
Timestamp: 0.60
Creating database login k0
Timestamp: 0.06
Running /var/www/html/tools/db/build/create-database.sql
Timestamp: 0.10
Running /var/www/html/tools/db/build/clean-database.sql
Timestamp: 1.43
Running /var/www/html/tools/db/build/search.sql
Timestamp: 0.10
Running /var/www/html/tools/db/build/langtags.sql
Timestamp: 0.04
Running /var/www/html/.data/langtags.json.sql
Timestamp: 9.16
Running /var/www/html/.data/language-subtag-registry.sql
Timestamp: 2.03
Running /var/www/html/.data/iso639-3.sql
Timestamp: 0.68
Running /var/www/html/.data/iso639-3-name-index.sql
Timestamp: 0.60
Running /var/www/html/.data/ethnologue_language_codes.sql
Timestamp: 0.35
Running /var/www/html/.data/ethnologue_country_codes.sql
Timestamp: 0.04
Running /var/www/html/.data/ethnologue_language_index.sql
Timestamp: 3.76
Running /var/www/html/.data/keyboards.sql
Timestamp: 10.68
Running /var/www/html/.data/models.sql
Timestamp: 0.21
Running /var/www/html/tools/db/build/search-prepare-data.sql
Timestamp: 1.91
Running /var/www/html/tools/db/build/indexes.sql
Timestamp: 1.57
Running /var/www/html/tools/db/build/full-text-indexes.sql
Timestamp: 0.30
Running /var/www/html/tools/db/build/search-queries.sql
Timestamp: 0.04
Running /var/www/html/tools/db/build/v_keyboard_downloads_month.sql
Timestamp: 0.03
Running /var/www/html/tools/db/build/v_keyboard_downloads_total.sql
Timestamp: 0.03
Running /var/www/html/tools/db/build/sp_country_search_10.sql
Timestamp: 0.04
Running /var/www/html/tools/db/build/sp_increment_download.sql
Timestamp: 0.03
Running /var/www/html/tools/db/build/sp_keyboard_search.sql
Timestamp: 0.14
Running /var/www/html/tools/db/build/sp_keyboard_search_10.sql
Timestamp: 0.03
Running /var/www/html/tools/db/build/sp_keyboard_search_alphabetically.sql
Timestamp: 0.04
Running /var/www/html/tools/db/build/sp_keyboard_search_by_country.sql
Timestamp: 0.04
Running /var/www/html/tools/db/build/sp_keyboard_search_by_country_iso3166_code.sql
Timestamp: 0.03
Running /var/www/html/tools/db/build/sp_keyboard_search_by_id.sql
Timestamp: 0.04
Running /var/www/html/tools/db/build/sp_keyboard_search_by_keyboard.sql
Timestamp: 0.04
Running /var/www/html/tools/db/build/sp_keyboard_search_by_language.sql
Timestamp: 0.04
Running /var/www/html/tools/db/build/sp_keyboard_search_by_language_bcp47_tag.sql
Timestamp: 0.03
Running /var/www/html/tools/db/build/sp_keyboard_search_by_legacy_id.sql
Timestamp: 0.03
Running /var/www/html/tools/db/build/sp_keyboard_search_by_popularity.sql
Timestamp: 0.04
Running /var/www/html/tools/db/build/sp_keyboard_search_by_script.sql
Timestamp: 0.04
Running /var/www/html/tools/db/build/sp_keyboard_search_by_script_iso15924_code.sql
Timestamp: 0.03
Running /var/www/html/tools/db/build/sp_language_search_10.sql
Timestamp: 0.03
Running /var/www/html/tools/db/build/model-queries.sql
Timestamp: 0.03
Running /var/www/html/tools/db/build/legacy-queries.sql
Timestamp: 0.06
Running /var/www/html/tools/db/build/legacy-statistics.sql
Timestamp: 0.55
Running /var/www/html/.data/dbdatasources.sql
Timestamp: 0.03
Running /var/www/html/tools/db/build/cjk/cjk_database.sql
Timestamp: 1.82
Running /var/www/html/tools/db/build/cjk/chinese_pinyin_import.sql
Timestamp: 4.10
Running /var/www/html/tools/db/build/cjk/japanese_import.sql
Timestamp: 9.81
Success

Which doesn't give us enough detail to know what the keyboards data version is.

Now checking the data file uploaded to downloads.keyman.com

mcdurdin commented 8 months ago

https://downloads.keyman.com/data/

[   ]   keyboard_info.zip   2024-02-14 23:47    916K     
[   ]   model_info.zip  2024-02-08 19:24    29K
mcdurdin commented 8 months ago

keyboard_info.zip from downloads.keyman.com has the following metadata file for itrans_roman.keyboard_info, which shows the correct version number:

{
  "license": "mit",
  "description": "A phonetic (ITRANS) keyboard for typing Indic languages in Roman script with accents or diacritics.",
  "id": "itrans_roman",
  "languages": {
    "sa-Latn": {
      "displayName": "Sanskrit (Latin)",
      "languageName": "Sanskrit",
      "scriptName": "Latin"
    },
    "hi-Latn": {
      "displayName": "Hindi (Latin)",
      "languageName": "Hindi",
      "scriptName": "Latin"
    },
    "ta-Latn": {
      "displayName": "Tamil (Latin)",
      "languageName": "Tamil",
      "scriptName": "Latin"
    }
  },
  "name": "Indic Romanization from ITRANS",
  "authorName": "Shree Devi Kumar",
  "lastModifiedDate": "2024-02-14T16:37:41Z",
  "sourcePath": "release/itrans/itrans_roman",
  "version": "1.2.0",
  "packageFilename": "itrans_roman.kmp",
  "jsFilename": "itrans_roman.js",
  "encodings": [
    "unicode"
  ],
  "jsFileSize": 15573,
  "packageFileSize": 11999,
  "packageIncludes": [
    "documentation",
    "welcome"
  ],
  "minKeymanVersion": "10.0",
  "helpLink": "https://help.keyman.com/keyboard/itrans_roman",
  "platformSupport": {
    "windows": "full",
    "macos": "full",
    "linux": "full",
    "android": "full",
    "ios": "full",
    "desktopWeb": "full",
    "mobileWeb": "full"
  }
}
mcdurdin commented 8 months ago

.data/keyboard_info/itrans_roman.keyboard_info shows:

  "version": "1.1.1",

So something is going wrong downloading and unzipping the https://downloads.keyman.com/data/keyboard_info.zip.

mcdurdin commented 8 months ago

The download of that file is the wrong version, checked https://api.keyman.com/.data/keyboard_info.zip, and itrans_roman.keyboard_info in it has version 1.1.1 also:

  "version": "1.1.1",
mcdurdin commented 8 months ago

Looks like the database build is missing the 'force' flag which means it was caching the downloaded files instead of re-downloading.