Need help with module - Githubissues

Th3Whit3Wolf commented 3 years ago

I am trying to write a module fro vscode marketplace.

I am trying to turn this curl command ("ritwickdey.LiveServer" is what needs to be a variable)

curl 'https://marketplace.visualstudio.com/_apis/public/gallery/extensionquery' \
  -H 'Accept: application/json;api-version=6.1-preview.1' \
  -H 'Content-Type: application/json' \
  --data-binary '{ "filters":[ { "criteria":[ { "filterType":8, "value":"Microsoft.VisualStudio.Code" }, { "filterType":10, "value": "ritwickdey.LiveServer" }, { "filterType":12, "value":"4096" } ], "pageNumber":1, "pageSize":2, "sortBy":0, "sortOrder":0 } ], "assetTypes":[ ], "flags":946 }' \
   | jq -r '.results[0].extensions[0] | {version: .versions[0].version}'

into a module and here is what I have so far

# nvchecker_source/vsmarketplace.py

from nvchecker.api import (
  VersionResult, Entry, AsyncCache, KeyManager,
  TemporaryError, session, GetVersionError,
)

API_URL = 'https://marketplace.visualstudio.com/_apis/public/gallery/extensionquery'

HEADERS = {
  'Accept': 'application/json;api-version=6.1-preview.1',
  'Content-Type': 'application/json'
}

QUERY_EXT =  '{"assetTypes":["Microsoft.VisualStudio.Services.Icons.Default","Microsoft.VisualStudio.Services.Icons.Branding","Microsoft.VisualStudio.Services.Icons.Small"],"filters":[{"criteria":[{"filterType":8,"value":"Microsoft.VisualStudio.Code"},{"filterType":10,"value":"%s"},{"filterType":12,"value":"37888"}],"direction":2,"pageSize":54,"pageNumber":1,"sortBy":0,"sortOrder":0,"pagingToken":null}],"flags":870}'

async def get_version(name: str, conf: Entry, *, cache: AsyncCache, **kwargs):
  name = conf.get('vsmarketplace') or name

  q = QUERY_EXT % name

  res = await session.post(
    API_URL,
    headers = HEADERS,
    json = q,
  )
  j = res.json()

  version = j['results'][0]['extensions'][0]['versions'][0]['version']
  return version

I get this error (sorry I'm using nix so the file paths are a bit long)

vscodeMarketplace-extension: unexpected error happened error=HTTPError(400, 'Bad Request', HTTPResponse(_body=None,_error_is_response_code=True,buffer=<_io.BytesIO object at 0x7feb0dcad590>,code=400,effective_url='https://marketplace.visualstudio.com/_apis/public/gallery/extensionquery',error=HTTP 400: Bad Request,headers=<tornado.httputil.HTTPHeaders object at 0x7feb0dcab5b0>,reason='Bad Request',request=<tornado.httpclient.HTTPRequest object at 0x7feb0df9d070>,request_time=0.6279661655426025,start_time=1626261859.5147,time_info={'queue': 2.0503997802734375e-05, 'namelookup': 0.001812, 'connect': 0.016938, 'appconnect': 0.104066, 'pretransfer': 0.104295, 'starttransfer': 0.105926, 'total': 0.61434, 'redirect': 0.0}))
    Traceback (most recent call last):
      File "/nix/store/pdwla0qblzyrsfifx240d52pmwm9pzgz-python3.8-nvchecker-2.4.dev0/lib/python3.8/site-packages/nvchecker/util.py", line 262, in run_one
        version = await self.func(
      File "/nix/store/pdwla0qblzyrsfifx240d52pmwm9pzgz-python3.8-nvchecker-2.4.dev0/lib/python3.8/site-packages/nvchecker_source/vsmarketplace.py", line 24, in get_version
        res = await session.post(
      File "/nix/store/pdwla0qblzyrsfifx240d52pmwm9pzgz-python3.8-nvchecker-2.4.dev0/lib/python3.8/site-packages/nvchecker/httpclient/base.py", line 54, in post
        return await self.request(
      File "/nix/store/pdwla0qblzyrsfifx240d52pmwm9pzgz-python3.8-nvchecker-2.4.dev0/lib/python3.8/site-packages/nvchecker/httpclient/base.py", line 78, in request
        return await self.request_impl(
      File "/nix/store/pdwla0qblzyrsfifx240d52pmwm9pzgz-python3.8-nvchecker-2.4.dev0/lib/python3.8/site-packages/nvchecker/httpclient/tornado_httpclient.py", line 87, in request_impl
        raise err_cls(
    nvchecker.httpclient.base.HTTPError: (400, 'Bad Request', HTTPResponse(_body=None,_error_is_response_code=True,buffer=<_io.BytesIO object at 0x7feb0dcad590>,code=400,effective_url='https://marketplace.visualstudio.com/_apis/public/gallery/extensionquery',error=HTTP 400: Bad Request,headers=<tornado.httputil.HTTPHeaders object at 0x7feb0dcab5b0>,reason='Bad Request',request=<tornado.httpclient.HTTPRequest object at 0x7feb0df9d070>,request_time=0.6279661655426025,start_time=1626261859.5147,time_info={'queue': 2.0503997802734375e-05, 'namelookup': 0.001812, 'connect': 0.016938, 'appconnect': 0.104066, 'pretransfer': 0.104295, 'starttransfer': 0.105926, 'total': 0.61434, 'redirect': 0.0}))

I am not sure what I am doing wrong.

lilydjwg commented 3 years ago

What you pass to the json argument is a string, not a dict. Try to remove the outmost quotes and rewrite it as a proper Python dict.

Th3Whit3Wolf commented 3 years ago

Should this be working?

from nvchecker.api import (
  VersionResult, Entry, AsyncCache, KeyManager,
  TemporaryError, session, GetVersionError,
)

API_URL = 'https://marketplace.visualstudio.com/_apis/public/gallery/extensionquery'

HEADERS = {
  'Accept': 'application/json;api-version=6.1-preview.1',
  'Content-Type': 'application/json'
}

async def get_version(name: str, conf: Entry, *, cache: AsyncCache, **kwargs):
  name = conf.get('vsmarketplace') or name

  q = {
    'filters': [
      {
        'criteria': [
          {
            'filterType': 8,
            'value': 'Microsoft.VisualStudio.Code'
          },
          {
            'filterType': 10,
            'value': name
          },
          {
            'filterType': 12,
            'value': '4096'
          }
        ],
        'pageNumber': 1,
        'pageSize': 2,
        'sortBy': 0,
        'sortOrder': 0
      }
    ],
    'assetTypes': [],
    'flags': 946
  }

  res = await session.post(
    API_URL,
    headers = HEADERS,
    json = q,
  )
  j = res.json()

  version = j['results'][0]['extensions'][0]['versions'][0]['version']
  return version

It still fails with

[E 07-14 14:29:22.383 core:328] vscodeMarketplace-extension: unexpected error happened error=IndexError('list index out of range')
    Traceback (most recent call last):
      File "/nix/store/z4dr34bha47409m75c4a735vlmdd6y2p-python3.8-nvchecker-2.4.dev0/lib/python3.8/site-packages/nvchecker/util.py", line 262, in run_one
        version = await self.func(
      File "/nix/store/z4dr34bha47409m75c4a735vlmdd6y2p-python3.8-nvchecker-2.4.dev0/lib/python3.8/site-packages/nvchecker_source/vsmarketplace.py", line 55, in get_version
        version = extension[0]
    IndexError: list index out of range

Th3Whit3Wolf commented 3 years ago

This is the json

{
  "results": [
    {
      "extensions": [
        {
          "publisher": {
            "publisherId": "17fd9a78-e430-4a78-add2-ade4a8830352",
            "publisherName": "ritwickdey",
            "displayName": "Ritwick Dey",
            "flags": "verified"
          },
          "extensionId": "b63c44fd-0457-4696-99e9-dbfdf70d77de",
          "extensionName": "LiveServer",
          "displayName": "Live Server",
          "flags": "validated, public",
          "lastUpdated": "2019-04-17T10:42:11.2Z",
          "publishedDate": "2017-06-27T21:14:04.557Z",
          "releaseDate": "2017-06-27T21:14:04.557Z",
          "shortDescription": "Launch a development local Server with live reload feature for static & dynamic pages",
          "versions": [
            {
              "version": "5.6.1",
              "flags": "validated",
              "lastUpdated": "2019-04-17T10:44:32.54Z",
              "files": [
                {
                  "assetType": "Microsoft.VisualStudio.Code.Manifest",
                  "source": "https://ritwickdey.gallerycdn.vsassets.io/extensions/ritwickdey/liveserver/5.6.1/1555497731217/Microsoft.VisualStudio.Code.Manifest"
                },
                {
                  "assetType": "Microsoft.VisualStudio.Services.Content.Changelog",
                  "source": "https://ritwickdey.gallerycdn.vsassets.io/extensions/ritwickdey/liveserver/5.6.1/1555497731217/Microsoft.VisualStudio.Services.Content.Changelog"
                },
                {
                  "assetType": "Microsoft.VisualStudio.Services.Content.Details",
                  "source": "https://ritwickdey.gallerycdn.vsassets.io/extensions/ritwickdey/liveserver/5.6.1/1555497731217/Microsoft.VisualStudio.Services.Content.Details"
                },
                {
                  "assetType": "Microsoft.VisualStudio.Services.Content.License",
                  "source": "https://ritwickdey.gallerycdn.vsassets.io/extensions/ritwickdey/liveserver/5.6.1/1555497731217/Microsoft.VisualStudio.Services.Content.License"
                },
                {
                  "assetType": "Microsoft.VisualStudio.Services.Icons.Default",
                  "source": "https://ritwickdey.gallerycdn.vsassets.io/extensions/ritwickdey/liveserver/5.6.1/1555497731217/Microsoft.VisualStudio.Services.Icons.Default"
                },
                {
                  "assetType": "Microsoft.VisualStudio.Services.Icons.Small",
                  "source": "https://ritwickdey.gallerycdn.vsassets.io/extensions/ritwickdey/liveserver/5.6.1/1555497731217/Microsoft.VisualStudio.Services.Icons.Small"
                },
                {
                  "assetType": "Microsoft.VisualStudio.Services.VsixManifest",
                  "source": "https://ritwickdey.gallerycdn.vsassets.io/extensions/ritwickdey/liveserver/5.6.1/1555497731217/Microsoft.VisualStudio.Services.VsixManifest"
                },
                {
                  "assetType": "Microsoft.VisualStudio.Services.VSIXPackage",
                  "source": "https://ritwickdey.gallerycdn.vsassets.io/extensions/ritwickdey/liveserver/5.6.1/1555497731217/Microsoft.VisualStudio.Services.VSIXPackage"
                }
              ],
              "properties": [
                {
                  "key": "Microsoft.VisualStudio.Services.Branding.Color",
                  "value": "#41205f"
                },
                {
                  "key": "Microsoft.VisualStudio.Services.Branding.Theme",
                  "value": "dark"
                },
                {
                  "key": "Microsoft.VisualStudio.Services.Links.Getstarted",
                  "value": "https://github.com/ritwickdey/vscode-live-server.git"
                },
                {
                  "key": "Microsoft.VisualStudio.Services.Links.Support",
                  "value": "https://github.com/ritwickdey/vscode-live-server/issues"
                },
                {
                  "key": "Microsoft.VisualStudio.Services.Links.Learn",
                  "value": "https://ritwickdey.github.io/vscode-live-server/"
                },
                {
                  "key": "Microsoft.VisualStudio.Services.Links.Source",
                  "value": "https://github.com/ritwickdey/vscode-live-server.git"
                },
                {
                  "key": "Microsoft.VisualStudio.Services.Links.GitHub",
                  "value": "https://github.com/ritwickdey/vscode-live-server.git"
                },
                {
                  "key": "Microsoft.VisualStudio.Code.Engine",
                  "value": "^1.20.0"
                },
                {
                  "key": "Microsoft.VisualStudio.Services.GitHubFlavoredMarkdown",
                  "value": "true"
                },
                {
                  "key": "Microsoft.VisualStudio.Code.ExtensionDependencies",
                  "value": ""
                },
                {
                  "key": "Microsoft.VisualStudio.Code.ExtensionPack",
                  "value": ""
                },
                {
                  "key": "Microsoft.VisualStudio.Code.LocalizedLanguages",
                  "value": ""
                }
              ],
              "assetUri": "https://ritwickdey.gallerycdn.vsassets.io/extensions/ritwickdey/liveserver/5.6.1/1555497731217",
              "fallbackAssetUri": "https://ritwickdey.gallery.vsassets.io/_apis/public/gallery/publisher/ritwickdey/extension/LiveServer/5.6.1/assetbyname"
            }
          ],
          "statistics": [
            {
              "statisticName": "install",
              "value": 13229510
            },
            {
              "statisticName": "averagerating",
              "value": 4.429936408996582
            },
            {
              "statisticName": "ratingcount",
              "value": 314
            },
            {
              "statisticName": "trendingdaily",
              "value": 0.005847833767192669
            },
            {
              "statisticName": "trendingmonthly",
              "value": 5.176912818047757
            },
            {
              "statisticName": "trendingweekly",
              "value": 1.041942793232796
            },
            {
              "statisticName": "updateCount",
              "value": 3277467
            },
            {
              "statisticName": "weightedRating",
              "value": 4.4291113571543805
            },
            {
              "statisticName": "downloadCount",
              "value": 51887
            }
          ],
          "deploymentType": 0
        }
      ],
      "pagingToken": null,
      "resultMetadata": [
        {
          "metadataType": "ResultCount",
          "metadataItems": [
            {
              "name": "TotalCount",
              "count": 1
            }
          ]
        },
        {
          "metadataType": "Categories",
          "metadataItems": [
            {
              "name": "Other",
              "count": 1
            }
          ]
        }
      ]
    }
  ]
}

lilydjwg commented 3 years ago

The latest code works for me. The code in your traceback is different than the code you pasted. Check if the correct code file is loaded?

Th3Whit3Wolf commented 3 years ago

Line count is slightly off because I didn't include the MIT License header in the code I pasted. Same code and traceback though.

Th3Whit3Wolf commented 3 years ago

Would you mind testing this PR?

Th3Whit3Wolf commented 3 years ago

Closed with 195

lilydjwg / nvchecker

Need help with module #194