search

hildogjr / KiCost

Build cost spreadsheet for a KiCad project.
MIT License
527 stars 97 forks source link

Digikey api error #560

Closed BrianVod closed 8 months ago

BrianVod commented 9 months ago

I am using Kicad 7, and trying to use the kicost for Digikey costing as well as other distributers. All the distributers I am using seem to work fine and print everything to the Excel spread flawlessly. However, the Digikey run, errors out with the error: "ERROR:Internal error 'NoneType' object has no attribute 'products' (kicost-int.py:1568)" I am not sure how to correct this. This is trying to access a production api on digikey. I also tried the sandbox and got the same error. The api is : api_digikey.py.

So I am looking for some adivice on how to find this error. Thanks in Advance.

set-soft commented 9 months ago

"ERROR:Internal error 'NoneType' object has no attribute 'products' (kicost-int.py:1568)"

I'm not sure which source code is mentioned in the above text. Can you provide a screen capture? (What is int?)

BrianVod commented 9 months ago

I am not sure what "int" I am confused by that.

Here is the entire file: 

# -*- coding: utf-8 -*-
# MIT license
#
# Copyright (C) 2021 by Salvador E. Tropea / Instituto Nacional de Tecnologia Industrial
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.

# Author information.
__author__ = 'Salvador Eduardo Tropea'
__webpage__ = 'https://github.com/set-soft'
__company__ = 'Instituto Nacional de Tecnologia Industrial - Argentina'

# Libraries.
import pprint

# KiCost definitions.
from .. import DistData, KiCostError, W_NOINFO, ERR_SCRAPE, W_APIFAIL
# Distributors definitions.
from .distributor import distributor_class, QueryCache, hide_secrets
from .log__ import debug_detailed, debug_overview, debug_obsessive, warning

available = True
try:
    from kicost_digikey_api_v3 import by_digikey_pn, by_manf_pn, by_keyword, DigikeyError, DK_API
except ImportError:
    available = False

    class DK_API(object):
        api_ops = None

DIST_NAME = 'digikey'
# Specs known by KiCost
SPEC_NAMES = {'tolerance': 'tolerance',
              'power (watts)': 'power',
              'voltage - rated': 'voltage',
              'manufacturer': 'manf',
              'size / dimension': 'size',
              'temperature coefficient': 'temp_coeff',
              'frequency': 'frequency',
              'package / case': 'footprint'}
__all__ = ['api_digikey']

class api_digikey(distributor_class):
    name = 'Digi-Key'
    type = 'api'
    # Currently enabled only by request
    enabled = available
    url = 'https://developer.digikey.com/'  # Web site API information.
    api_distributors = [DIST_NAME]
    # Options supported by this API
    config_options = {'client_id':str,
                      'client_secret':str,
                      'exclude_market_place_products': bool,
                      'sandbox': bool,
                      'locale_site': ('US', 'CA', 'JP', 'UK', 'DE', 'AT', 'BE', 'DK', 'FI', 'GR', 'IE',
                                      'IT', 'LU', 'NL', 'NO', 'PT', 'ES', 'KR', 'HK', 'SG', 'CN', 'TW',
                                      'AU', 'FR', 'IN', 'NZ', 'SE', 'MX', 'CH', 'IL', 'PL', 'SK', 'SI',
                                      'LV', 'LT', 'EE', 'CZ', 'HU', 'BG', 'MY', 'ZA', 'RO', 'TH', 'PH'),
                      'locale_language': ('en', 'ja', 'de', 'fr', 'ko', 'zhs,' 'zht', 'it', 'es', 'he',
                                          'nl', 'sv', 'pl', 'fi', 'da', 'no'),
                      'locale_currency': ('USD', 'CAD', 'JPY', 'GBP', 'EUR', 'HKD', 'SGD', 'TWD', 'KRW',
                                          'AUD', 'NZD', 'INR', 'DKK', 'NOK', 'SEK', 'ILS', 'CNY', 'PLN',
                                          'CHF', 'CZK', 'HUF', 'RON', 'ZAR', 'MYR', 'THB', 'PHP'),
                      'locale_ship_to_country': str}
    # Legacy environment mapping, others will be automatically filled by `register`
    env_prefix = 'DIGIKEY'
    env_ops = {'DIGIKEY_STORAGE_PATH': 'cache_path',
               'DIGIKEY_CLIENT_SANDBOX': 'sandbox'}

    @staticmethod
    def configure(ops):
        DK_API.api_ops = {}
        cache_ttl = 7
        cache_path = None
        if not available:
            debug_obsessive('Digi-Key API not available')
            return
        for k, v in ops.items():
            if k == 'client_id':
                DK_API.id = v
            elif k == 'client_secret':
                DK_API.secret = v
            elif k == 'enable' and available:
                api_digikey.enabled = v
            elif k == 'sandbox':
                DK_API.sandbox = v
            elif k == 'cache_ttl':
                cache_ttl = v
            elif k == 'cache_path':
                cache_path = v
            elif k == 'exclude_market_place_products':
                DK_API.exclude_market_place_products = v
            elif k.startswith('locale_'):
                DK_API.api_ops[k] = v
        if api_digikey.enabled and (DK_API.id is None or DK_API.secret is None or cache_path is None):
            warning(W_APIFAIL, "Can't enable Digi-Key without a `client_id`, `client_secret` and `cache_path`")
            api_digikey.enabled = TRUE
        debug_obsessive('Digi-Key API configured to enabled {} id {} secret {} path {}'.
                        format(api_digikey.enabled, hide_secrets(DK_API.id), hide_secrets(DK_API.secret), cache_path))
        if not api_digikey.enabled:
            return
        # Try to configure the plug-in
        cache = QueryCache(cache_path, cache_ttl)
        try:
            DK_API.configure(cache, a_logger=distributor_class.logger)
        except DigikeyError as e:
            warning(W_APIFAIL, 'Failed to init Digi-Key API, reason: {}'.format(e.args[0]))
            api_digikey.enabled = FALSE

    @staticmethod
    def _query_part_info(parts, distributors, currency):
        '''Fill-in the parts with price/qty/etc info from KitSpace.'''
        if DIST_NAME not in distributors:
            debug_overview('# Skipping Digi-Key plug-in')
            return
        debug_overview('# Getting part data from Digi-Key...')
        field_cat = DIST_NAME + '#'

        # Setup progress bar to track progress of server queries.
        progress = distributor_class.progress(len(parts), distributor_class.logger)
        for part in parts:
            data = None
            # Get the Digi-Key P/N for this part
            part_stock = part.fields.get(field_cat)
            if part_stock:
                debug_detailed('\n**** Digi-Key P/N: {}'.format(part_stock))
                o = by_digikey_pn(part_stock)
                data = o.search()
                if data is None:
                    warning(W_NOINFO, 'The \'{}\' Digi-Key code is not valid'.format(part_stock))
                    o = by_keyword(part_stock)
                    data = o.search()
            else:
                # No Digi-Key P/N, search using the manufacturer code
                part_manf = part.fields.get('manf', '')
                part_code = part.fields.get('manf#')
                if part_code:
                    if part_manf:
                        debug_detailed('\n**** Manufacturer: {} P/N: {}'.format(part_manf, part_code))
                    else:
                        debug_detailed('\n**** P/N: {}'.format(part_code))
                    o = by_manf_pn(part_code)
                    data = o.search()
                    if data is None:
                        o = by_keyword(part_code)
                        data = o.search()
            if data is None:
                warning(W_NOINFO, 'No information found at Digi-Key for part/s \'{}\''.format(part.refs))
            else:
                debug_obsessive('* Part info before adding data:')
                debug_obsessive(pprint.pformat(part.__dict__))
                debug_obsessive('* Data found:')
                debug_obsessive(str(data))
                part.datasheet = data.primary_datasheet
                part.lifecycle = data.product_status.lower()
                specs = {sp.parameter.lower(): (sp.parameter, sp.value) for sp in data.parameters}
                specs['rohs'] = ('RoHS', data.ro_hs_status)
                part.update_specs(specs)
                dd = part.dd.get(DIST_NAME, DistData())
                dd.qty_increment = dd.moq = data.minimum_order_quantity
                dd.url = data.product_url
                dd.part_num = data.digi_key_part_number
                dd.qty_avail = data.quantity_available
                dd.currency = data.search_locale_used.currency
                dd.price_tiers = {p.break_quantity: p.unit_price for p in data.standard_pricing}
                # Extra information
                if data.product_description:
                    dd.extra_info['desc'] = data.product_description
                value = ''
                for spec in ('capacitance', 'resistance', 'inductance'):
                    val = specs.get(spec, None)
                    if val:
                        value += val[1] + ' '
                if value:
                    dd.extra_info['value'] = value
                for spec, name in SPEC_NAMES.items():
                    val = specs.get(spec, None)
                    if val:
                        dd.extra_info[name] = val[1]
                part.dd[DIST_NAME] = dd
                debug_obsessive('* Part info after adding data:')
                debug_obsessive(pprint.pformat(part.__dict__))
                debug_obsessive(pprint.pformat(dd.__dict__))
            progress.update(1)
        progress.close()

    @staticmethod
    def query_part_info(parts, distributors, currency):
        msg = None
        try:
            api_digikey._query_part_info(parts, distributors, currency)
        except DigikeyError as e:
            msg = e.args[0]
        if msg is not None:
            raise KiCostError(msg, ERR_SCRAPE)
        return set([DIST_NAME])

distributor_class.register(api_digikey, 100)
BrianVod commented 9 months ago

Here is the full error:

Starting cost processing ...

ERROR:Internal error: 'NoneType' object has no attribute 'products' (kicost - init.py:1568)

Let me know if you need anything else.

Thanks

--Brian

From: Salvador E. Tropea @.> Sent: Tuesday, March 5, 2024 8:36 AM To: hildogjr/KiCost @.> Cc: BrianVod @.>; Author @.> Subject: Re: [hildogjr/KiCost] Digikey api error (Issue #560)

"ERROR:Internal error 'NoneType' object has no attribute 'products' (kicost-int.py:1568)"

I'm not sure which source code is mentioned in the above text. Can you provide a screen capture? (What is int?)

— Reply to this email directly, view it on GitHub https://github.com/hildogjr/KiCost/issues/560#issuecomment-1978914222 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AHP4V5GFNSNXLDIDYZHHBH3YWXJ37AVCNFSM6AAAAABEFPU7C6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZYHEYTIMRSGI . You are receiving this because you authored the thread. https://github.com/notifications/beacon/AHP4V5CO3KVDREJZ27LPOODYWXJ37A5CNFSM6AAAAABEFPU7C6WGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTTV6PK24.gif Message ID: @. @.> >

set-soft commented 8 months ago

I fail to find any __init__.py file with line 1568. Try running KiCost with the --debug 10 option.

BrianVod commented 8 months ago

Ok Ill give that a try. Thanks for looking into this.

From: Salvador E. Tropea @.> Sent: Wednesday, March 6, 2024 6:32 AM To: hildogjr/KiCost @.> Cc: BrianVod @.>; Author @.> Subject: Re: [hildogjr/KiCost] Digikey api error (Issue #560)

I fail to find any init.py file with line 1568. Try running KiCost with the --debug 10 option.

— Reply to this email directly, view it on GitHub https://github.com/hildogjr/KiCost/issues/560#issuecomment-1980769847 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AHP4V5GFUYEZRLRKBUGWINDYW4EFDAVCNFSM6AAAAABEFPU7C6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBQG43DSOBUG4 . You are receiving this because you authored the thread. https://github.com/notifications/beacon/AHP4V5DMVEZQQRI3JFTQMQLYW4EFDA5CNFSM6AAAAABEFPU7C6WGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTTWCATDO.gif Message ID: @. @.> >

BrianVod commented 8 months ago

Ok, I finally got to run the debug and here is what it says:

ERROR:In configuration file: Error loading YAML while parsing a block mapping

in "C:\Users\brian.config\kicost\config.yaml", line 12, column 3

expected , but found ''

in "C:\Users\brian.config\kicost\config.yaml", line 16, column 6 (kicost - init.py:1568)

So it looks to be something in the config file. I used the sample on the site as a guide but I must have something incorrect.

Here is my config file text:

KiCost configuration file

kicost:

version: 1

Cache Time To Live in days, -1 is forever

Default is 7

cache_ttl: -1

Base directory for the APIs caches

cache_path: ~/.cache/kicost

APIs:

Digi-Key:

Digi-Key Client ID for a registered APP

client_id:'id'

#Digi-Key Client Secret for a registered APP

 client_secret: 'secret'

# Use the sandbox server, doesn't count the usage, but returns old data

 sandbox: false

# Only enabled if the client_id and client_secret are defined

 enable: true

# Directory for the APIs caches

 cache_path: ~/.cache/kicost/Digi-Key

# Exclude products offered by 3rd party associates (marketplace)

 exclude_market_place_products: false

Element14:

# Element14 includes: Farnell, Newark and CPC

# Element14 Product Search API key

 key: 'key'

# Only enabled if the key is defined

 enable: True

# Country used for Farnell queries.

# Supported countries: BG,CZ,DK,AT,CH,DE,IE,IL,UK,ES,EE,FI,FR,HU,IT,LT,

# LV,BE,NL,NO,PL,PT,RO,RU,SK,SI,SE,TR,CN,AU,NZ,HK,SG,MY,PH,TH,IN,KR,VN

# farnell_country: UK

# Country used for Newark queries.

 Supported countries: US,CA,MX

 newark_country: US

# Country used for CPC queries.

# Supported countries: UK,IE

# cpc_country: UK

# Directory for the APIs caches

 cache_path: ~/.cache/kicost/Element14

Mouser:

# Mouser Part API key

 key: a1c0ce7f-546f-473a-9b21-f815a4bd26f6

# Only enabled if the key is defined

 enable: True

# Directory for the APIs caches

 cache_path: ~/.cache/kicost/Mouser

Nexar:

# Nexar client ID

 client_id: 36cb4a25-ffc4-4613-b869-cbd2c9fefa87

# Nexar client secret

 client_secret: 8JycdKSpgvvuy39gmHs4CzDrTbHhJps8TaAG

# Only enabled if the client_id and client_secret are defined

 enable: True

# Country where we are buying

 country: US

# Directory for the APIs caches

 cache_path: ~/.cache/kicost/Nexar

TME:

# TME token (anonymous or private)

# token: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

# TME application secret

# app_secret: XXXXXXXXXXXXXXXXXXXX

# Only enabled if the token and app_secret are defined

# enable: false

# Country where we are buying

# country: US

# Language for the texts

# language: EN

# Directory for the APIs caches

# cache_path: ~/.cache/kicost/TME

From: Salvador E. Tropea @.> Sent: Wednesday, March 6, 2024 6:32 AM To: hildogjr/KiCost @.> Cc: BrianVod @.>; Author @.> Subject: Re: [hildogjr/KiCost] Digikey api error (Issue #560)

I fail to find any init.py file with line 1568. Try running KiCost with the --debug 10 option.

— Reply to this email directly, view it on GitHub https://github.com/hildogjr/KiCost/issues/560#issuecomment-1980769847 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AHP4V5GFUYEZRLRKBUGWINDYW4EFDAVCNFSM6AAAAABEFPU7C6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBQG43DSOBUG4 . You are receiving this because you authored the thread. https://github.com/notifications/beacon/AHP4V5DMVEZQQRI3JFTQMQLYW4EFDA5CNFSM6AAAAABEFPU7C6WGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTTWCATDO.gif Message ID: @. @.> >

BrianVod commented 8 months ago

A little more information for you:

If I do not use the config file then Kicost runs and Farnell, Mouser and Newark run fine and populate the CBOM as it should.

But when running with Digikey in the mix then it fails, also when I use NEXAR it fails with :

ERROR:Internal error: 'data' (kicost - init.py:1568)

So to sum up whats happening;

With the config file:

  1. Kicost does not start with the config file and debug shows the error I sent earlier

Without the config file:

  1. Kicost Runs
  2. Digikey Fails
  3. NEXAR Fails : ERROR:Internal error: 'data' (kicost - init.py:1568)
  4. Newark Runs
  5. Farnell Runs
  6. Mouser Runs

It seemed at first NEXAR worked, then it stopped. Is there a way to uninstall Kicost and start over as if only Kicad exits? I tried “pip uninstall Kicost, it said it removed it but everything was still there. I tried reinstalling over the top but that didn’t change anything.

If I could uninstall it I would try a fresh install and see of that helps.

Thanks,

--Brian

From: Salvador E. Tropea @.> Sent: Wednesday, March 6, 2024 6:32 AM To: hildogjr/KiCost @.> Cc: BrianVod @.>; Author @.> Subject: Re: [hildogjr/KiCost] Digikey api error (Issue #560)

I fail to find any init.py file with line 1568. Try running KiCost with the --debug 10 option.

— Reply to this email directly, view it on GitHub https://github.com/hildogjr/KiCost/issues/560#issuecomment-1980769847 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AHP4V5GFUYEZRLRKBUGWINDYW4EFDAVCNFSM6AAAAABEFPU7C6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBQG43DSOBUG4 . You are receiving this because you authored the thread. https://github.com/notifications/beacon/AHP4V5DMVEZQQRI3JFTQMQLYW4EFDA5CNFSM6AAAAABEFPU7C6WGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTTWCATDO.gif Message ID: @. @.> >

set-soft commented 8 months ago

Ok, I finally got to run the debug and here is what it says: ERROR:In configuration file: Error loading YAML while parsing a block mapping

Indentation of YAML files is important. Your configuration file is broken, please read about YAML format.

Note that in order to get some cost you need a valid configuration file.

BrianVod commented 8 months ago

The config file seems to be working now but I still get an error when I enable NEXAR. I copied the error from Kicost. If I don’t enable the nexar api then I can get costs from Newark, Mouser, Farnell, which tells me element14 api is working.. Digikey still not working but I think that’s on their end.

“Starting cost processing ...

ERROR:Internal error: 'data' (kicost - init.py:1568)”

I don’t where or what this could be coming from.

Let me know if need something from me to review and I will get it to you.

Thanks,

--Brian

From: Salvador E. Tropea @.> Sent: Thursday, March 7, 2024 10:51 AM To: hildogjr/KiCost @.> Cc: BrianVod @.>; Author @.> Subject: Re: [hildogjr/KiCost] Digikey api error (Issue #560)

Ok, I finally got to run the debug and here is what it says: ERROR:In configuration file: Error loading YAML while parsing a block mapping

Indentation of YAML files is important. Your configuration file is broken, please read about YAML format.

Note that in order to get some cost you need a valid configuration file.

— Reply to this email directly, view it on GitHub https://github.com/hildogjr/KiCost/issues/560#issuecomment-1983989837 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AHP4V5HE5OCPOZNYJQS4U6DYXCLGPAVCNFSM6AAAAABEFPU7C6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBTHE4DSOBTG4 . You are receiving this because you authored the thread. https://github.com/notifications/beacon/AHP4V5B4KFBC4DKP3LTUOH3YXCLGPA5CNFSM6AAAAABEFPU7C6WGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTTWIFEE2.gif Message ID: @. @.> >

BrianVod commented 8 months ago

one other note, what I also getting is this error in the terminal: WARNING:(WC018) Unknown option Element14 includes for API Element14 (kicost - log.py:107) WARNING:(WC018) Unknown option Supported countries for API Element14 (kicost - log.py:107)