Part search problem

[AGG2017]

Hi, It is a great tool. I have no issues installing it on windows 7 machine and some small problems adapting it to a Linux machine. I tried to insert in the database some of my parts from the last 10 years to see how it works (almost 1000 parts) and I had 2 main problems. One of them is that DK changed some old part numbers and they don't exist anymore or are under different name that are not linked to the old one - nothing can be done about that. The second problem is that the part number exists but it is not properly transferred as request. It is because in your script digikey.py you selected to use html.escape(part_id) but it is not the right one for this purpose. I changed that to: from urllib.parse import quote ... url = ... + quote(part_id,safe=''); This will make possible to search for part numbers like 24FC64F-I/P-ND, DS1307+T&RCT-ND, etc.

I also have experimented with adding a function to search for the legacy 1D barcodes. The input from the user is checked first if it looks like 22 digits legacy 1D barcode (like 1642795000000250526056) and it converts it to a Digikey part number before trying to search for product information. This makes the input to the script much easier when you have the parts in anti-static envelopes with barcode labels and by using the barcode reader to enter them in milliseconds to the script. That works just fine but the newer 2D barcodes produce much more data (including the Digikey part number inside) and it needs more time to make it working.

[alkgrove]

Hi, Thanks for the comments. I verified that partlocater has an issue with string with '/' in it. It looks like the urllib.parse quote has the option to parse '/' as not a path that html.escape doesn't do. I tested your solution and it seems to work. I post it to the github after doing some more checking on it. I haven't dealt with the barcodes. I do know that digikey has a specific barcode API. From their site: "Converts a legacy product barcode to Digi-Key and Manufacturer part number and quantity. The barcode this takes in is a one-dimensional barcode that was previously located on the label on the anti-static bag. This barcode can still be found on older products." Info on this can be found: https://developer.digikey.com/products/barcode/barcoding/productbarcode partlocater uses this API https://developer.digikey.com/products/product-information/partsearch/productdetails

It could be added to partlocater. The hard part of partlocater was really dealing with their restrictive OATH2 and token system. Once you have the token, you can query digikey with the barcode (both 1D and 2D) and get the proper digikey partnumber from them.

[AGG2017]

I used exactly the same Digikey API for the 1D barcode and a very small modification of the function digikey_get_part() mostly by changing the url. The barcode readers act as a keyboard and use the standard HID drivers so, nothing needs to be changed in the script to accept barcodes as input part number. The problem is that this reading is not the part number by itself but an internal DK product ID, bought quantity and some shipping information. The API converts the first part of the barcode (product ID) to DK part number. Then you can search in your database if it mutch or to request a DK part search if not found. I did it that way and it works. I recently started using partkeepr and I tried to batch enter some DK parts. Then found DK has API that can help and when trying to use it I also like you had hard time with their OATH2. Then started looking for a solution and found your project. Now with your help I solved OATH2 for my scripts but I have problem with the day limit of 1000. So, I implemented some kind of data caching. In the database I register all DK API requests and if a new request is already in the cached table I return the results from the table, not from DK API. That helps a lot if you check often (especially with the fast barcode reader) the same parts and that will avoid the API limits. For the 2D barcode I don't want to use the API (to avoid counting the requests and to get faster to the day limit) and because I see the DK part number is in the 2D barcode I will try to understand the format and extract the part number without the API help. DK switched recently to use only 2D barcodes so that needs to be done for faster part processing.

[GeorgeIoak]

I've been watching this project with hopes to contribute or combine with my efforts. @AGG2017 I have some code that I've used to scan bar codes and react to whether it's a 1D or 2D code being scanned. My goal was to scan in all the parts I have and enter them into a database. The nice thing about the 2D scans is that they also contain your order information so I tried to capture that as well. With that you get dates as well as everything in that order.

I'd like to share my efforts but not sure of the best way. If you have any interest in looking at my mess, I mean code, I can strip out my keys so you can take a look. It isn't anywhere near as polished as @alkgrove work but it might be helpful

[AGG2017]

@GeorgeIoak I'm trying to do exactly the same thing. A lot of components bought last 20+ years from DK and without a database search what I already have, it is not rare to buy the same stuff that I already have somewhere. But entering thousands of components in a database is not a job for a day and I started looking for a way to automate the process. Good 1D&2D barcode scanners are cheap enough here (under $50 CAD) and must be used to speed up entering the part number + the quantity bought. 1D format is pretty simple and using the DK API https://api.digikey.com/Barcoding/v3/ProductBarcodes/ it is very easy to get the real DK part name and then all the details for the product. The quantity is already in the 1D barcode. The 2D format is more complex and I may need to see only the code that is trying to extract the information from the recent DK 2D barcodes. I have some idea but I didn't put enough efforts to understand how to extract the most important elements. It looks 2D codes will last for many years ahead and it is better to become part of our projects. Thanks in advance if you want to share that part of the code.

[GeorgeIoak]

It's been a while since I've worked on this but I recall the last part that hung me up was determining the different categories and how to load them into my database. My code worked but I didn't have a way to handle if a new category was returned that I didn't have in my table.

I think this might help you with handling 1D and 2D. I've got notes and other test files but they all have my keys embedded in them so I would need an easy way to strip those from getting uploaded.

import http.client
import json

access_token = "xxxxx" #Enter your own access token here

headers = {
    'x-ibm-client-id': "secrect client ID",
    'x-digikey-locale-ste': "US",
    'x-digikey-locale-language': "en",
    'x-digikey-locale-currency': "USD",
    'x-digikey-locale-shiptocountry': "US",
    'authorization': access_token,
    'content-type': "application/json",
    'accept': "application/json"
    }

conn = http.client.HTTPSConnection("api.digikey.com")

barcode = input("Enter Barcode: ") #1165926000000004174667

if barcode[0] == "[":
    print("You have a QR Code")
    sonumber = barcode.split('')[4][2:]
    QtyShipped = barcode.split('')[8][1:]
    InvoiceId = barcode.split('')[5][3:]
    print("The Sales Order # is: " + sonumber)
    # Need to encode seperator characters
    # \u241E for Record Seperator and the Group Separator character is \u241D
    qrcode = barcode.replace("", "\u241D") #Format needed w/ Tera Scanner
#    qrcode = (barcode).replace(" ", "\u241D") #Format needed w/ Android App scanner

#   print(qrcode)
    payload = {
        "QRCode" : qrcode
    }
#    print(json.dumps(payload))

    conn.request("POST", "/services/barcode/v1/productqrcode", json.dumps(payload), headers=headers)
else:
    print("You have a 1D Bar Code")
    conn.request("GET", '/services/barcode/v1/productbarcode/'+ barcode, headers=headers)

res = conn.getresponse()
data = res.read()

print(data.decode("utf-8"))

y = json.loads(data)
print(y["ManufacturerPartNumber"])
print(y["DigiKeyPartNumber"])
DK_PartNum = y["DigiKeyPartNumber"]

payload = {
    "Part" : DK_PartNum,
    "IncludeAllAssociatedProducts" : 'false',
    "IncludeAllForUseWithProducts" : 'false'
}

conn.request("POST", "/services/partsearch/v2/partdetails", json.dumps(payload), headers)

res = conn.getresponse()
data = res.read()

data_return = json.loads(data)
print(json.dumps(data_return, indent=4))

with open(y["DigiKeyPartNumber"]+".json", "w") as write_file:
    json.dump(data_return, write_file, indent=4, sort_keys=True)
write_file.close()

#Let's get information about the associated Sales Order
conn.request("GET", "/services/orderStatus/v2/orderStatus/306086/" + sonumber + "?rootAccount=false", headers=headers)

res = conn.getresponse()
data = res.read()

data_return = json.loads(data)
print(json.dumps(data_return, indent=4))

with open(sonumber+".json", "w") as write_file:
    json.dump(data_return, write_file, indent=4, sort_keys=True)
write_file.close()

""" #Aim of this block is to get the list of the columns
columns = []
column = []
for data in data_return:
    column = list(data_return.keys())
    for col in column:
        if col not in columns:
            columns.append(col)

print(columns)

#Here we get values of the columns in the JSON file in the right order.   
value = []
values = [] 
for data in data_return:
    for i in columns:
        value.append(str(dict(data).get(i)))
    values.append(list(value)) 
    value.clear() """

[AGG2017]

This is exactly what I needed to complete this 2D puzzle. In fact, I think it is not QR code (which uses 3 square markers at 3 of the edges) but Data Matrix format (with two solid line markers at 2 of the sides). It doesn't matter. Thanks again!

[GeorgeIoak]

This part of my code worked very well in my testing. My struggles happened with digesting the json data and inserting it into the database. The returned data has a LimitedTaxonomy Table and then a Children Table and I had trouble with determining how many levels that could have.

[alkgrove]

I checked in the fix to improperly quoted URL, thank you for finding and suggesting the fix.

Partlocater was made for new designs to find parts and putting them in along with symbol, footprint and basename references. I did some lame additions for inventory but it really is not a good inventory database. A barcode app would be different as it is more about managing existing inventory. That is, you may want a separate database which has tables for barcode, DK partnumber, quantity and location of parts. You may want other entries as well regarding date of purchase, sales order number and so on. It would have an entry for each bag of parts.

I considered at one time batch mode entry which goes through a text file list of parts and adds the ones not currently in the database. The idea was command line variation of partlocater that was passed a text file list of parts and the database. This is something I could do if there is any interest.

You'd still need a barcode app that would add to it's own inventory database. It would be a simple matter then to do a SQL query on that database for a list of unique DK partnumbers and could even be further filtered by excluding the ones already in the partlocater database. This is written to a text file that could be bulk added to the partlocater database.

I would think that doing an inventory database biggest advantage is knowing what you have, how many and where your inventory is. You may want entry boxes for existing quantity and where it can be found.

This would be my guess, if I were more organized. I keep parts in containers by category. It's kind of a start but that isn't great - I still have large piles of parts.

[AGG2017]

Thanks for the commit that fix this issue. Sorry for the off-topic about the bar-codes. It was just a side-note about my excitement that by combining all that staff I found a way to automate and organize my component inventory after so many years of total mess. This information and possible application for inventory purposes can be also helpful to somebody else that can decide to use the access to DK data for more complex set of scripts or to import in already done one like Partkeepr. Thanks again for all your efforts and sharing your results with us!