peeter123 / digikey-api

Python module for the Digikey PartSearch API
GNU General Public License v3.0
85 stars 35 forks source link

API v2 is no longer available for new app registrations #3

Closed laf0rge closed 4 years ago

laf0rge commented 4 years ago

digikey has closed the path to regitster new clients/applications for API v2. Instead the have a "contract-breaking" v3 now :/ The change seems to have happened at some point in October 2019.

peeter123 commented 4 years ago

Thanks for your message, for now the V2 API keeps working for now with already registerd applications.

However, I'm planning to migrate to the V3 API in the future.

JBRO8081 commented 4 years ago

but with little modification in oauth2 i was able to work with new version. aslo changed to new url. but aplication side require much more porting.

self.auth_code=self.auth_code.replace('&scope','')

TomOudeNijhuis commented 4 years ago

I have the parts (renamed it to products to match the API call) working. Had to make changes to models, api, client and oauth2. Solved your issue above by changing code to:

split_result = urlsplit(self.path)
params = parse_qs(split_result.query)
self.auth_code = params['code'][0]

Unfortunately there is no compatibility with the v2 API. Would you like to merge my changes?

peeter123 commented 4 years ago

@TomOudeNijhuis Thanks, if you can make a pull request on digkey-api-v3 branch I can re-use your modifications!

auphofBSF commented 4 years ago

@peeter123 this python api to digikey is great, Thank You and small thanks to @TomOudeNijhuis for the little uoauth2 fixes, I have forked and created a working V3 dev branch forked from digikey-api-v3 https://github.com/auphofBSF/digikey-api/tree/dev20w02apiV3. The readme example (extended to show search) now works on V3 It still requires checking and model alignment to Digikey's changes. It also requires refactoring of the term part to product. Digikey have gone from using the term part to product. So it would probably be good to refactor the term in the model, client and api.
Before I go ahead and do this, it is worth considering keeping a class part as mirror of product for easy retrofit. Keen to hear from other users on the merit ? Pending this I would create a pull request

peeter123 commented 4 years ago

@auphofBSF Thanks for your work, I haven't gotten around to it. I think it is best to indeed keep the part class to avoid breaking applications using the pip module (however, if we switch to V3 they need to create an application at Digikey anyway). It might be nice to implement something like this (Stack Overflow).

auphofBSF commented 4 years ago

@auphofBSF Thanks for your work, I haven't gotten around to it. I think it is best to indeed keep the part class to avoid breaking applications using the pip module (however, if we switch to V3 they need to create an application at Digikey anyway). It might be nice to implement something like this (Stack Overflow).

That Idea looks promising, I will try it

auphofBSF commented 4 years ago

@peeter123 I have not been able to put the time into supporting V2 My main requirement has been to access the ordersupport api and to this extent I found the digikey api V3 swagger configuration files for each of the API's I also new of the swagger code generator that produced an operating python api, so instead of hand coding all the classes, attributes etc I automated the digikey api specification files into customized installed python api's client packages and provided a wrapper utilizing my forked V3 branch of your digikey_api to handle the OAuth2 protocol.

digikey spec files ---> swagger_codegen ---> python api classes https://github.com/auphofBSF/community-digikey-api-v3-lite/blob/DEV/docs/overview-community-digikey-api-v3.svg

I have just finished tidying this up into a one stop installation and have shared it publicly as https://github.com/auphofBSF/community-digikey-api-v3-lite , this automatically install the code generator module https://github.com/auphofBSF/community-digikey-api-codegen-python-clients It supports the full productinformation and ordersupport of V3 Digikey API.

Only real local prerequisite is having Java installed as the swagger codegen is a java app and the python package setup calls to first generate the python code.

Installation is simply by pip install git+https://github.com/auphofBSF/community-digikey-api-v3-lite@master I could look at making this available on PyPi

This now provides a robust V3 API on which a mapping to V2 could be done. But I don't know how much of requirement this so would welcome discussion on best way to proceed

Any feedback appreciated

lbrolliar commented 4 years ago

@auphofBSF I was able to get the community-digikey-api-v3-lite installed with the pip command above. I had to install the community-digikey-api-codegen-python-clients separately. (This may be due to me not realizing I didn't have Java already installed.)

I think I have a error in my setup at Digikey. The code is installed; however, when I run the code it prints out: 2020-03-10 11:03:04,363 - digikey.oauth2 - WARNING: Token storage does not exist or malformed, creating new.

and the Web browser displays: Oops Looks like something is not right. Please contact your administrator. 400 - Unknown or invalid client_id

I am using a Sandbox App at Digikey and I think it may be the Callback URL.

What do you put in the Callback URL on Digikey?

Thank you in advance for any assistance you might be able to provide.

auphofBSF commented 4 years ago

@lbrolliar good to hear you have got it working sofar, I have never used the digikey sandbox App but I imagine the call back would be similar. for the call back on the production app I have just used https://localhost I remember this was somewhat problematic and it is one of things you do not do repeatedly. If you can make any additional notes on the process it would be great to add it to the documentation

lbrolliar commented 4 years ago

@auphofBSF thank you for your response. Hopefully, if I can get past the authentication it will be smooth sailing and help pave the way for others.

I set up a Digikey production app and tested it and the result was the same as with my sandbox app. (e.g. A web page displaying: Oops Looks like something is not right. Please contact your administrator. 400 - Unknown or invalid client_id).

Thinking my client app might not be properly set up, I opened a Digikey support ticket. From my understanding of Digikey's response, the URL to perform the Digikey authentication for V3 API's needs to be of the form given below. (I could not quickly tell where in the code where the authentication URL is set to test this out with the Python code. )

Sandbox API URL: https://sandbox-api.digikey.com/v1/oauth2/authorize?response_type=code&client_id=123456789abcdefg&redirect_uri=https://client-app-redirect-uri/

Production API URL: https://api.digikey.com/v1/oauth2/authorize?response_type=code&client_id=123456789abcdefg&redirect_uri=https://client-app-redirect-uri/

Digikey’s support team directed me to: https://developer.digikey.com/documentation/oauth

Note: I was able to get a login/"Request for Approval" screen to come up in a web browser, when I typed the URL of the form above in the browser address bar with my production API Client ID. (e.g. A URL of the form - https://api.digikey.com/v1/oauth2/authorize?response_type=code&client_id=123456789abcdefg&redirect_uri=https://localhost)

The Digikey Version 2 API URL does not recognize a version 3 client_id. (e.g. https://sso.digikey.com/as/authorization.oauth2?response_type=code&client_id=123456789abcdefg&redirect_uri=https://localhost)

auphofBSF commented 4 years ago

@lbrolliar I am sorry you are experiencing this frustration. I have verified the steps and freshly connected a new instance to a production app. This fresh instance on an Ubuntu Bionic Virtual machine, with a new conda python virtual environment with python 3.6.9 only extra dependencies installed where pandas and tabulate as per the example/example-requirements.txt

1) I confirmed login to http://developer.digikey.com , go to organization and check app is there then view the app details image

2) with the example https://github.com/auphofBSF/community-digikey-api-v3-lite/blob/master/example/example.py Ensure the DIGIKEY_CLIENT_ID and DIGIKEY_CLIENT_SECRET are set to same as listed in Production APP, also ensure the DIGIKEY_STORAGE_PATH folder exists, it does not get automatically created image

3) in a terminal go to the directory where the example.py is saved and run python example.py On my machine Chrome is the default browser and you should be presented with a logon screen image

4) after successful login , the request for approval should display image

5) after approving you should be alerted to a NET::ERR_CERT_AUTHORITY_INVALID, go into BUTTON (ADVANCED) and agree to proceed to localhost image

6) At this point you should get an ERROR: image

7) Go back to your terminal where you issued python example.py Ctrl ^C to break the execution that should look like this. image

8) rerun python example.py no digikey login should appear but the Request for Approval should appear again as per 4) , accepting this should give you retrieved Auth code image

9) The example.py program should have now run through looking something like this image

You will now see the token_storage.json has been created in your nominated DIGIKEY_STORAGE_PATH as described in 2). I am not overly familiar with OAUTH2 and for convenience used the the work that was done in trying to get @peeter123 digikey-api to work with the new DIGIKEY api. Its my understanding once you have this token_storage.json you can then copy it and use in some other location and the token gets automatically refreshed periodically, all this is done with the forked and modified digikey-api automatically installed as a dependency from this branch's last commit https://github.com/auphofBSF/digikey-api/tree/dev20w02apiV3

There needs to be some work done to improve the initial OAUTH2 function of creating the token_storage.json, I think the primary issue is getting around the insecure local host not having a certificate

I did look into it and it was a body of knowledge and integration I could not take on then and probably not now, It would be great to have a contribution to this as it is a fixed and defined function.

However these steps only are required to setup the initial token and I have just lived with that as there was no apparent issue through on going use. There may be some issue if an earlier backup of the token_storage.json is used.

See how you get on with this detail and hopefully we can Identify at what step you`re failing.

lbrolliar commented 4 years ago

Success!!!

@auphofBSF, Thanks for the pics! They helped greatly in debugging. I noticed the url in your browser was going to auth.digikey.com; however, mine was still going to sso.digikey.com. Thus, I figured something must be setup incorrectly on my machine.

I did a fresh install into a new Python environment everything worked as you detailed above. (Previously, I had installed the community-digikey-api-v3-lite in the same environment where I installed the code for the V2 Digikey API, digikey-api 0.2.1.) The clean install seems to be the way to go at least for me.

Below details the process I went through get code installed and working on a Windows platform.

Platform: Windows 10, Anaconda Navigator 1.9.12, Spyder IDE 4.0.1. Steps:

  1. Create new Python 3.6 environment inside of Anaconda Navigator.
  2. Install pandas and tabulate using Anaconda Navigator.
  3. Open the new environment in Anaconda Nav. Install Anaconda Power Shell and Spyder.
  4. In Anaconda Power Shell install community-digikey-api-v3-lite pip install git+https://github.com/auphofBSF/community-digikey-api-v3-lite@master
  5. Create a directory on your drive for storing the token’s (e.g. C:\Users\username\digikey-cache-dir)
  6. Launch Spyder.
  7. Modify the three lines in the example script.
  8. Proceed with Step 1 under @auphofBSF’s post above. Note: I did not have to do the Ctrl-C in the step above. After I clicked through to the “Error response” page the program exited on its own. Below is the example display. digikey9a

Thanks again!

auphofBSF commented 4 years ago

great news, You're more than welcome,

lbrolliar commented 4 years ago

Quick question. Is there code for doing a keyword search to find the Digikey part number?

peeter123 commented 4 years ago

Hi All,

I have started work on digikey-api-v3 based on the swagger autogenerate work done by @auphofBSF (thanks!).

For now only the Product Information API is included and is 'wrapped' so you do not need to think about access tokens and such.

It should be relatively stable, please try it and if you have any comments or improvements please let me know or submit a pull-request.

eeintech commented 4 years ago

@peeter123 I just gave it a shot and, also following @auphofBSF detailed instructions above, and was able to get the V3 to work :)

I know it may seem obvious but in the digikey-api-v3 branch README, you may want to add the proper way to clone the repo (and also mention that pip won't work right now): git clone --branch digikey-api-v3 git@github.com:peeter123/digikey-api.git

Otherwise, the sample code works, I added prints and this is what I'm getting for the first request:

2020-06-29 13:03:32,811 - digikey.v3.api -   INFO: Get product details for: 296-6501-1-ND
2020-06-29 13:03:32,811 - digikey.v3.api -  DEBUG: CALL wrapped -> PartSearchApi.product_details_with_http_info
2020-06-29 13:03:34,120 - digikey.v3.api -  DEBUG: Requests remaining: [995/1000]
{'additional_value_fee': 0.0,
 'alternate_packaging': ...
 'associated_products': [],
 'date_last_buy_chance': None,
 'detailed_description': '555 Type, Timer/Oscillator (Single) IC 100kHz 8-SOIC',
 'digi_key_part_number': '296-6501-1-ND',
 'dk_plus_restriction': False,
 'for_use_with_products': [],
 'kit_contents': [],
 'kits': [],
 'lead_status': 'Lead Free',
 'limited_taxonomy': ...
 'manufacturer': {'parameter': 'Manufacturer',
                  'parameter_id': -1,
                  'value': 'Texas Instruments',
                  'value_id': '296'},
 'manufacturer_lead_weeks': '6 weeks',
 'manufacturer_page_url': 'https://www.digikey.com/en/supplier-centers/texas-instruments',
 'manufacturer_part_number': 'NE555DR',
 'manufacturer_public_quantity': 0,
 'mating_products': [],
 'media_links': ...
 'minimum_order_quantity': 1,
 'my_pricing': [],
 'non_stock': False,
 'obsolete': False,
 'packaging': ...
 'parameters': ...
 'primary_datasheet': 'http://www.ti.com/general/docs/suppproductinfo.tsp?distId=10&gotoUrl=http%3A%2F%2Fwww.ti.com%2Flit%2Fgpn%2Fne555',
 'primary_photo': 'https://media.digikey.com/Renders/Texas%20Instr%20Renders/296;4040047-3;D;8.jpg',
 'primary_video': '',
 'product_description': 'IC OSC SGL TIMER 100KHZ 8-SOIC',
 'product_status': 'Active',
 'product_url': 'https://www.digikey.com/product-detail/en/texas-instruments/NE555DR/296-6501-1-ND/372490',
 'quantity_available': 56144,
 'quantity_on_order': 0,
 'ro_hs_status': 'ROHS3 Compliant',
 'rohs_subs': [],
 'search_locale_used': ...
 'series': ...
 'standard_package': 1,
 'standard_pricing': ...
 'suggested_subs': [],
 'supplier_direct_ship': False,
 'tariff_description': '',
 'unit_price': 0.36}
kjgoodrick commented 4 years ago

Hi Peter,

I've also been doing a little work based off @auphofBSF's work, though not the swagger version. Primarily this involves adding support for filtering and sorting. You can see that in my fork here: https://github.com/kjgoodrick/digikey-api/tree/dev20w02apiV3

I looked at trying to add this in off the swagger version, but it wasn't clear to me how that would work with the autogenerated code. I'm not sure if this made obsolete by your new version, but I'll play around with it and see.

peeter123 commented 4 years ago

@kjgoodrick I think that should be possible in the autogenerated version, you can pass a KeywordSearchRequest model as a kwargs to the keyword_search function. The KeywordSearchRequest model also has an entry for adding Filters and SortParameters. You can check the available models in digikey/v3/productinformation/models

RainZhangZZFLS commented 4 years ago

Hi Peter, Thanks for your contribution to this digiky-api. I have tried digikey-api-v3 and it seems git clone directly does not work, so I just simply download the ZIP and pip install it. Now the problem is the client ID and client Secret provided by the SandBox still does not work. image

I still received the error message like this 2020-07-03 16:07:49,055 - digikey.oauth.oauth2 - WARNING: Token storage does not exist or malformed, creating new. Thus I am commenting to see if any possible solution, I notice there exists some success cases.

peeter123 commented 4 years ago

@RainZhangZZFLS You created a sandbox application, you need to create a production application for it to work.

You can however now use sandbox API as well (see updated branch)

RainZhangZZFLS commented 4 years ago

@RainZhangZZFLS You created a sandbox application, you need to create a production application for it to work.

You can however now use sandbox API as well (see updated branch)

That finally works! Thanks Peter! Thanks Anthony for swagger part!

RainZhangZZFLS commented 4 years ago

Hi Peter, I have another question on how to extract certain information from the result = digikey.keyword_search(body=search_request) For example, like 'manufacturer_part_number': 'CRCW080510K0FKEA', which can be displayed using 'print'. But how to display it separately or output as a string?

eeintech commented 4 years ago

@RainZhangZZFLS To save and access data in a dictionary, you'll need to append the .to_dict() to the request:

result = digikey.keyword_search(body=search_request).to_dict()

Then you can use result['manufacturer_part_number'] to retrieve the part number string.

RainZhangZZFLS commented 4 years ago

@RainZhangZZFLS To save and access data in a dictionary, you'll need to append the .to_dict() to the request:

result = digikey.keyword_search(body=search_request).to_dict()

Then you can use result['manufacturer_part_number'] to retrieve the part number string.

Thanks for your reply! That really helps!

eeintech commented 4 years ago

@peeter123 Are you planning on merging the digikey-api-v3 branch to master?

peeter123 commented 4 years ago

@eeintech Yes I think that should be possible on short term.

NOTE: only the partsearch API has been wrapped / implemented

peeter123 commented 4 years ago

I have released 0.4.0 with support for the PartSearch v3 API,

If you are interested in helping implementing the other APIs please create a pull request👍