Image upload fails (digikey to inventree)

[HannesMorre]

Image upload is not working for any part imported from Digikey to InvenTree.

[eeintech]

@HannesMorre Image availability from supplier websites is constantly changing, I believe they are trying to avoid web scrapping therefore implement more security features to prevent outsiders from easily getting their data, I will have to look into this again to see if it's possible to find a workaround.

[HannesMorre]

Okay, thanks for the information. This one is not a big isssue for me, just wanted to let you know.

[eeintech]

@HannesMorre Digi-Key doesn't have any issue on my side so I can't replicate... Is this problem with a specific part?

[HannesMorre]

No specific part, all parts I try to add to Inventree throws this error. When my colleague tries to do this, he gets the same error.

[eeintech]

Are you and your colleague able to download image from Digi-Key website and add them manually to InvenTree?

[HannesMorre]

Yes, we can

[eeintech]

Ok it was to make sure it is related to Ki-nTree and not your network 😄

In which shell are you running Ki-nTree: Command prompt, WSL, Powershell or Git Bash?

[HannesMorre]

I am running it in cmd and my colleague in Git Bash i think

[eeintech]

Ok I have tested Ki-nTree in Git bash and cannot reproduce... so pending someone's got a better insight into this issue than I do, I do not know what else can we troubleshoot.

[Fivefold]

I do see these errors too (with Digikey), but only the Image download failed (HTTPS Error) error, two times for each part creation:

[MAIN]  Generating Internal Part Number
[INFO]  Internal Part Number = TRA-000020-00
[INFO]  Success: Added new part to InvenTree
[INFO]  Warning: Image download failed (HTTP Error)
[INFO]  Warning: Image download failed (HTTP Error)

[MAIN]  Creating parameters
[INFO]  Success: The following parameters were created:

The actual image does get uploaded to InvenTree though, so I don't have any actual issues here. I don't think it's relevant but I'm using Windows Terminal

[eeintech]

@Fivefold There are 3 different tries so 2 warnings means 2 fails but the last attempt succeeded 😉

[lobra07]

i can confirm this problem with my parts . it does give an error for the digikey parts but vary rarely it works , out of 30 just 2 of them have no problem. there is some issues when the datasheet is from TI it does give an error like this part number is 296-39549-2-ND

[INFO]  Success: Added new part to InvenTree
[INFO]  Warning: Image download failed (HTTP Error)
[INFO]  Warning: Image download failed (HTTP Error)
[INFO]  Warning: Image download returned the wrong file type
[TREE]  Warning: Failed to upload part image
Exception in thread Thread-360 (create_part):
Traceback (most recent call last):
  File "E:\PROGRAMS_FILES\PYTHON\lib\threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "E:\PROGRAMS_FILES\PYTHON\lib\threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "d:\GITHUB_CODE_PROJECTS\Ki-nTree\kintree\gui\views\main.py", line 1298, in create_part
    new_part, part_pk, part_info = inventree_interface.inventree_create(
  File "d:\GITHUB_CODE_PROJECTS\Ki-nTree\kintree\database\inventree_interface.py", line 583, in inventree_create
    datasheet_link = inventree_api.upload_part_datasheet(inventree_part['datasheet'], part_pk)
  File "d:\GITHUB_CODE_PROJECTS\Ki-nTree\kintree\database\inventree_api.py", line 385, in upload_part_datasheet
    if not download_with_retry(datasheet_url,
  File "d:\GITHUB_CODE_PROJECTS\Ki-nTree\kintree\common\tools.py", line 124, in download_with_retry
    file = download(url, fileoutput=full_path, silent=silent, **kwargs)
  File "d:\GITHUB_CODE_PROJECTS\Ki-nTree\kintree\common\tools.py", line 93, in download
    (file, headers) = urllib.request.urlretrieve(url, filename=fileoutput)
  File "E:\PROGRAMS_FILES\PYTHON\lib\urllib\request.py", line 251, in urlretrieve
    tfp = open(filename, 'wb')
OSError: [Errno 22] Invalid argument: 'C:\\Users\\Musta\\kintree\\cache\\datasheets\\suppproductinfo.tsp?distId=10&gotoUrl=https%3A%2F%2Fwww.ti.com%2Flit%2Fgpn%2Fxtr115.pdf'

[eeintech]

Hello @lobra07

Unfortunately there is no much Ki-nTree can do to clean-up the URL passed by suppliers like Digi-Key, it does it to me too sometimes and the only workaround I found is to replace the URL prior of creating the part in InvenTree:

1. Search for part

2. Replace datasheet URL with one found on TI website

3. Part creation works fine

4. Attachment is present

[lobra07]

thanks for your response i have fixed for my self by doing this in Ki-nTree\kintree\database\inventree_interface.py line 263

inventree_part['supplier_link'] = part_info['supplier_link'].replace(' ', '%20')

inventree_part['datasheet'] = part_info['datasheet'].replace(' ', '%20')
if(('gotoUrl=' in part_info['datasheet']) ==  True):
        indexs= part_info['datasheet'].index('gotoUrl=')
        inventree_part['datasheet'] = part_info['datasheet'][indexs+8:]
        inventree_part['datasheet']= inventree_part['datasheet'].replace('%3A',':')
        inventree_part['datasheet']= inventree_part['datasheet'].replace('%2F','/')

cprint(inventree_part['datasheet'])

but for the main issue of not being able to use the Digikey image , it because of Python version i was using python 3.10.5 when updating to python 3.11.7 the problem disappeared

[eeintech]

@lobra07 Umm I am running Python 3.10 and didn't have issue with the image. But good to know it is fixed for you and thanks for the code snippet!

[eeintech]

@HannesMorre Is it still an issue for you? Have you tried Ki-nTree on a different network?

[HannesMorre]

we haven't tried on a different network yet, this is still an issue

[lobra07]

I'm not a web developer. But there is this on stackoverflow . If you are using a old version of SSL/TLS .. to solve it, updating to use URLlib3 to the latest version may solve it

https://stackoverflow.com/questions/77009915/code-to-retrieve-images-using-requests-doesnt-work-on-python-3-7-but-works-on-p

[HannesMorre]

So I need to update Python to 3.11?

[lobra07]

Doing that fixed the problem for me , yes

[eeintech]

@HannesMorre Any luck with upgrading python version?

[HannesMorre]

Python version is now at 3.12 Image uploads are still not working

[eeintech]

@HannesMorre Sorry to hear, not sure what else to check besides the recommendation above...

[eeintech]

@HannesMorre Please try:

• Python 3.11 (I haven't tested Python 3.12 and actually shouldn't work with Digi-Key as of now because of dependencies problems with this Python version)
• Running Ki-nTree on a different network (like home network)

Hope one of those attempts will lead us somewhere.

[HannesMorre]

I updated kintree to 1.1.0 with python 3.9.13 and now the image download works. I did run in some other issues though, but that is for another issue.

[eeintech]

Thanks for the update!