search

tomquirk / linkedin-api

👨‍💼Linkedin API for Python
MIT License
1.71k stars 401 forks source link

`search_companies`: AttributeError: 'list' object has no attribute 'get' #344

Open bschilder opened 9 months ago

bschilder commented 9 months ago

Hi, thanks for the awesome package! Was hoping you could help me with this.

Description

search_companies unable to find info for certain keyword searches.

Reprex

The following company search yields an error:

from linkedin_api import Linkedin
from decouple import config 
api = Linkedin(config('LINKEDIN_USER'), config('LINKEDIN_PASS'))

companies = api.search_companies(keywords="Johnson & Johnson")
>>> companies = api.search_companies(keywords="Johnson & Johnson")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/bms20/Desktop/120-80/src/linkedin-api/linkedin_api/linkedin.py", line 460, in search_companies
    data = self.search(params, **kwargs)
  File "/Users/bms20/Desktop/120-80/src/linkedin-api/linkedin_api/linkedin.py", line 244, in search
    data_clusters = data.get("data", []).get("searchDashClustersByAll", [])
AttributeError: 'list' object has no attribute 'get'

But i can see this same search yields multiple results on LinkedIn: https://www.linkedin.com/search/results/companies/?keywords=Johnson%20%26%20Johnson&origin=SWITCH_SEARCH_VERTICAL&sid=%3AZ2

Versions

linkedin-api              2.0.1                     dev_0    <develop>
``` about-time 4.2.1 pyhd8ed1ab_0 conda-forge alive-progress 3.1.4 pyhd8ed1ab_0 conda-forge anyio 3.5.0 py39hca03da5_0 appnope 0.1.2 py39hca03da5_1001 argon2-cffi 21.3.0 pyhd3eb1b0_0 argon2-cffi-bindings 21.2.0 py39h1a28f6b_0 asttokens 2.0.5 pyhd3eb1b0_0 attrs 23.1.0 py39hca03da5_0 babel 2.11.0 py39hca03da5_0 backcall 0.2.0 pyhd3eb1b0_0 beautifulsoup4 4.12.2 py39hca03da5_0 biopython 1.78 py39h1a28f6b_0 blas 2.116 openblas conda-forge blas-devel 3.9.0 16_osxarm64_openblas conda-forge bleach 4.1.0 pyhd3eb1b0_0 bottleneck 1.3.5 py39heec5a64_0 brotli 1.0.9 h1a28f6b_7 brotli-bin 1.0.9 h1a28f6b_7 brotlipy 0.7.0 py39h1a28f6b_1002 bs4 4.12.2 py38hd3eb1b0_0 ca-certificates 2023.08.22 hca03da5_0 certifi 2023.7.22 py39hca03da5_0 cffi 1.15.1 py39h80987f9_3 charset-normalizer 2.0.4 pyhd3eb1b0_0 click 8.0.4 py39hca03da5_0 comm 0.1.2 py39hca03da5_0 contourpy 1.0.5 py39h525c30c_0 cryptography 41.0.3 py39hd4332d6_0 cycler 0.11.0 pyhd3eb1b0_0 debugpy 1.6.7 py39h313beb8_0 decorator 5.1.1 pyhd3eb1b0_0 defusedxml 0.7.1 pyhd3eb1b0_0 entrypoints 0.4 py39hca03da5_0 et_xmlfile 1.1.0 py39hca03da5_0 exceptiongroup 1.0.4 py39hca03da5_0 executing 0.8.3 pyhd3eb1b0_0 fonttools 4.25.0 pyhd3eb1b0_0 freetype 2.12.1 h1192e45_0 fuzzywuzzy 0.18.0 py39hca03da5_0 giflib 5.2.1 h80987f9_3 grapheme 0.6.0 pyhd8ed1ab_0 conda-forge habanero 1.2.3 pyh1a96a4e_0 conda-forge icu 73.1 h313beb8_0 idna 3.4 py39hca03da5_0 importlib-metadata 6.0.0 py39hca03da5_0 importlib_metadata 6.0.0 hd3eb1b0_0 importlib_resources 5.2.0 pyhd3eb1b0_1 ipykernel 6.25.0 py39h33ce5c2_0 ipython 8.15.0 py39hca03da5_0 ipython_genutils 0.2.0 pyhd3eb1b0_1 ipywidgets 8.0.4 py39hca03da5_0 iso3166 2.1.1 pyhd8ed1ab_0 conda-forge itables 1.5.4 pypi_0 pypi jedi 0.18.1 py39hca03da5_1 jinja2 2.11.3 pyhd3eb1b0_0 joblib 1.2.0 py39hca03da5_0 jpeg 9e h80987f9_1 json5 0.9.6 pyhd3eb1b0_0 jsonschema 4.17.3 py39hca03da5_0 jupyter_client 8.1.0 py39hca03da5_0 jupyter_contrib_core 0.4.0 pyhd8ed1ab_0 conda-forge jupyter_contrib_nbextensions 0.5.1 pyhd8ed1ab_2 conda-forge jupyter_core 5.3.0 py39hca03da5_0 jupyter_highlight_selected_word 0.2.0 pyhd8ed1ab_1006 conda-forge jupyter_latex_envs 1.4.6 pyhd8ed1ab_1002 conda-forge jupyter_nbextensions_configurator 0.6.1 pyhd8ed1ab_0 conda-forge jupyter_server 1.13.5 pyhd3eb1b0_0 jupyterlab 3.3.2 pyhd3eb1b0_0 jupyterlab-plotly-extension 1.0.0 py_0 conda-forge jupyterlab_server 2.10.3 pyhd3eb1b0_1 jupyterlab_widgets 3.0.5 py39hca03da5_0 kiwisolver 1.4.4 py39h313beb8_0 kneed 0.8.5 pyhd8ed1ab_0 conda-forge lcms2 2.12 hba8e193_0 lerc 3.0 hc377ac9_0 libblas 3.9.0 16_osxarm64_openblas conda-forge libbrotlicommon 1.0.9 h1a28f6b_7 libbrotlidec 1.0.9 h1a28f6b_7 libbrotlienc 1.0.9 h1a28f6b_7 libcblas 3.9.0 16_osxarm64_openblas conda-forge libcxx 14.0.6 h848a8c0_0 libdeflate 1.17 h80987f9_0 libffi 3.4.4 hca03da5_0 libgfortran 5.0.0 11_3_0_hca03da5_28 libgfortran5 11.3.0 h009349e_28 libiconv 1.16 h1a28f6b_2 liblapack 3.9.0 16_osxarm64_openblas conda-forge liblapacke 3.9.0 16_osxarm64_openblas conda-forge libllvm14 14.0.6 h7ec7a93_3 libopenblas 0.3.21 openmp_hc731615_3 conda-forge libpng 1.6.39 h80987f9_0 libsodium 1.0.18 h1a28f6b_0 libtiff 4.5.1 h313beb8_0 libuv 1.44.2 h80987f9_0 libwebp 1.3.2 ha3663a8_0 libwebp-base 1.3.2 h80987f9_0 libxml2 2.10.4 h0dcf63f_1 libxslt 1.1.37 h80987f9_1 linkedin-api 2.0.1 dev_0 llvm-openmp 14.0.6 hc6e5704_0 llvmlite 0.40.0 py39h514c7bf_0 lxml 4.9.3 py39h50ffb84_0 lz4-c 1.9.4 h313beb8_0 markupsafe 2.0.1 py39h1a28f6b_0 matplotlib 3.7.2 py39hca03da5_0 matplotlib-base 3.7.2 py39h46d7db6_0 matplotlib-inline 0.1.6 py39hca03da5_0 mistune 0.8.4 py39h1a28f6b_1000 munkres 1.1.4 py_0 nbclassic 0.5.5 py39hca03da5_0 nbconvert 5.6.1 pyhd8ed1ab_2 conda-forge nbformat 5.9.2 py39hca03da5_0 ncurses 6.4 h313beb8_0 nest-asyncio 1.5.6 py39hca03da5_0 nltk 3.8.1 py39hca03da5_0 nodejs 18.16.0 h62f6fdd_1 notebook 6.5.2 pyha770c72_0 conda-forge notebook-shim 0.2.2 py39hca03da5_0 numba 0.57.1 py39h46d7db6_0 numexpr 2.8.4 py39h79ee842_1 numpy 1.24.3 py39h1398885_0 numpy-base 1.24.3 py39h90707a3_0 openblas 0.3.21 openmp_hf78f355_3 conda-forge openpyxl 3.0.10 py39h1a28f6b_0 openssl 3.0.11 h1a28f6b_2 packaging 23.1 py39hca03da5_0 pandas 1.4.4 py39hc377ac9_0 pandocfilters 1.5.0 pyhd3eb1b0_0 parso 0.8.3 pyhd3eb1b0_0 pexpect 4.8.0 pyhd3eb1b0_3 pickleshare 0.7.5 pyhd3eb1b0_1003 pillow 9.4.0 py39h313beb8_1 pip 23.2.1 py39hca03da5_0 platformdirs 3.10.0 py39hca03da5_0 plotly 5.9.0 py39hca03da5_0 prometheus_client 0.14.1 py39hca03da5_0 prompt-toolkit 3.0.36 py39hca03da5_0 psutil 5.9.0 py39h1a28f6b_0 ptyprocess 0.7.0 pyhd3eb1b0_2 pure_eval 0.2.2 pyhd3eb1b0_0 pycparser 2.21 pyhd3eb1b0_0 pygments 2.15.1 py39hca03da5_1 pynndescent 0.5.10 py39hca03da5_0 pyopenssl 23.2.0 py39hca03da5_0 pyparsing 3.0.9 py39hca03da5_0 pyrsistent 0.18.0 py39h1a28f6b_0 pysocks 1.7.1 py39hca03da5_0 python 3.9.18 hb885b13_0 python-dateutil 2.8.2 pyhd3eb1b0_0 python-decouple 3.8 pyhd8ed1ab_0 conda-forge python-fastjsonschema 2.16.2 py39hca03da5_0 python-levenshtein 0.12.2 py39h1a28f6b_0 pytrends 4.9.2 pypi_0 pypi pytz 2023.3.post1 py39hca03da5_0 pyyaml 6.0 py39h80987f9_1 pyzmq 25.1.0 py39h313beb8_0 readline 8.2 h1a28f6b_0 regex 2022.7.9 py39h1a28f6b_0 requests 2.31.0 py39hca03da5_0 scikit-learn 1.3.0 py39h46d7db6_0 scipy 1.11.1 py39h20cbe94_0 seaborn 0.12.2 py39hca03da5_0 send2trash 1.8.0 pyhd3eb1b0_1 setuptools 68.0.0 py39hca03da5_0 six 1.16.0 pyhd3eb1b0_1 sniffio 1.2.0 py39hca03da5_1 soupsieve 2.4 py39hca03da5_0 sqlite 3.41.2 h80987f9_0 stack_data 0.2.0 pyhd3eb1b0_0 tbb 2021.8.0 h48ca7d4_0 tenacity 8.2.2 py39hca03da5_0 terminado 0.17.1 py39hca03da5_0 testpath 0.6.0 py39hca03da5_0 threadpoolctl 2.2.0 pyh0d69192_0 tk 8.6.12 hb8d0fd4_0 tornado 6.3.2 py39h80987f9_0 tqdm 4.65.0 py39h86d0a89_0 traitlets 5.7.1 py39hca03da5_0 typing-extensions 4.7.1 py39hca03da5_0 typing_extensions 4.7.1 py39hca03da5_0 tzdata 2023c h04d1e81_0 umap-learn 0.5.3 py39hca03da5_0 urllib3 1.26.16 py39hca03da5_0 wcwidth 0.2.5 pyhd3eb1b0_0 webencodings 0.5.1 py39hca03da5_1 websocket-client 0.58.0 py39hca03da5_4 wheel 0.38.4 py39hca03da5_0 widgetsnbextension 4.0.5 py39hca03da5_0 xlrd 2.0.1 pyhd3eb1b0_1 xz 5.4.2 h80987f9_0 yaml 0.2.5 h1a28f6b_0 zeromq 4.3.4 hc377ac9_0 zipp 3.11.0 py39hca03da5_0 zlib 1.2.13 h5a0b063_0 zstd 1.5.5 hd90d995_0 ```
bschilder commented 9 months ago

Ok, so I've tested this out and it seems it's related to the ampersand. This works much better. 

companies = api.search_companies(keywords="Johnson and Johnson", limit=10)
companies
[{'urn_id': '1207', 'name': 'Johnson & Johnson', 'headline': 'Hospitals and Health Care • New Brunswick, NJ', 'subline': '9M followers'}, {'urn_id': '1204', 'name': 'The Janssen Pharmaceutical Companies of Johnson & Johnson', 'headline': 'Pharmaceutical Manufacturing • Raritan, New Jersey', 'subline': '997K followers'}, {'urn_id': '1193', 'name': 'Johnson & Johnson Vision', 'headline': 'Medical Equipment Manufacturing • Jacksonville, FL', 'subline': '152K followers'}, {'urn_id': '11542591', 'name': 'Johnson & Johnson MedTech', 'headline': 'Hospitals and Health Care • New Brunswick, New Jersey', 'subline': '203K followers'}, {'urn_id': '657948', 'name': 'Johnson & Johnson Medical', 'headline': 'Hospitals and Health Care', 'subline': '41K followers'}, {'urn_id': '34760890', 'name': 'Johnson & Johnson MedTech UK & Ireland 🇬🇧 🇮🇪', 'headline': 'Hospitals and Health Care • Wokingham, England', 'subline': '107K followers'}, {'urn_id': '2247', 'name': 'Johnson Controls', 'headline': 'Industrial Machinery Manufacturing • Cork, Ireland', 'subline': '1M followers'}, {'urn_id': '66923029', 'name': 'Johnson & Johnson Consumer Health', 'headline': 'Retail Health and Personal Care Products • Skillman, New Jersey', 'subline': '63K followers'}, {'urn_id': '98725034', 'name': 'Johnson & Johnson Innovative Medicine', 'headline': 'Pharmaceutical Manufacturing', 'subline': '37K followers'}, {'urn_id': '4039', 'name': 'SC Johnson', 'headline': 'Manufacturing • Racine, WI', 'subline': '402K followers'}]

Is there some known set of special characters that cause search_companies to fail?

Thanks!, Brian

markwartV commented 9 months ago

Same issue since 2 weeks: https://github.com/tomquirk/linkedin-api/issues/343#issue-1898070753