digitalmethodsinitiative / zeeschuimer

A browser extension to collect social media data with.
Other
185 stars 14 forks source link

Linkedin 4Cat Connection not working #33

Closed supersambo closed 6 months ago

supersambo commented 6 months ago

I'am trying to upload a dataset I collected via Zeeschuimer to a 4CAT instance. The upload is working for Instagram and Tiktok (comments) data. However when I try to upload a Linkedin dataset, it appears to be processing and uploading but does not result in anything. It does not appear under Uploaded Datasets in Zeeschuimer, neither does it show up in 4CAT.

Any ideas?

dale-wahl commented 6 months ago

Can you check your 4CAT logs? You can view the last part of the log in the Control Panel via View Logs. If nothing is there, you can try uploading the dataset again (to trigger an error that would be recent enough to appear there) or look at the full logs in the logs folder (either in the Docker container or locally depending on how you installed 4CAT).

supersambo commented 6 months ago

thanks for the quick reply! Here is what seems to be the relevant part of the logs:

INFO:4cat-backend:Running processor linkedin-search on dataset c13e80aa486f4cf25e27845d8684264c
INFO:4cat-backend:Querying: {'datasource': 'linkedin', 'file': '/usr/src/app/data/linkedin-dataset-c13e80aa486f4cf25e27845d8684264c.importing'}
ERROR:4cat-backend:Processor linkedin-search raised NameError while processing dataset c13e80aa486f4cf25e27845d8684264c in search.py:83->search.py:375->search.py:210->processor.py:754->search_linkedin.py:121:
   name 'post' is not defined
dale-wahl commented 6 months ago

Ah, looks like a bug in 4CAT. We’ll have to fix it there and release a new version. We’ll post here when that is done. Thanks for the report.

stijn-uva commented 6 months ago

This was fixed in the most recent 4CAT release!

the-hugo commented 4 months ago

For me it appears that the transfer to 4CAT is quite slow. (1kB / sec).

image

I am running it on docker. Anyone having the same issue?