search

PaulMcInnis / JobFunnel

Scrape job websites into a single spreadsheet with no duplicates.
MIT License
1.88k stars 218 forks source link

Unable to run funnel --help #91

Closed manohar52 closed 4 years ago

manohar52 commented 4 years ago

Issue Template

After installation, when I try to run funnel --help, i get error saying:

funnel : The term 'funnel' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again. At line:1 char:1

thebigG commented 4 years ago

Hi there,

Could you give us more information about your OS/hardware? I suspect this is Windows?

If it is Windows, please let us know exactly how you installed JobFunnel? Are you using PowerShell? CommandLine prompt?Windows Terminal? Also did you install The new JobFunnel 3.0? That has not been released yet and nor has it been thoroughly/peer-reviewed tested, so you should not be using that version yet. I will be reviewing later it today, but I suspect it will be some time before we can release it because Paul probably wants as many people to review it as possible.

Like I said, please give us more details about the environment you have JobFunnel running under.

manohar52 commented 4 years ago

Hi.. I ran this command: pip install git+https://github.com/PaulMcInnis/JobFunnel.git

OS: Windows I ran it on the powershell terminal of VS code. Also I have a folder generated after I run the above command: JobFunnel-2.1.9-py3.8.egg-info So I am guessing the version is 2.1.9..

Sorry I am kind of a noob. Just trying to experiment on Linkedin Thanks in advance

thebigG commented 4 years ago

I recommend not installing JobFunnel through VS code's powershell. Try to install it by using the stand-alone PowerShell terminal. I don't use Windows, but I highly suspect VS Code is isolating JobFunnel on its own virtual environment. Again, not a Windows user, just a hypothesis.

Also, regarding LinkedIn, are you trying to write a scraper for LinkedIn? That would be really awesome to have, but I don't recommend it. Last I checked, they really do not like people to scrape their site.

I would check in with @PaulMcInnis(the maintainer of the repo) about writing a scraper for LinkedIn. Because for all I know, I could be wrong about the whole LinkedIn situation.

I don't have a Windows Environment ready at the moment, but I will try to reproduce your issue if I get time. No promises though. I highly suspect this might be because of VS Code.

It might also help if you give us your Python and pip version.

Hope this helps!

manohar52 commented 4 years ago

Thank you.. will try that.. Regarding Linkedin,.. i am just trying to learn about web scraping... if Linkedin doesn't like it, probably will try it with other websites.. My python version: 3.8 Pip: 20.1 Again, I appreciate all ur help!

thebigG commented 4 years ago

Not a problem! That's really awesome you're implementing a new web scraper for JobFunnel. I know I said 3.0 isn't ready yet, but if it is for development, then I do suggest using 3.0 :smile:. For development, NOT regular use. The reason I really think you should be using 3.0 is because we have a a much better architecture for the code base that makes it easier for web scraper authors like you to deal with stuff like internationalization and dynamic/static scraping. 3.0 is currently being worked on(and ALMOST ready to be shipped). 3.0 is currently on the ABCJobFunnel branch.

Since you are trying to develop on JobFunnel, do NOT install it with pip install git+https://github.com/PaulMcInnis/JobFunnel.git. That's for regular use. To make your life easier as a developer, install it like this:

git clone git@github.com:PaulMcInnis/JobFunnel.git jobfunnel
pip install -e ./jobfunnel
funnel --help

Like I said, 3.0 is not ready yet. It is currently under review on #90. So be on the lookout for any changes/decisions that are made for this new release if you will be developing for JobFunnel.

Happy coding!

PaulMcInnis commented 4 years ago

Hey there @manohar52, I'll chime in to echo @thebigG and recommend that you check out the branch ABCJobFunnel on this repo, though there are still some bugs with CLI handling currently, it is quite functional. Once this branch is merged into master it should be simple enough to rebase.

To write a scraper, you should start by taking a look at the indeed scraper and the base scraper, these should be enough to get you started.

There is currently an issue #52 where we are keeping a list of websites we want to scrape jobs from, google's job site looks like a really good one.

PaulMcInnis commented 4 years ago

As for the bug you are reporting, it seems that there may be an issue with VSCode and Powershell, which may be worth fixing if we can determine what's causing it. For further discussion on writing a scraper feel free to create a new issue with the feature label.