shughes-uk / EveCharacterSearch

Django project that scrapes the eve-online character bazaar and indexes character sales threads for easy searching and filtering
MIT License
6 stars 6 forks source link

Eve Character Search

preview

Eve Character Search is at the front a dynamic webpage that allows searching a live database of eve online character sales from the Character Bazaar Forums, and a scraping tool that monitors the forums and scrapes detailed information on the character for sale into a database for the webpage.

ECS leverages Django and BeautifulSoup.

Getting Started

OS X

Make sure Xcode Command Line Tools installed and up to date.

$ xcode-select --install

Use Homebrew to install pyenv and virtualwrapper.

$ brew install pyenv-virtualenvwrapper

Use pyenv to install a python to use with virtualwrapper.

$ pyenv install 2.7.12
$ pyenv global 2.7.12

Configure your shell session for use with virtualenvwrapper.

$ export PYENV_ROOT=/usr/local/var/pyenv
$ eval "$(pyenv init -)"
$ pyenv virtualenvwrapper

Create a new virtualenv for EveCharacterSearch.

mkvirtualenv -r requirements.txt eve
workon eve

Start a local instance of EveCharacterSearch.

cp ./evecharsearch/settings_example.py ./evecharsearch/settings.py
python manage.py makemigrations
python manage.py migrate
python manage.py update_api
python manage.py scrape_threads # use --pages to scrap more than 1 page
python manage.py runserver

MIT © shughes-uk et al