search

niryariv / opentaba-server

BSD 3-Clause "New" or "Revised" License
4 stars 13 forks source link

Problem with local testing and logger #62

Closed alonisser closed 10 years ago

alonisser commented 10 years ago

@shevron maybe you can help (since it's connected to your code):

when I run the test locally, when preparing the setup steps I get this error, when I run python worker.py:

No handlers could be found for logger "tools.scrapelib"
[2014-01-06 21:45] ERROR: horse: SystemExit: 1
Traceback (most recent call last):
  File "/home/alon/.virtualenvs/opentaba-server/local/lib/python2.7/site-packages/rq/worker.py", line 393, in perform_job
    rv = job.perform()
  File "/home/alon/.virtualenvs/opentaba-server/local/lib/python2.7/site-packages/rq/job.py", line 328, in perform
    self._result = self.func(*self.args, **self.kwargs)
  File "/home/alon/Projects/opentaba-server/tools/scrapelib.py", line 132, in scrape_gush
    html = get_gush_html(gush_id)
  File "/home/alon/Projects/opentaba-server/tools/scrapelib.py", line 32, in get_gush_html
    exit(1)
  File "/home/alon/.virtualenvs/opentaba-server/lib/python2.7/site.py", line 403, in __call__
    raise SystemExit(code)
SystemExit: 1

since no scraping is done, some of the local tests also fail. in travis-ci everything works fine , So i'm not sure what the issue is - any Ideas? directions?

shevron commented 10 years ago

The first line is a warning from logging, but the actual error may be unrelated to logging. Can you please check if you have the logging module installed or not?

On Mon, Jan 6, 2014 at 11:49 PM, Alonisser notifications@github.com wrote:

@shevron https://github.com/shevron maybe you can help (since it's connected to your code):

when I run the test locally, when preparing the setup steps I get this error, when I run python worker.py:

No handlers could be found for logger "tools.scrapelib" [2014-01-06 21:45] ERROR: horse: SystemExit: 1 Traceback (most recent call last): File "/home/alon/.virtualenvs/opentaba-server/local/lib/python2.7/site-packages/rq/worker.py", line 393, in perform_job rv = job.perform() File "/home/alon/.virtualenvs/opentaba-server/local/lib/python2.7/site-packages/rq/job.py", line 328, in perform self._result = self.func(_self.args, *_self.kwargs) File "/home/alon/Projects/opentaba-server/tools/scrapelib.py", line 132, in scrape_gush html = get_gush_html(gush_id) File "/home/alon/Projects/opentaba-server/tools/scrapelib.py", line 32, in get_gush_html exit(1) File "/home/alon/.virtualenvs/opentaba-server/lib/python2.7/site.py", line 403, in call raise SystemExit(code) SystemExit: 1

since no scraping is done, some of the local tests also fail. in travis-ci everything works fine , So i'm not sure what the issue is - any Ideas? directions?

— Reply to this email directly or view it on GitHubhttps://github.com/niryariv/opentaba-server/issues/62 .

alonisser commented 10 years ago

yes, I have logging module, it's a python standard library module.. (I checked to make sure It's installed) what happens when you run the tests? everything works correctly?

shevron commented 10 years ago

Yes, all tests pass for me.

I've seen some issues with logging in the past - there is a Python standard library module and an old implementation ( https://pypi.python.org/pypi/logging) which are close but not 100% compatible. It is possible that if you have the pypi module installed, remove it or if you don't, install it and things will change, I'm really not sure :\

Just for the sake of debugging, can you try to replace https://github.com/niryariv/opentaba-server/blob/master/tools/scrapelib.py#L31with a simple print and see if the problem is gone?

On Wed, Jan 8, 2014 at 10:18 PM, Alonisser notifications@github.com wrote:

yes, I have logging module, it's a python standard library module.. (I checked to make sure It's installed) what happens when you run the tests? everything works correctly?

— Reply to this email directly or view it on GitHubhttps://github.com/niryariv/opentaba-server/issues/62#issuecomment-31873734 .

alonisser commented 10 years ago

I changed it with a print and I get : ('ERROR: %s', Exception('Unexpected status code: 503',))

so this is an error with scraping this from mmi @florpor can you shed some light on this

@shevron : about the logging problem also I found in our requirements.txt file this lib: loogbook probably installed by some dependency (or by us?) . I don't think It supposed to override logging, but who knows. can you check this out?

florpor commented 10 years ago

@alonisser what exact address are you getting 503 service unavailable for? Anyways the scrapelib will soon change along with the addresses so it's very possible this 503 is just because of the old mmi site dying.

alonisser commented 10 years ago

The gush in the tests.. On Jan 10, 2014 1:42 PM, "florpor" notifications@github.com wrote:

@alonisser https://github.com/alonisser what exact address are you getting 503 service unavailable for? Anyways the scrapelib will soon change along with the addresses so it's very possible this 503 is just because of the old mmi site dying.

— Reply to this email directly or view it on GitHubhttps://github.com/niryariv/opentaba-server/issues/62#issuecomment-32021074 .

florpor commented 10 years ago

so http://mmi.gov.il/IturTabot/taba2.asp?Gush=30649&fromTaba1=true ? the page works now, do you still get 503?

alonisser commented 10 years ago

@florpor still getting this. did you try to follow the sequence on your local dev machine? am I missing something? I'm trying (as in the test) inside the virtualenv:

python create_db.py --force
python scrape.py -g 30649
python worker.py # where I can the error message, about the 503 code
niryariv commented 10 years ago

@alonisser does it work for you now?

alonisser commented 10 years ago

Not sure what the question is.. didn't touch the server side for a long time (focused on the user subscription) Can you clarify?

Twitter:@alonisser https://twitter.com/alonisser LinkedIn Profile http://www.linkedin.com/in/alonisser Facebook https://www.facebook.com/alonisser _Tech blog:_4p-tech.co.il/blog _Personal Blog:_degeladom.wordpress.com Tel:972-54-6734469

On Sun, Jun 15, 2014 at 2:41 PM, Nir Yariv notifications@github.com wrote:

@alonisser https://github.com/alonisser does it work for you now?

— Reply to this email directly or view it on GitHub https://github.com/niryariv/opentaba-server/issues/62#issuecomment-46113590 .

niryariv commented 10 years ago

IIRC the ticket is about an issue you were getting when running the server on your dev machine - is it still relevant?

alonisser commented 10 years ago

Not sure.. I'll try tommorow

Twitter:@alonisser https://twitter.com/alonisser LinkedIn Profile http://www.linkedin.com/in/alonisser Facebook https://www.facebook.com/alonisser _Tech blog:_4p-tech.co.il/blog _Personal Blog:_degeladom.wordpress.com Tel:972-54-6734469

On Sun, Jun 15, 2014 at 3:47 PM, Nir Yariv notifications@github.com wrote:

IIRC the ticket is about an issue you were getting when running the server on your dev machine - is it still relevant?

— Reply to this email directly or view it on GitHub https://github.com/niryariv/opentaba-server/issues/62#issuecomment-46114861 .

niryariv commented 10 years ago

seems ok now - closing