search

safe-refuge / safeway-data

Data mining tools for the Safeway app
4 stars 4 forks source link

Feature/point of interest structure upgrade #22

Closed moorchegue closed 2 years ago

moorchegue commented 2 years ago

Addresses #21.

This is really a work in progress, but I'd like to share it early anyway, since it might help catch some gotchas early on as well.

moorchegue commented 2 years ago

As I was fixing moldova_dopomoga crawler (probably source data changed), I noticed that there's also a map there (now): https://www.google.com/maps/d/embed?mid=1RiyRvxOE8q3Z-eo7e298nLn_I0nKXqq6&ehbc=2E312F&ll=47.152161219376964%2C28.378650750000023&z=8

It seems to have all the same info, and also coordinates for all the points. Would you like me to update the crawler to scrape that info?

littlepea commented 2 years ago

As I was fixing moldova_dopomoga crawler (probably source data changed), I noticed that there's also a map there (now): https://www.google.com/maps/d/embed?mid=1RiyRvxOE8q3Z-eo7e298nLn_I0nKXqq6&ehbc=2E312F&ll=47.152161219376964%2C28.378650750000023&z=8

It seems to have all the same info, and also coordinates for all the points. Would you like me to update the crawler to scrape that info?

It would be nice if it's easy to do 👍

moorchegue commented 2 years ago

Sure, no problem!

wdunn001 commented 2 years ago

Make sure to pull latest.

moorchegue commented 2 years ago

Hm… looks like poetry install fails on some package in the CI… Have anybody ran into this? I installed locally into a venv with pip install -e . , that worked.

Other than that I think I fixed all outstanding issues. Could you guys take a peek, please?

littlepea commented 2 years ago

Hm… looks like poetry install fails on some package in the CI… Have anybody ran into this? I installed locally into a venv with pip install -e . , that worked.

Other than that I think I fixed all outstanding issues. Could you guys take a peek, please?

On master branch CI is passing...

@wdunn001 any ideas?

wdunn001 commented 2 years ago

appears to be related to this https://github.com/python-poetry/poetry/issues/728.

Seems there is an outage in PyPi judging from their status page. Probably not related to poetry.

https://status.python.org/

https://stackoverflow.com/questions/72551057/poetry-gives-toomanyindirects-error-suddenly

wdunn001 commented 2 years ago

checked tests are working correctly now something in your latest PR is breaking tests.

littlepea commented 2 years ago

I'm gonna merge it so that we can have the new schema on master, we can address any remaining issues in a separate PR