wagtail / wagtail-localize-git

Translate Wagtail content using a git repository and PO files
Other
6 stars 3 forks source link

suppress Beautiful Soup warnings for URLs during `sync_git` #20

Open Pomax opened 3 years ago

Pomax commented 3 years ago

Any translation field with values that look like URLs currently cause the sync_git command to print warnings to the console like:

INFO - Pull: Importing changes in file 'locales/pt-BR/pages/homepage/blog/day-unite-against-hate-speech-and-misinformation.po'
/app/.heroku/python/lib/python3.7/site-packages/bs4/__init__.py:424: MarkupResemblesLocatorWarning: "http://share.mozilla.org/352/209304/facebook" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.
  MarkupResemblesLocatorWarning
/app/.heroku/python/lib/python3.7/site-packages/bs4/__init__.py:424: MarkupResemblesLocatorWarning: "http://share.mozilla.org/352/209305/twitter" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.
  MarkupResemblesLocatorWarning
INFO - Pull: Importing changes in file 'locales/pt-BR/pages/homepage/blog/join-mozilla-telling-major-companies-stophateforprofit.po'
INFO - Pull: Importing changes in file 'locales/pt-BR/pages/homepage/blog/mozilla-explains-ethical-ai-the-choice-is-not-yours-sometimes.po'
/app/.heroku/python/lib/python3.7/site-packages/bs4/__init__.py:424: MarkupResemblesLocatorWarning: "https://www.youtube.com/embed/xoue4-ohk1Y?rel=0&cc_load_policy=1&cc_lang_pref=pt" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.
  MarkupResemblesLocatorWarning
INFO - Pull: Importing changes in file 'locales/pt-BR/pages/homepage/blog/mozilla-explains-how-dating-apps-might-be-keeping-you-single.po'
/app/.heroku/python/lib/python3.7/site-packages/bs4/__init__.py:424: MarkupResemblesLocatorWarning: "https://www.youtube.com/embed/YTlOrapai-M?rel=0&cc_load_policy=1&cc_lang_pref=pt" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.
  MarkupResemblesLocatorWarning
INFO - Pull: Importing changes in file 'locales/pt-BR/pages/homepage/blog/mozilla-explains-what-is-ai.po'
/app/.heroku/python/lib/python3.7/site-packages/bs4/__init__.py:424: MarkupResemblesLocatorWarning: "https://www.youtube.com/embed/P-iiN0c2uic?rel=0&cc_load_policy=1&cc_lang_pref=pt" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.
  MarkupResemblesLocatorWarning
INFO - Pull: Importing changes in file 'locales/pt-BR/pages/homepage/blog/mozilla-explains-why-did-i-watch-that.po'
/app/.heroku/python/lib/python3.7/site-packages/bs4/__init__.py:424: MarkupResemblesLocatorWarning: "https://www.youtube.com/embed/pt9YCVX7VOk?rel=0&cc_load_policy=1&cc_lang_pref=pt" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.
  MarkupResemblesLocatorWarning
/app/.heroku/python/lib/python3.7/site-packages/bs4/__init__.py:424: MarkupResemblesLocatorWarning: "https://www.youtube.com/embed/S0pE87hADUU?rel=0&cc_load_policy=1&cc_lang_pref=pt" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.
  MarkupResemblesLocatorWarning
INFO - Pull: Importing changes in file 'locales/pt-BR/pages/homepage/blog/mozilla-explains-why-does-youtube-recommend-conspiracy-theory-videos.po'
/app/.heroku/python/lib/python3.7/site-packages/bs4/__init__.py:424: MarkupResemblesLocatorWarning: "https://www.youtube.com/embed/r98HbsaN9nw?rel=0&cc_load_policy=1&cc_lang_pref=pt" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.
  MarkupResemblesLocatorWarning
INFO - Pull: Importing changes in file 'locales/pt-BR/pages/homepage/blog/new-iphone-update-how-to-opt-out-of-cross-app-tracking.po'
/app/.heroku/python/lib/python3.7/site-packages/bs4/__init__.py:424: MarkupResemblesLocatorWarning: "http://share.mozilla.org/352/217446/twitter" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.
  MarkupResemblesLocatorWarning
/app/.heroku/python/lib/python3.7/site-packages/bs4/__init__.py:424: MarkupResemblesLocatorWarning: "http://share.mozilla.org/352/217447/facebook" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.
  MarkupResemblesLocatorWarning
/app/.heroku/python/lib/python3.7/site-packages/bs4/__init__.py:424: MarkupResemblesLocatorWarning: "http://share.mozilla.org/352/217448/email" looks like a URL. Beautiful Soup is not an HTTP client. You should probably use an HTTP client like requests to get the document behind the URL, and feed that document to Beautiful Soup.
  MarkupResemblesLocatorWarning

And while true, these warnings are also essentially meaningless, as sync_git is not concerned with validation, it just needs to sync data, whatever that data is. Could the command be updated so that even if it still uses BS4, with loglevel=error rather than what it's using atm?

zerolab commented 1 year ago
from bs4 import MarkupResemblesLocatorWarning

warnings.filterwarnings("ignore", category=MarkupResemblesLocatorWarning, module=r"^wagtail_localize_git\..+")