search

Mat-O-Lab / MapToMethod

Tool to map content of JSON-LD files (output of CSVtoCSVW) describing CSV files to Information Content Entities in knowledge graphs describing methods in the method folder of the MSEO Ontology repository at https://github.com/Mat-O-Lab/MSEO.
1 stars 0 forks source link

Problems booting up the provided image #8

Closed rmeissn closed 1 year ago

rmeissn commented 1 year ago

When booting up the provided image, the service seems to request something from github (not sure what this is), but runs into a rate limit (no idea why this applies to my server). Would it be possible to fix that by skipping downloading things from github and including them in the image at built time?

I can't use the image because of this rate limit/bug.

[2023-01-31 14:18:24 +0000] [1] [INFO] Starting gunicorn 20.1.0
[2023-01-31 14:18:24 +0000] [1] [INFO] Listening at: http://0.0.0.0:8080 (1)
[2023-01-31 14:18:24 +0000] [1] [INFO] Using worker: sync
[2023-01-31 14:18:24 +0000] [3] [INFO] Booting worker with pid: 3
[2023-01-31 14:18:24 +0000] [4] [INFO] Booting worker with pid: 4
[2023-01-31 14:18:24 +0000] [5] [INFO] Booting worker with pid: 5
[2023-01-31 14:18:27 +0000] [3] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/gunicorn/arbiter.py", line 589, in spawn_worker
    worker.init_process()
  File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/base.py", line 134, in init_process
    self.load_wsgi()
  File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/base.py", line 146, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/local/lib/python3.8/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/local/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 58, in load
    return self.load_wsgiapp()
  File "/usr/local/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 48, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/local/lib/python3.8/site-packages/gunicorn/util.py", line 359, in import_app
    mod = importlib.import_module(module)
  File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/src/wsgi.py", line 1, in <module>
    from app import app
  File "/src/app.py", line 23, in <module>
    app.methods_dict = maptomethod.get_methods()
  File "/src/maptomethod.py", line 71, in get_methods
    mseo_repo = github.Github().get_repo("Mat-O-Lab/MSEO")
  File "/usr/local/lib/python3.8/site-packages/github/MainClass.py", line 330, in get_repo
    headers, data = self.__requester.requestJsonAndCheck("GET", url)
  File "/usr/local/lib/python3.8/site-packages/github/Requester.py", line 353, in requestJsonAndCheck
    return self.__check(
  File "/usr/local/lib/python3.8/site-packages/github/Requester.py", line 378, in __check
    raise self.__createException(status, responseHeaders, output)
github.GithubException.RateLimitExceededException: 403 {"message": "API rate limit exceeded for 139.18.211.135. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)", "documentation_url": "https://docs.github.com/rest/overview/resources-in-the-rest-api#rate-limiting"}
[2023-01-31 14:18:27 +0000] [3] [INFO] Worker exiting (pid: 3)
{'development': <class 'config.DevelopmentConfig'>, 'production': <class 'config.ProductionConfig'>, 'default': <class 'config.DevelopmentConfig'>}
[2023-01-31 14:18:27 +0000] [5] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/gunicorn/arbiter.py", line 589, in spawn_worker
    worker.init_process()
  File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/base.py", line 134, in init_process
    self.load_wsgi()
  File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/base.py", line 146, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/local/lib/python3.8/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/local/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 58, in load
    return self.load_wsgiapp()
  File "/usr/local/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 48, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/local/lib/python3.8/site-packages/gunicorn/util.py", line 359, in import_app
    mod = importlib.import_module(module)
  File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/src/wsgi.py", line 1, in <module>
    from app import app
  File "/src/app.py", line 23, in <module>
    app.methods_dict = maptomethod.get_methods()
  File "/src/maptomethod.py", line 71, in get_methods
    mseo_repo = github.Github().get_repo("Mat-O-Lab/MSEO")
  File "/usr/local/lib/python3.8/site-packages/github/MainClass.py", line 330, in get_repo
    headers, data = self.__requester.requestJsonAndCheck("GET", url)
  File "/usr/local/lib/python3.8/site-packages/github/Requester.py", line 353, in requestJsonAndCheck
    return self.__check(
  File "/usr/local/lib/python3.8/site-packages/github/Requester.py", line 378, in __check
    raise self.__createException(status, responseHeaders, output)
github.GithubException.RateLimitExceededException: 403 {"message": "API rate limit exceeded for 139.18.211.135. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)", "documentation_url": "https://docs.github.com/rest/overview/resources-in-the-rest-api#rate-limiting"}
[2023-01-31 14:18:27 +0000] [5] [INFO] Worker exiting (pid: 5)
{'development': <class 'config.DevelopmentConfig'>, 'production': <class 'config.ProductionConfig'>, 'default': <class 'config.DevelopmentConfig'>}
[2023-01-31 14:18:27 +0000] [4] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/gunicorn/arbiter.py", line 589, in spawn_worker
    worker.init_process()
  File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/base.py", line 134, in init_process
    self.load_wsgi()
  File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/base.py", line 146, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/local/lib/python3.8/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/local/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 58, in load
    return self.load_wsgiapp()
  File "/usr/local/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 48, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/local/lib/python3.8/site-packages/gunicorn/util.py", line 359, in import_app
    mod = importlib.import_module(module)
  File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/src/wsgi.py", line 1, in <module>
    from app import app
  File "/src/app.py", line 23, in <module>
    app.methods_dict = maptomethod.get_methods()
  File "/src/maptomethod.py", line 71, in get_methods
    mseo_repo = github.Github().get_repo("Mat-O-Lab/MSEO")
  File "/usr/local/lib/python3.8/site-packages/github/MainClass.py", line 330, in get_repo
    headers, data = self.__requester.requestJsonAndCheck("GET", url)
  File "/usr/local/lib/python3.8/site-packages/github/Requester.py", line 353, in requestJsonAndCheck
    return self.__check(
  File "/usr/local/lib/python3.8/site-packages/github/Requester.py", line 378, in __check
    raise self.__createException(status, responseHeaders, output)
github.GithubException.RateLimitExceededException: 403 {"message": "API rate limit exceeded for 139.18.211.135. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)", "documentation_url": "https://docs.github.com/rest/overview/resources-in-the-rest-api#rate-limiting"}
[2023-01-31 14:18:27 +0000] [4] [INFO] Worker exiting (pid: 4)
{'development': <class 'config.DevelopmentConfig'>, 'production': <class 'config.ProductionConfig'>, 'default': <class 'config.DevelopmentConfig'>}
[2023-01-31 14:18:27 +0000] [1] [WARNING] Worker with pid 4 was terminated due to signal 15
[2023-01-31 14:18:27 +0000] [1] [WARNING] Worker with pid 3 was terminated due to signal 15
[2023-01-31 14:18:28 +0000] [1] [INFO] Shutting down: Master
[2023-01-31 14:18:28 +0000] [1] [INFO] Reason: Worker failed to boot.
ThHanke commented 1 year ago

fixed in v1.0.3 needed ontologies are now downloaded when building images! So github API for download is only used when building container.