When running as nohup python3 -u -m scripts > log.txt & on the server:
After the popularity script finished running and it moved on to release frequency, this exception happened
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python3.5/dist-packages/django/db/backends/mysql/base.py", line 71, in execute
return self.cursor.execute(query, args)
File "/home/sarah/.local/lib/python3.5/site-packages/MySQLdb/cursors.py", line 206, in execute
res = self._query(query)
File "/home/sarah/.local/lib/python3.5/site-packages/MySQLdb/cursors.py", line 312, in _query
db.query(q)
File "/home/sarah/.local/lib/python3.5/site-packages/MySQLdb/connections.py", line 224, in query
_mysql.connection.query(self, query)
MySQLdb._exceptions.OperationalError: (2006, 'MySQL server has gone away')
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/lib/python3.5/runpy.py", line 184, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/local/NVMe2/home/sarah/Projects/LibraryMetricScripts/scripts/__main__.py", line 4, in <module>
app.run()
File "/local/NVMe2/home/sarah/Projects/LibraryMetricScripts/scripts/app.py", line 26, in run
get_release_freq()
File "/local/NVMe2/home/sarah/Projects/LibraryMetricScripts/scripts/ReleaseFrequency/releasefrequency.py", line 47, in get_release_freq
getReleaseDates(config_dict["TOKEN"])
File "/local/NVMe2/home/sarah/Projects/LibraryMetricScripts/scripts/ReleaseFrequency/releasefrequency.py", line 28, in getReleaseDates
for library in libraries:
File "/usr/local/lib/python3.5/dist-packages/django/db/models/query.py", line 274, in __iter__
self._fetch_all()
File "/usr/local/lib/python3.5/dist-packages/django/db/models/query.py", line 1242, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/usr/local/lib/python3.5/dist-packages/django/db/models/query.py", line 55, in __iter__
results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
File "/usr/local/lib/python3.5/dist-packages/django/db/models/sql/compiler.py", line 1140, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python3.5/dist-packages/django/db/backends/utils.py", line 67, in execute
return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
File "/usr/local/lib/python3.5/dist-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/usr/local/lib/python3.5/dist-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python3.5/dist-packages/django/db/utils.py", line 89, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/usr/local/lib/python3.5/dist-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python3.5/dist-packages/django/db/backends/mysql/base.py", line 71, in execute
return self.cursor.execute(query, args)
File "/home/sarah/.local/lib/python3.5/site-packages/MySQLdb/cursors.py", line 206, in execute
res = self._query(query)
File "/home/sarah/.local/lib/python3.5/site-packages/MySQLdb/cursors.py", line 312, in _query
db.query(q)
File "/home/sarah/.local/lib/python3.5/site-packages/MySQLdb/connections.py", line 224, in query
_mysql.connection.query(self, query)
django.db.utils.OperationalError: (2006, 'MySQL server has gone away')
This could be related to this problem https://stackoverflow.com/questions/14163429/python-and-django-operationalerror-2006-mysql-server-has-gone-away but the max_allowed_packet was already large enough and it was failing on a small query (~50 libraries). I simply commented out the popularity so it doesn't run again and it starts right away with frequency and that worked out so I'm not sure if there was some operational error that caused MySQL to go down or if this is going to be a recurrent issue. It shouldn't be so because the popularity scripts didn't really access the database a lot while running, unless the problem is that each loop for a library takes so long because of the sleeping for github API that it keeps the sql connection open for a long time and that causes it to timeout.
When running as
nohup python3 -u -m scripts > log.txt &on the server:After the popularity script finished running and it moved on to release frequency, this exception happened
This could be related to this problem https://stackoverflow.com/questions/14163429/python-and-django-operationalerror-2006-mysql-server-has-gone-away but the max_allowed_packet was already large enough and it was failing on a small query (~50 libraries). I simply commented out the popularity so it doesn't run again and it starts right away with frequency and that worked out so I'm not sure if there was some operational error that caused MySQL to go down or if this is going to be a recurrent issue. It shouldn't be so because the popularity scripts didn't really access the database a lot while running, unless the problem is that each loop for a library takes so long because of the sleeping for github API that it keeps the sql connection open for a long time and that causes it to timeout.