39 add weighted average to satellite data extraction to reduce nulls

[ChristinaLast]

Description

This branch adds the weighted average to satellite query functionality in branch #17 Fixes #39

Type of change

Please delete options that are not relevant.

• [X] Bug fix (non-breaking change which fixes an issue)
• [X] New feature (non-breaking change which adds functionality)
• [X] Breaking change (fix or feature that would cause existing functionality to not work as expected)
• [X] This change requires a documentation update

How has this been tested?

Please describe the tests that you ran to verify your changes.

openaq-engine feature-builder

Checklist:

• [X] My code follows the style guidelines of this project
• [X] I have performed a self-review of my own code
• [X] I have commented my code, particularly in hard-to-understand areas
• [X] I have made corresponding changes to the documentation
• [ X] My changes generate no new warnings
• [ ] I have added tests that prove my fix is effective or that my feature works
• [X] New and existing unit tests pass locally with my changes

[jit-1]

(openaq-engine-py3.8) [prithviraj@ip-172-31-81-98 openaq-engine]$ openaq-engine feature-builder Traceback (most recent call last): File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/sqlalchemy/engine/base.py", line 3361, in _wrap_pool_connect return fn() File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/sqlalchemy/pool/base.py", line 327, in connect return _ConnectionFairy._checkout(self) File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/sqlalchemy/pool/base.py", line 894, in _checkout fairy = _ConnectionRecord.checkout(pool) File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/sqlalchemy/pool/base.py", line 493, in checkout rec = pool._do_get() File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/sqlalchemy/pool/impl.py", line 146, in _do_get self._decoverflow() File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 70, in exit compat.raise( File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/sqlalchemy/util/compat.py", line 211, in raise_ raise exception File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/sqlalchemy/pool/impl.py", line 143, in _do_get return self._create_connection() File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/sqlalchemy/pool/base.py", line 273, in _create_connection return ConnectionRecord(self) File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/sqlalchemy/pool/base.py", line 388, in init self.connect() File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/sqlalchemy/pool/base.py", line 691, in connect pool.logger.debug("Error on connect(): %s", e) File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 70, in exit compat.raise( File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/sqlalchemy/util/compat.py", line 211, in raise_ raise exception File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/sqlalchemy/pool/base.py", line 686, in __connect self.dbapi_connection = connection = pool._invoke_creator(self) File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/sqlalchemy/engine/create.py", line 578, in connect return dialect.connect(*cargs, *cparams) File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/sqlalchemy/engine/default.py", line 598, in connect return self.dbapi.connect(cargs, cparams) File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/psycopg2/init.py", line 122, in connect conn = _connect(dsn, connection_factory=connection_factory, kwasync) psycopg2.OperationalError: could not translate host name "None" to address: Name or service not known

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/prithviraj/unicef/openaq-engine/.venv/bin/openaq-engine", line 33, in sys.exit(load_entry_point('openaq-engine', 'console_scripts', 'openaq-engine')()) File "/home/prithviraj/unicef/openaq-engine/.venv/lib/python3.8/site-packages/click/core.py", line 1130, in call return self.main(args, kwargs) File "/home/prithviraj/unicef/openaq-engine/.venv/lib/python3.8/site-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) File "/home/prithviraj/unicef/openaq-engine/.venv/lib/python3.8/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/prithviraj/unicef/openaq-engine/.venv/lib/python3.8/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, ctx.params) File "/home/prithviraj/unicef/openaq-engine/.venv/lib/python3.8/site-packages/click/core.py", line 760, in invoke return __callback(args, kwargs) File "/home/prithviraj/unicef/openaq-engine/openaq_engine/main.py", line 75, in feature_builder build_features.execute(engine) File "/home/prithviraj/unicef/openaq-engine/openaq_engine/src/features/build_features.py", line 41, in execute df = get_data(cohort_query) File "/home/prithviraj/unicef/openaq-engine/openaq_engine/src/utils/utils.py", line 177, in get_data with connect_to_db() as conn: File "/usr/lib64/python3.8/contextlib.py", line 113, in enter return next(self.gen) File "/home/prithviraj/unicef/openaq-engine/openaq_engine/setup_environment.py", line 98, in connect_to_db conn = engine.connect() File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/sqlalchemy/engine/base.py", line 3315, in connect return self._connection_cls(self, close_with_result=close_with_result) File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/sqlalchemy/engine/base.py", line 96, in init else engine.raw_connection() File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/sqlalchemy/engine/base.py", line 3394, in raw_connection return self._wrap_pool_connect(self.pool.connect, _connection) File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/sqlalchemy/engine/base.py", line 3364, in _wrap_pool_connect Connection._handle_dbapi_exception_noconnection( File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/sqlalchemy/engine/base.py", line 2198, in _handle_dbapi_exceptionnoconnection util.raise( File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/sqlalchemy/util/compat.py", line 211, in raise_ raise exception File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/sqlalchemy/engine/base.py", line 3361, in _wrap_pool_connect return fn() File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/sqlalchemy/pool/base.py", line 327, in connect return _ConnectionFairy._checkout(self) File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/sqlalchemy/pool/base.py", line 894, in _checkout fairy = _ConnectionRecord.checkout(pool) File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/sqlalchemy/pool/base.py", line 493, in checkout rec = pool._do_get() File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/sqlalchemy/pool/impl.py", line 146, in _do_get self._decoverflow() File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 70, in exit compat.raise( File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/sqlalchemy/util/compat.py", line 211, in raise_ raise exception File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/sqlalchemy/pool/impl.py", line 143, in _do_get return self._create_connection() File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/sqlalchemy/pool/base.py", line 273, in _create_connection return ConnectionRecord(self) File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/sqlalchemy/pool/base.py", line 388, in init self.connect() File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/sqlalchemy/pool/base.py", line 691, in connect pool.logger.debug("Error on connect(): %s", e) File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 70, in exit compat.raise( File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/sqlalchemy/util/compat.py", line 211, in raise_ raise exception File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/sqlalchemy/pool/base.py", line 686, in __connect self.dbapi_connection = connection = pool._invoke_creator(self) File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/sqlalchemy/engine/create.py", line 578, in connect return dialect.connect(*cargs, *cparams) File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/sqlalchemy/engine/default.py", line 598, in connect return self.dbapi.connect(cargs, cparams) File "/home/prithviraj/unicef/openaq-engine/.venv/lib64/python3.8/site-packages/psycopg2/init.py", line 122, in connect conn = _connect(dsn, connection_factory=connection_factory, **kwasync) sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) could not translate host name "None" to address: Name or service not known

(Background on this error at: https://sqlalche.me/e/14/e3q8)