search

opendatacube / datacube-core

Open Data Cube analyses continental scale Earth Observation data through time
http://www.opendatacube.org
Apache License 2.0
505 stars 176 forks source link

product update and add cli does not show failed message for odc user with no write permission #1290

Closed pindge closed 2 years ago

pindge commented 2 years ago

Expected behaviour

when updating and adding, if the odc database user doesn't have write permission display appropriate message as output

Actual behaviour

displays message to say updated and adding, but no follow-up message to report back status nor error message

image

image

odc=> \du                                                                 List of roles
     Role name      |                         Attributes                         |                          Member of                          
--------------------+------------------------------------------------------------+-------------------------------------------------------------
 agdc_admin         | Create role, Cannot login                                  | {agdc_manage}
 agdc_ingest        | Cannot login                                               | {agdc_user}
...
 sandbox_reader     |                                                            | {}
odc=> select * from information_schema.role_table_grants where grantee='sandbox_reader' ;
  grantor  |    grantee     | table_catalog | table_schema |           table_name            | privilege_type | is_grantable | with_hierarchy 
-----------+----------------+---------------+--------------+---------------------------------+----------------+--------------+----------------
 odc_admin | sandbox_reader | odc           | agdc         | dataset                         | SELECT         | NO           | YES
 odc_admin | sandbox_reader | odc           | agdc         | dataset_type                    | SELECT         | NO           | YES
 odc_admin | sandbox_reader | odc           | agdc         | metadata_type                   | SELECT         | NO           | YES
 odc_admin | sandbox_reader | odc           | agdc         | dataset_source                  | SELECT         | NO           | YES
 odc_admin | sandbox_reader | odc           | agdc         | dataset_location                | SELECT         | NO           | YES
 odc_admin | sandbox_reader | odc           | agdc         | dv_mangrove_cover_test_dataset  | SELECT         | NO           | YES
 odc_admin | sandbox_reader | odc           | agdc         | dv_s2b_nrt_granule_test_dataset | SELECT         | NO           | YES
(7 rows)

Steps to reproduce the behaviour

... Include code, command line parameters as appropriate ...

Environment information

Note: Stale issues will be automatically closed after a period of six months with no activity. To ensure critical issues are not closed, tag them with the Github pinned tag. If you are a community member and not a maintainer please escalate this issue to maintainers via GIS StackExchange or Slack.

pindge commented 2 years ago

This above behaviour is due to product already exist in the database and no attempt writing to the dataset_type table occurred, 

(odc) ubuntu@:~/datacube-core$ export DATACUBE_DB_URL=postgresql://odc_reader:123@localhost/odc
(odc) ubuntu@:~/datacube-core$ datacube product add https://raw.githubusercontent.com/GeoscienceAustralia/dea-config/master/products/others/cemp_insar/cemp_insar_alos_velocity.yaml
Adding "cemp_insar_alos_velocity" (this might take a while) DONE
(odc) ubuntu@:~/datacube-core$ datacube product update https://raw.githubusercontent.com/GeoscienceAustralia/dea-config/master/products/others/cemp_insar/cemp_insar_alos_velocity.yaml
Updated "cemp_insar_alos_velocity"

When the product isn't in the database, a sql permission error is displayed on the output.

(odc) ubuntu@:~/datacube-core$ datacube system check
Version:       1.8.6
Config files:  
Host:          localhost:5432
Database:      odc
User:          odc_reader
Environment:   None
Index Driver:  default

Valid connection:       YES
(odc) ubuntu@:~/datacube-core$ datacube product add https://raw.githubusercontent.com/GeoscienceAustralia/dea-config/master/products/others/cemp_insar/cemp_insar_alos_velocity.yaml
Adding "cemp_insar_alos_velocity" (this might take a while)Traceback (most recent call last):
  File "/home/ubuntu/yes/envs/odc/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context
    self.dialect.do_execute(
  File "/home/ubuntu/yes/envs/odc/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute
    cursor.execute(statement, parameters)
psycopg2.errors.InsufficientPrivilege: permission denied for table dataset_type

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ubuntu/yes/envs/odc/bin/datacube", line 8, in <module>
    sys.exit(cli())
  File "/home/ubuntu/yes/envs/odc/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/ubuntu/yes/envs/odc/lib/python3.8/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/ubuntu/yes/envs/odc/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/ubuntu/yes/envs/odc/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/ubuntu/yes/envs/odc/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/ubuntu/yes/envs/odc/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/ubuntu/yes/envs/odc/lib/python3.8/site-packages/datacube/ui/click.py", line 200, in new_func
    return f(parsed_config, *args, **kwargs)
  File "/home/ubuntu/yes/envs/odc/lib/python3.8/site-packages/datacube/ui/click.py", line 232, in with_index
    return f(index, *args, **kwargs)
  File "/home/ubuntu/yes/envs/odc/lib/python3.8/site-packages/datacube/scripts/product.py", line 57, in add_products
    index.products.add(type_, allow_table_lock=allow_exclusive_lock)
  File "/home/ubuntu/yes/envs/odc/lib/python3.8/site-packages/datacube/index/_products.py", line 105, in add
    connection.insert_product(
  File "/home/ubuntu/yes/envs/odc/lib/python3.8/site-packages/datacube/drivers/postgres/_api.py", line 785, in insert_product
    res = self._connection.execute(
  File "/home/ubuntu/yes/envs/odc/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1306, in execute
    return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS)
  File "/home/ubuntu/yes/envs/odc/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection
    return connection._execute_clauseelement(
  File "/home/ubuntu/yes/envs/odc/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement
    ret = self._execute_context(
  File "/home/ubuntu/yes/envs/odc/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context
    self._handle_dbapi_exception(
  File "/home/ubuntu/yes/envs/odc/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2043, in _handle_dbapi_exception
    util.raise_(
  File "/home/ubuntu/yes/envs/odc/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 208, in raise_
    raise exception
  File "/home/ubuntu/yes/envs/odc/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context
    self.dialect.do_execute(
  File "/home/ubuntu/yes/envs/odc/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.InsufficientPrivilege) permission denied for table dataset_type

[SQL: INSERT INTO agdc.dataset_type (name, metadata, metadata_type_ref, definition) VALUES (%(name)s, %(metadata)s, %(metadata_type_ref)s, %(definition)s) RETURNING agdc.dataset_type.id]
[parameters: {'name': 'cemp_insar_alos_velocity', 'metadata': '{"product": {"name": "cemp_insar_alos_velocity"}}', 'metadata_type_ref': 1, 'definition': '{"name": "cemp_insar_alos_velocity", "description": "CEMP InSAR ALOS Velocity", "metadata_type": "eo3", "metadata": {"product": {"name": "cemp_insar_ ... (302 characters truncated) ...  "nodata": -9999, "units": "mm/year"}, {"name": "upstd", "aliases": ["updownstandarddev"], "dtype": "float32", "nodata": -9999, "units": "mm/year"}]}'}]
(Background on this error at: https://sqlalche.me/e/14/f405)