search

irods / irods_capability_storage_tiering

BSD 3-Clause "New" or "Revised" License
5 stars 10 forks source link

Storage Tiering does not handle collections/data-objects with only group permissions #273

Open korydraughn opened 1 month ago

korydraughn commented 1 month ago

Let's assume an iRODS zone is configured to only allow group permissions on data objects and collections (i.e. no direct user permissions on things).

Please investigate whether the storage tiering plugin handles group permissions?

_Originally reported at https://groups.google.com/d/msgid/irod-chat/fe929223-1d8e-44e5-b669-3da9bd95ee99n%40googlegroups.com?utm_medium=email&utm_source=footer_

alanking commented 1 month ago

Possibly related to https://github.com/irods/irods/issues/7816

alanking commented 1 month ago

Reading the Google Group conversation again, https://github.com/irods/irods/issues/7816 may be unrelated. Looks like there could be an issue if multiple users or groups are included, where "multiple" is the operative word. Then again, I might be misunderstanding again, so I'll just stop typing now.

jadgraaf commented 1 month ago

Issue seems not the be limitted to groups, even adding multiple users causes the same issue on tiering. With more then 1 user defined on data in irods these kinds of errors are thrown.

After removing users and only have 1 user (rods) the tiering resumes.

Error sample:

{"log_category":"legacy","log_level":"error","log_message":"data movement scheduling failed - [-818000]::[iRODS Exception:\n file: /irods_plugin_source/storage_tiering.cpp\n function: void irods::storage_tiering::set_migration_metadata_flag_for_object(rcComm_t , const std::string &, const std::string &, const std::string &)\n line: 972\n code: -818000 (CAT_NO_ACCESS_PERMISSION)\n message:\n failed to set migration scheduled flag for [/nki/home/bioimaging/2024/test2/test.tif]\nstack trace:\n--------------\n 0# irods::stacktrace::dump() const in /lib/libirods_common.so.4.3.1\n 1# irods::exception::assemble_full_display_what() const in /lib/libirods_common.so.4.3.1\n 2# irods::exception::what() const in /lib/libirods_common.so.4.3.1\n 3# irods::query_processor::execute(irods::thread_pool&, RcComm&)::'lambda'()::operator()() in /usr/lib/irods/plugins/rule_engines/libirods_rule_engine_plugin-unified_storage_tiering.so\n 4# boost::asio::detail::executor_op<boost::asio::detail::binder0<irods::query_processor::execute(irods::thread_pool&, RcComm&)::'lambda'()>, std::__1::allocator, boost::asio::detail::scheduler_operation>::do_complete(void, boost::asio::detail::scheduler_operation, boost::system::error_code const&, unsigned long) in /usr/lib/irods/plugins/rule_engines/libirods_rule_engine_plugin-unified_storage_tiering.so\n 5# boost::asio::detail::scheduler::do_run_one(boost::asio::detail::conditionally_enabled_mutex::scoped_lock&, boost::asio::detail::scheduler_thread_info&, boost::system::error_code const&) in /lib/libirods_server.so.4.3.1\n 6# boost::asio::detail::scheduler::run(boost::system::error_code&) in /lib/libirods_server.so.4.3.1\n 7# boost::asio::detail::posix_thread::func::run() in /lib/libirods_server.so.4.3.1\n 8# boost_asio_detail_posix_thread_function in /lib/libirods_server.so.4.3.1\n 9# 0x00007F6BD7930609 in /lib/x86_64-linux-gnu/libpthread.so.0\n10# clone in /lib/x86_64-linux-gnu/libc.so.6\n\n]","request_api_name":"EXEC_RULE_EXPRESSION_AN","request_api_number":1206,"request_api_version":"d","request_client_user":"rods","request_host":"172.31.32.83","request_proxy_user":"rods","request_release_version":"rods4.3.1","server_host":"p-irods-001","server_pid":2630598,"server_timestamp":"2024-07-25T13:29:39.186Z","server_type":"agent","server_zone":"nki"} {"log_category":"legacy","log_level":"error","log_message":"data movement scheduling failed - [-818000]::[iRODS Exception:\n file: /irods_plugin_source/storage_tiering.cpp\n function: void irods::storage_tiering::set_migration_metadata_flag_for_object(rcComm_t , const std::string &, const std::string &, const std::string &)\n line: 972\n code: -818000 (CAT_NO_ACCESS_PERMISSION)\n message:\n failed to set migration scheduled flag for [/nki/home/bioimaging/2024/test2/Thumbs.db]\nstack trace:\n--------------\n 0# irods::stacktrace::dump() const in /lib/libirods_common.so.4.3.1\n 1# irods::exception::assemble_full_display_what() const in /lib/libirods_common.so.4.3.1\n 2# irods::exception::what() const in /lib/libirods_common.so.4.3.1\n 3# irods::query_processor::execute(irods::thread_pool&, RcComm&)::'lambda'()::operator()() in /usr/lib/irods/plugins/rule_engines/libirods_rule_engine_plugin-unified_storage_tiering.so\n 4# boost::asio::detail::executor_op<boost::asio::detail::binder0<irods::query_processor::execute(irods::thread_pool&, RcComm&)::'lambda'()>, std::1::allocator, boost::asio::detail::scheduler_operation>::do_complete(void, boost::asio::detail::scheduler_operation, boost::system::error_code const&, unsigned long) in /usr/lib/irods/plugins/rule_engines/libirods_rule_engine_plugin-unified_storage_tiering.so\n 5# boost::asio::detail::scheduler::do_run_one(boost::asio::detail::conditionally_enabled_mutex::scoped_lock&, boost::asio::detail::scheduler_thread_info&, boost::system::error_code const&) in /lib/libirods_server.so.4.3.1\n 6# boost::asio::detail::scheduler::run(boost::system::error_code&) in /lib/libirods_server.so.4.3.1\n 7# boost::asio::detail::posix_thread::func::run() in /lib/libirods_server.so.4.3.1\n 8# boost_asio_detail_posix_thread_function in /lib/libirods_server.so.4.3.1\n 9# 0x00007F6BD7930609 in /lib/x86_64-linux-gnu/libpthread.so.0\n10# clone in /lib/x86_64-linux-gnu/libc.so.6\n\n]","request_api_name":"EXEC_RULE_EXPRESSION_AN","request_api_number":1206,"request_api_version":"d","request_client_user":"rods","request_host":"172.31.32.83","request_proxy_user":"rods","request_release_version":"rods4.3.1","server_host":"p-irods-001","server_pid":2630598,"server_timestamp":"2024-07-25T13:29:39.186Z","server_type":"agent","server_zone":"nki"} {"log_category":"legacy","log_level":"error","log_message":"iRODS Exception:\n file: /irods_plugin_source/storage_tiering.cpp\n function: void irods::storage_tiering::migrate_violating_data_objects(rcComm_t , const std::string &, const std::string &, const std::string &, const std::string &)\n line: 674\n code: -35000 (SYS_INVALID_OPR_TYPE)\n message:\n scheduling failed for [13] objects for query [SELECT DATA_NAME, COLL_NAME, USER_NAME, USER_ZONE, DATA_REPL_NUM WHERE META_DATA_ATTR_NAME = 'irods::access_time' AND META_DATA_ATTR_VALUE < '1721914171' AND META_DATA_ATTR_UNITS <> 'irods::storage_tiering::migration_scheduled' AND DATA_RESC_ID IN ('35069',)]\nstack trace:\n--------------\n 0# irods::stacktrace::dump() const in /lib/libirods_common.so.4.3.1\n 1# irods::exception::assemble_full_display_what() const in /lib/libirods_common.so.4.3.1\n 2# irods::exception::what() const in /lib/libirods_common.so.4.3.1\n 3# irods::log(irods::exception const&) in /lib/libirods_common.so.4.3.1\n 4# irods::storage_tiering::migrate_violating_data_objects(RcComm, std::__1::basic_string<char, std::1::char_traits, std::1::allocator > const&, std::1::basic_string<char, std::1::char_traits, std::1::allocator > const&, std::1::basic_string<char, std::1::char_traits, std::1::allocator > const&, std::1::basic_string<char, std::1::char_traits, std::__1::allocator > const&) in /usr/lib/irods/plugins/rule_engines/libirods_rule_engine_plugin-unified_storage_tiering.so\n 5# irods::storage_tiering::apply_policy_for_tier_group(std::1::basic_string<char, std::1::char_traits, std::1::allocator > const&) in /usr/lib/irods/plugins/rule_engines/libirods_rule_engine_plugin-unified_storage_tiering.so\n 6# exec_rule_expression(std::1::tuple<>&, std::__1::basic_string<char, std::1::char_traits, std::1::allocator > const&, MsParamArray*, irods::callback) in /usr/lib/irods/plugins/rule_engines/libirods_rule_engine_plugin-unified_storage_tiering.so\n 7# std::1::function::func<irods::error ()(std::1::tuple<>&, std::1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&, MsParamArray, irods::callback), std::1::allocator<irods::error (*)(std::1::tuple<>&, std::1::basic_string<char, std::__1::char_traits, std::1::allocator > const&, MsParamArray, irods::callback)>, irods::error (std::1::tuple<>&, std::1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&, MsParamArray, irods::callback)>::operator()(std::1::tuple<>&, std::1::basic_string<char, std::1::char_traits, std::1::allocator > const&, MsParamArray*&&, irods::callback&&) in /usr/lib/irods/plugins/rule_engines/libirods_rule_engine_plugin-unified_storage_tiering.so\n 8# irods::pluggable_rule_engine<std::1::tuple<> >::exec_rule_expression(std::1::tuple<>&, std::1::basic_string<char, std::1::char_traits, std::1::allocator > const&, MsParamArray*, irods::callback) in /lib/libirods_server.so.4.3.1\n 9# irods::rule_engine_context_manager<std::1::tuple<>, RuleExecInfo, (irods::rule_execution_manager_pack)0>::exec_rule_expression(std::1::basic_string<char, std::__1::char_traits, std::1::allocator > const&, std::1::basic_string<char, std::__1::char_traits, std::1::allocator > const&, MsParamArray) in /lib/libirods_server.so.4.3.1\n10# rsExecRuleExpression(RsComm, ExecRuleExpression) in /lib/libirods_server.so.4.3.1\n11# irods::api_call_adaptor<ExecRuleExpression>::operator()(irods::plugin_context&, RsComm, ExecRuleExpression) in /lib/libirods_server.so.4.3.1\n12# std::1::function::__func<irods::api_call_adaptor<ExecRuleExpression>, std::1::allocator<irods::api_call_adaptor<ExecRuleExpression> >, irods::error (irods::plugin_context&, RsComm, ExecRuleExpression)>::operator()(irods::plugin_context&, RsComm&&, ExecRuleExpression&&) in /lib/libirods_server.so.4.3.1\n13# int irods::api_entry::call_handler<ExecRuleExpression>(RsComm, ExecRuleExpression) in /lib/libirods_server.so.4.3.1\n14# rsApiHandler(RsComm, int, BytesBuf, BytesBuf) in /lib/libirods_server.so.4.3.1\n15# readAndProcClientMsg(RsComm, int) in /lib/libirods_server.so.4.3.1\n16# agentMain(RsComm*) in /lib/libirods_server.so.4.3.1\n17# runIrodsAgentFactory(sockaddr_un) in /lib/libirods_server.so.4.3.1\n18# main::$_5::operator()() const at rodsServer.cpp:?\n19# main in /usr/sbin/irodsServer\n20# libc_start_main in /lib/x86_64-linux-gnu/libc.so.6\n21# _start in /usr/sbin/irodsServer\n\n","request_api_name":"EXEC_RULE_EXPRESSION_AN","request_api_number":1206,"request_api_version":"d","request_client_user":"rods","request_host":"172.31.32.83","request_proxy_user":"rods","request_release_version":"rods4.3.1","server_host":"p-irods-001","server_pid":2630598,"server_timestamp":"2024-07-25T13:29:39.365Z","server_type":"agent","server_zone":"nki"} {"log_category":"database","log_level":"info","log_message":"checkAndGetObjectId cmlExecuteNoAnswerSql(rollback) succeeded","request_api_name":"MOD_AVU_METADATA_AN","request_api_number":706,"request_api_version":"d","request_client_user":"ja.d.graaf","request_host":"172.31.32.83","request_proxy_user":"rods","request_release_version":"rods4.3.1","server_host":"p-irods-001","server_pid":2630854,"server_timestamp":"2024-07-25T13:30:08.134Z","server_type":"agent","server_zone":"nki"} {"log_category":"legacy","log_level":"info","log_message":"rsModAVUMetadata: rcModAVUMetadata failed","request_api_name":"MOD_AVU_METADATA_AN","request_api_number":706,"request_api_version":"d","request_client_user":"ja.d.graaf","request_host":"172.31.32.83","request_proxy_user":"rods","request_release_version":"rods4.3.1","server_host":"p-irods-001","server_pid":2630854,"server_timestamp":"2024-07-25T13:30:08.134Z","server_type":"agent","server_zone":"nki"} {"log_category":"database","log_level":"info","log_message":"checkAndGetObjectId cmlExecuteNoAnswerSql(rollback) succeeded","request_api_name":"MOD_AVU_METADATA_AN","request_api_number":706,"request_api_version":"d","request_client_user":"ja.d.graaf","request_host":"172.31.32.83","request_proxy_user":"rods","request_release_version":"rods4.3.1","server_host":"p-irods-001","server_pid":2630852,"server_timestamp":"2024-07-25T13:30:08.136Z","server_type":"agent","server_zone":"nki"}

korydraughn commented 1 month ago

I take it that rods is a rodsadmin, correct?

Do the additional users have enough permissions to read/write the data objects being tiered?

jadgraaf commented 1 month ago

I think I found a workaround:

I replaced the default query with a specific query and added that to the resource with imeta set -R fast_resc irods::storage_tiering::query archive_query specific.

First tried this query: SELECT DISTINCT R_DATA_MAIN.data_name, R_COLL_MAIN.coll_name, R_USER_MAIN.user_name, R_USER_MAIN.zone_name, R_DATA_MAIN.data_repl_num FROM R_DATA_MAIN INNER JOIN R_COLL_MAIN ON R_DATA_MAIN.coll_id = R_COLL_MAIN.coll_id INNER JOIN R_OBJT_ACCESS r_data_access ON R_DATA_MAIN.data_id = r_data_access.object_id INNER JOIN R_OBJT_METAMAP r_data_metamap ON R_DATA_MAIN.data_id = r_data_metamap.object_id INNER JOIN R_META_MAIN r_data_meta_main ON r_data_metamap.meta_id = r_data_meta_main.meta_id INNER JOIN R_USER_MAIN ON r_data_access.user_id = R_USER_MAIN.user_id WHERE r_data_meta_main.meta_attr_name = 'irods::data_type' AND r_data_meta_main.meta_attr_value = 'raw' AND R_DATA_MAIN.resc_id IN ('35069') ORDER BY R_COLL_MAIN.coll_name, R_DATA_MAIN.data_name, R_DATA_MAIN.data_repl_num

With result output: data_name coll_name user_name zone_name data_repl_num _metadata.json /nki/home/bioimaging/2024/A public nki 0 _metadata.json /nki/home/bioimaging/2024/A rods nki 0 test.tif /nki/home/bioimaging/2024/A public nki 0 test.tif /nki/home/bioimaging/2024/A rods nki 0

Notice the row per user/group with access (rods has owner, public has read)

WIth this result the tiering plugin did not work and produced the errors.

After that changed the query to: SELECT DISTINCT R_DATA_MAIN.data_name, R_COLL_MAIN.coll_name, R_USER_MAIN.user_name, R_USER_MAIN.zone_name, R_DATA_MAIN.data_repl_num FROM R_DATA_MAIN INNER JOIN R_COLL_MAIN ON R_DATA_MAIN.coll_id = R_COLL_MAIN.coll_id INNER JOIN R_OBJT_ACCESS r_data_access ON R_DATA_MAIN.data_id = r_data_access.object_id INNER JOIN R_OBJT_METAMAP r_data_metamap ON R_DATA_MAIN.data_id = r_data_metamap.object_id INNER JOIN R_META_MAIN r_data_meta_main ON r_data_metamap.meta_id = r_data_meta_main.meta_id INNER JOIN R_USER_MAIN ON r_data_access.user_id = R_USER_MAIN.user_id WHERE r_data_meta_main.meta_attr_name = 'irods::data_type' AND r_data_meta_main.meta_attr_value = 'raw' AND R_DATA_MAIN.resc_id IN ('35069') AND R_USER_MAIN.user_name = 'rods' ORDER BY R_COLL_MAIN.coll_name, R_DATA_MAIN.data_name, R_DATA_MAIN.data_repl_num

note the added wher username=rods to filter the results back to 1 row per data item.

Output: data_name coll_name user_name zone_name data_repl_num _metadata.json /nki/home/bioimaging/2024/A rods nki 0 test.tif /nki/home/bioimaging/2024/A rods nki 0

With this output the query plugin functions.

So the multiple rows per data item when more then 1 user/group has access to the item might have something to do with it? In any case it solves the problem for me.