After upgrade CRUD from 0.10.0 to 1.4.2 we got errors for different operations for old spaces.

[seet61]

We have old prodject with crud 0.10.0. After upgrade dependency to 1.4.2 we got a errors for different operation.

"line": 145,
"class_name": "SelectError",
"err": "attempt to call a nil value",
"file": "...arantool/tarantool_pep/.rocks/share/tarantool/errors.lua",
"stack": "stack traceback:\n\t...arantool/tarantool_pep/.rocks/share/tarantool/errors.lua:145: in function <...arantool/tarantool_pep/.rocks/share/tarantool/errors.lua:139>\n\t[C]: in function 'pcall'\n\t...tarantool_pep/.rocks/share/tarantool/crud/stats/init.lua:411: in function 'select'\n\t/usr/share/tarantool/tarantool_pep/app/roles/handlers.lua:199: in function </usr/share/tarantool/tarantool_pep/app/roles/handlers.lua:182>\n\t[C]: in function 'pcall'\n\t...pep/.rocks/share/tarantool/metrics/collectors/shared.lua:125: in function 'sub'\n\t...ool/tarantool_pep/.rocks/share/tarantool/http/server.lua:727: in function <...ool/tarantool_pep/.rocks/share/tarantool/http/server.lua:705>\n\t[C]: in function 'pcall'\n\t...ool/tarantool_pep/.rocks/share/tarantool/http/server.lua:813: in function 'process_client'\n\t...ool/tarantool_pep/.rocks/share/tarantool/http/server.lua:1286: in function <...ool/tarantool_pep/.rocks/share/tarantool/http/server.lua:1284>\n\t[C]: in function 'pcall'\n\tbuiltin/socket.lua:1084: in function <builtin/socket.lua:1082>",
"str": "SelectError: attempt to call a nil value"

"line": 183,
"class_name": "SelectError",
"err": "...ool/tarantool_pep/.rocks/share/tarantool/crud/select.lua:71: attempt to call field 'check_sharding_hash' (a nil value)",
"file": "...pep/.rocks/share/tarantool/crud/common/sharding/init.lua",
"stack": "stack traceback:\n\t...pep/.rocks/share/tarantool/crud/common/sharding/init.lua:183: in function <...pep/.rocks/share/tarantool/crud/common/sharding/init.lua:168>\n\t[C]: in function 'xpcall'\n\t...arantool/tarantool_pep/.rocks/share/tarantool/errors.lua:145: in function <...arantool/tarantool_pep/.rocks/share/tarantool/errors.lua:139>\n\t[C]: in function 'pcall'\n\t...tarantool_pep/.rocks/share/tarantool/crud/stats/init.lua:411: in function 'select'\n\t/usr/share/tarantool/tarantool_pep/app/roles/handlers.lua:199: in function </usr/share/tarantool/tarantool_pep/app/roles/handlers.lua:182>\n\t[C]: in function 'pcall'\n\t...pep/.rocks/share/tarantool/metrics/collectors/shared.lua:125: in function 'sub'\n\t...ool/tarantool_pep/.rocks/share/tarantool/http/server.lua:727: in function <...ool/tarantool_pep/.rocks/share/tarantool/http/server.lua:705>\n\t[C]: in function 'pcall'\n\t...ool/tarantool_pep/.rocks/share/tarantool/http/server.lua:813: in function 'process_client'\n\t...ool/tarantool_pep/.rocks/share/tarantool/http/server.lua:1286: in function <...ool/tarantool_pep/.rocks/share/tarantool/http/server.lua:1284>\n\t[C]: in function 'pcall'\n\tbuiltin/socket.lua:1084: in function <builtin/socket.lua:1082>",
"str": "SelectError: ...ool/tarantool_pep/.rocks/share/tarantool/crud/select.lua:71: attempt to call field 'check_sharding_hash' (a nil value)"

After return 0.10.0 we don't have errors. Why?

cartridge == 2.8.4

# tarantool -v
Tarantool 2.10.8-0-g4472d9247
Target: Linux-x86_64-RelWithDebInfo
Build options: cmake . -DCMAKE_INSTALL_PREFIX=/usr -DENABLE_BACKTRACE=ON
Compiler: GNU-8.3.1
C_FLAGS:-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches   -m64 -mtune=generic -fexceptions -funwind-tables -fno-common -fopenmp -msse2 -Wformat -Wformat-security -Werror=format-security -fstack-protector-strong -fPIC -fmacro-prefix-map=/build/usr/src/debug/tarantool-2.10.8=. -std=c11 -Wall -Wextra -Wno-strict-aliasing -Wno-char-subscripts -Wno-format-truncation -Wno-gnu-alignof-expression -fno-gnu89-inline -Wno-cast-function-type -Werror
CXX_FLAGS:-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches   -m64 -mtune=generic -fexceptions -funwind-tables -fno-common -fopenmp -msse2 -Wformat -Wformat-security -Werror=format-security -fstack-protector-strong -fPIC -fmacro-prefix-map=/build/usr/src/debug/tarantool-2.10.8=. -std=c++11 -Wall -Wextra -Wno-strict-aliasing -Wno-char-subscripts -Wno-format-truncation -Wno-invalid-offsetof -Wno-gnu-alignof-expression -Wno-cast-function-type -Werror

[DifferentialOrange]

After upgrade dependency to 1.4.2 we got a errors for different operation.

Can you describe the upgrade process? Does it involves having an intermediate cluster with 0.10.0 routers and 1.4.2 storages? Vice versa? Mixed 0.10.0 and 1.4.2 storage replicasets? Code hot reload?

It also would be nice to provide the example of operations you have called to trigger the errors. Also, please provide errors library version.

It's rather hard to understand what exactly had happened based on error messages, but it seems that .rocks are likely messed up. For now, the only way of getting an .rocks/share/tarantool/crud/select.lua:71: attempt to call field 'check_sharding_hash' (a nil value) error I can see is having crud/select.lua from crud 1.4.2 and crud/common/sharding.lua from 0.10.0 in the same .rocks folder, which is really weird.

[seet61]

@DifferentialOrange ss usually we change version at rockspec file, build rpm by ci/cd and install at test servers. We have 2 servers. On each router, master and replica.

After install rpm each instance was restarted.

This errors reproduces at dev zone. https://cloud.mail.ru/public/YQfH/x3Kqdy1xL At rockspec 'crud == 0.10.0'

[root@WSRU-BIL-028 tarantool_pep]# cartridge build
• Build application in /mnt/d/Projects/tarantool_pep
• Running `cartridge.pre-build`
• Running `tarantoolctl rocks make` 
• Application was successfully built
[root@WSRU-BIL-028 tarantool_pep]# ./start.sh
• tarantool_pep.s1-master... OK
• tarantool_pep.s2-master... OK
• tarantool_pep.s1-replica... OK
• tarantool_pep.router... OK
• tarantool_pep.s2-replica... OK
[root@WSRU-BIL-028 tarantool_pep]# tarantoolctl connect admin:tarantool_pep@localhost:3401
connected to localhost:3401
localhost:3401> crud = require('crud')
---
...
localhost:3401> crud.select('Sherlock', {{'==', 'Sherlock_for_notification_idx', {'79028525759', '40627'}}}, {timeout = 5, prefer_replica = false, balance = true})
---
- metadata: [{'name': 'msisdn', 'type': 'string'}, {'name': 'subscriber', 'type': 'string'},{'name': 'subscriberId', 'type': 'string'}, {'name': 'partner_name', 'type': 'string'},                                                                                                                                                                                        {'name': 'partner_subs_name', 'type': 'string'}, {'name': 'payment_type', 'type': 'string'},                                                                                                                                                                                   {'name': 'epc_component_name', 'type': 'string'}, {'name': 'epc_product_offering_id','type': 'string'}, {'name': 'billing_serv_id', 'type': 'string'}, {'name': 'subs_status','type': 'string'}, {'name': 'subs_order_date', 'type': 'string'}, {'name': 'subs_activate_date','type': 'string'}, {'name': 'ttl', 'type': 'unsigned'}, {'name': 'bucket_id','type': 'unsigned'}]
rows: []
... 

After that change at rockspec 'crud == 1.4.2'.

[root@WSRU-BIL-028 tarantool_pep]# cartridge build 
• Build application in /mnt/d/Projects/tarantool_pep
• Running `cartridge.pre-build`
• Running `tarantoolctl rocks make`
• Application was successfully built
[root@WSRU-BIL-028 tarantool_pep]# ./start.sh      
• tarantool_pep.s1-replica... OK                
• tarantool_pep.s2-master... OK                 
• tarantool_pep.s1-master... OK                 
• tarantool_pep.router... OK                    
• tarantool_pep.s2-replica... OK             
[root@WSRU-BIL-028 tarantool_pep]# tarantoolctl connect admin:tarantool_pep@localhost:3401
 connected to localhost:3401
 localhost:3401> crud = require('crud')
 ---
 ... 
 localhost:3401> crud.select('Sherlock', {{'==', 'Sherlock_for_notification_idx', {'79028525759', '40627'}}}, {timeout = 5, prefer_replica = false, balance = true}) 
 ---
 - null
 - line: 145
 class_name: SelectError
 err: attempt to call a nil value 
 file: '...Projects/tarantool_pep/.rocks/share/tarantool/errors.lua' 
 stack: "stack traceback:\n\t...Projects/tarantool_pep/.rocks/share/tarantool/errors.lua:145: 
 in function <...Projects/tarantool_pep/.rocks/share/tarantool/errors.lua:139>\n\t[C]:  
 in function 'pcall'\n\tbuiltin/box/console.lua:412: in function <builtin/box/console.lua:385>\n\t[C]: 
 at 0x55bbf21c4cf0" 
 str: 'SelectError: attempt to call a nil value'   
 ...  
 localhost:3401>   

[seet61]

Its working correctly only if you remove .rocks at project in dev or at dirrectory prodject before install rpm.

[DifferentialOrange]

Its working correctly only if you remove .rocks at project in dev or at dirrectory prodject before install rpm.

Well, having two cruds installed at the same time for an application isn't a healthy approach.

[seet61]

But why it happends? it is not first my update of deps at tarantool. I have this error first time.

[DifferentialOrange]

Let's explore whether you really have several cruds installed or not.

local _ = require('crud')

local response = {}

for package_name, package_contents in pairs(package.loaded) do
    if package_name:find('crud') ~= nil and type(package_contents) == 'table' then
        for field_name, field in pairs(package_contents) do
            if type(field) == 'function' then
                local source = debug.getinfo(field).source

                table.insert(response, {
                    ['package'] = package_name,
                    ['function'] = field_name,
                    source = source,
                })
            end
        end
    end
end

return response

Running this script should explore what files were used to load the modules, If source: contents are inconsistent, it means that several packages are used and your update procedure doesn't replace old rock files with new ones.

[seet61]

localhost:3401> response
---
- - function: call
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/get.lua'
    package: crud.get
  - function: init
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/get.lua'
    package: crud.get
  - function: get_sharding_func_hash
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/sharding/storage_metadata_cache.lua'
    package: crud.common.sharding.storage_metadata_cache
  - function: get_sharding_key_hash
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/sharding/storage_metadata_cache.lua'
    package: crud.common.sharding.storage_metadata_cache
  - function: drop_caches
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/sharding/storage_metadata_cache.lua'
    package: crud.common.sharding.storage_metadata_cache
  - function: get_instance
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/sharding/router_metadata_cache.lua'
    package: crud.common.sharding.router_metadata_cache
  - function: drop_caches
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/sharding/router_metadata_cache.lua'
    package: crud.common.sharding.router_metadata_cache
  - function: drop_instance
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/sharding/router_metadata_cache.lua'
    package: crud.common.sharding.router_metadata_cache
  - function: observe_map_reduces
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/local_registry.lua'
    package: crud.stats.local_registry
  - function: observe_fetch
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/local_registry.lua'
    package: crud.stats.local_registry
  - function: destroy
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/local_registry.lua'
    package: crud.stats.local_registry
  - function: observe
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/local_registry.lua'
    package: crud.stats.local_registry
  - function: init
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/local_registry.lua'
    package: crud.stats.local_registry
  - function: get
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/local_registry.lua'
    package: crud.stats.local_registry
  - function: get
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/stash.lua'
    package: crud.common.stash
  - function: setup_cartridge_reload
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/stash.lua'
    package: crud.common.stash
  - function: get_comparators_by_type
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/compare/type_comparators.lua'
    package: crud.compare.type_comparators
  - function: execute
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/select/executor.lua'
    package: crud.select.executor
  - function: stop
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/cartridge/roles/crud-storage.lua'
    package: cartridge.roles.crud-storage
  - function: init
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/cartridge/roles/crud-storage.lua'
    package: cartridge.roles.crud-storage
  - function: gen_func
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/compare/comparators.lua'
    package: crud.compare.comparators
  - function: get_cmp_operator
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/compare/comparators.lua'
    package: crud.compare.comparators
  - function: gen_tuples_comparator
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/compare/comparators.lua'
    package: crud.compare.comparators
  - function: update_key_parts_by_field_names
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/compare/comparators.lua'
    package: crud.compare.comparators
  - function: observe_map_reduces
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/metrics_registry.lua'
    package: crud.stats.metrics_registry
  - function: is_supported
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/metrics_registry.lua'
    package: crud.stats.metrics_registry
  - function: observe_fetch
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/metrics_registry.lua'
    package: crud.stats.metrics_registry
  - function: observe
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/metrics_registry.lua'
    package: crud.stats.metrics_registry
  - function: init
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/metrics_registry.lua'
    package: crud.stats.metrics_registry
  - function: destroy
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/metrics_registry.lua'
    package: crud.stats.metrics_registry
  - function: get
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/metrics_registry.lua'
    package: crud.stats.metrics_registry
  - function: call
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/update.lua'
    package: crud.update
  - function: init
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/update.lua'
    package: crud.update
  - function: build_collectors
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/registry_utils.lua'
    package: crud.stats.registry_utils
  - function: init_collectors_if_required
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/registry_utils.lua'
    package: crud.stats.registry_utils
  - function: collect
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/map_call_cases/batch_postprocessor.lua'
    package: crud.common.map_call_cases.batch_postprocessor
  - function: enable
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/ratelimit.lua'
    package: crud.ratelimit
  - function: disable
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/ratelimit.lua'
    package: crud.ratelimit
  - function: new
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/ratelimit.lua'
    package: crud.ratelimit
  - function: get_tarantool_iter
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/compare/conditions.lua'
    package: crud.compare.conditions
  - function: parse
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/compare/conditions.lua'
    package: crud.compare.conditions
  - function: object
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/upsert.lua'
    package: crud.upsert
  - function: init
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/upsert.lua'
    package: crud.upsert
  - function: tuple
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/upsert.lua'
    package: crud.upsert
  - function: construct_as_index_obj_cache
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/sharding/sharding_key.lua'
    package: crud.common.sharding.sharding_key
  - function: extract_from_pk
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/sharding/sharding_key.lua'
    package: crud.common.sharding.sharding_key
  - function: enable
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/init.lua'
    package: crud.stats
  - function: reset
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/init.lua'
    package: crud.stats
  - function: is_enabled
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/init.lua'
    package: crud.stats
  - function: update_map_reduces
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/init.lua'
    package: crud.stats
  - function: get_default_driver
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/init.lua'
    package: crud.stats
  - function: update_fetch_stats
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/init.lua'
    package: crud.stats
  - function: wrap
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/init.lua'
    package: crud.stats
  - function: is_driver_supported
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/init.lua'
    package: crud.stats
  - function: disable
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/init.lua'
    package: crud.stats
  - function: get
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/init.lua'
    package: crud.stats
  - function: get_vshard_call_name
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/call.lua'
    package: crud.common.call
  - function: any
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/call.lua'
    package: crud.common.call
  - function: single
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/call.lua'
    package: crud.common.call
  - function: map
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/call.lua'
    package: crud.common.call
  - function: call
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/schema.lua'
    package: crud.schema
  - function: object
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/insert.lua'
    package: crud.insert
  - function: init
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/insert.lua'
    package: crud.insert
  - function: tuple
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/insert.lua'
    package: crud.insert
  - function: replace_object_many
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/init.lua'
    package: crud
  - function: readview
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/readview.lua'
    package: crud
  - function: upsert
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/init.lua'
    package: crud
  - function: len
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/init.lua'
    package: crud
  - function: delete
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/init.lua'
    package: crud
  - function: replace_object
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/init.lua'
    package: crud
  - function: truncate
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/init.lua'
    package: crud
  - function: init_router
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud.lua'
    package: crud
  - function: upsert_object
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/init.lua'
    package: crud
  - function: reset_stats
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/init.lua'
    package: crud
  - function: stats
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/init.lua'
    package: crud
  - function: cut_rows
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud
  - function: replace_many
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/init.lua'
    package: crud
  - function: stop_router
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud.lua'
    package: crud
  - function: schema
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/schema.lua'
    package: crud
  - function: storage_info
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud
  - function: stop_storage
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud.lua'
    package: crud
  - function: insert_many
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/init.lua'
    package: crud
  - function: cut_objects
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud
  - function: max
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/init.lua'
    package: crud
  - function: replace
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/init.lua'
    package: crud
  - function: pairs
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/init.lua'
    package: crud
  - function: update
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/init.lua'
    package: crud
  - function: insert
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/init.lua'
    package: crud
  - function: upsert_many
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/init.lua'
    package: crud
  - function: insert_object
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/init.lua'
    package: crud
  - function: select
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/init.lua'
    package: crud
  - function: insert_object_many
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/init.lua'
    package: crud
  - function: unflatten_rows
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud
  - function: init_storage
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud.lua'
    package: crud
  - function: min
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/init.lua'
    package: crud
  - function: count
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/init.lua'
    package: crud
  - function: upsert_object_many
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/init.lua'
    package: crud
  - function: get
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/stats/init.lua'
    package: crud
  - function: fetch_sharding_key_on_router
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/sharding/sharding_metadata.lua'
    package: crud.common.sharding.sharding_metadata
  - function: update_sharding_key_cache
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/sharding/sharding_metadata.lua'
    package: crud.common.sharding.sharding_metadata
  - function: fetch_sharding_func_on_router
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/sharding/sharding_metadata.lua'
    package: crud.common.sharding.sharding_metadata
  - function: fetch_on_storage
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/sharding/sharding_metadata.lua'
    package: crud.common.sharding.sharding_metadata
  - function: init
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/sharding/sharding_metadata.lua'
    package: crud.common.sharding.sharding_metadata
  - function: reload_sharding_cache
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/sharding/sharding_metadata.lua'
    package: crud.common.sharding.sharding_metadata
  - function: update_sharding_func_cache
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/sharding/sharding_metadata.lua'
    package: crud.common.sharding.sharding_metadata
  - function: new
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/compare/keydef.lua'
    package: crud.compare.keydef
  - function: stop
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/cartridge/roles/crud-router.lua'
    package: cartridge.roles.crud-router
  - function: apply_config
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/cartridge/roles/crud-router.lua'
    package: cartridge.roles.crud-router
  - function: init
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/cartridge/roles/crud-router.lua'
    package: cartridge.roles.crud-router
  - function: validate_config
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/cartridge/roles/crud-router.lua'
    package: cartridge.roles.crud-router
  - function: construct_as_callable_obj_cache
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/sharding/sharding_func.lua'
    package: crud.common.sharding.sharding_func
  - function: max
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/borders.lua'
    package: crud.borders
  - function: init
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/borders.lua'
    package: crud.borders
  - function: min
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/borders.lua'
    package: crud.borders
  - function: call
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/count.lua'
    package: crud.count
  - function: init
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/count.lua'
    package: crud.count
  - function: call
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/len.lua'
    package: crud.len
  - function: init
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/len.lua'
    package: crud.len
  - function: call
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/delete.lua'
    package: crud.delete
  - function: init
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/delete.lua'
    package: crud.delete
  - function: tuples
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/upsert_many.lua'
    package: crud.upsert_many
  - function: init
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/upsert_many.lua'
    package: crud.upsert_many
  - function: objects
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/upsert_many.lua'
    package: crud.upsert_many
  - function: new
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/compare/plan.lua'
    package: crud.compare.plan
  - function: pairs
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/select/compat/select.lua'
    package: crud.select
  - function: call
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/select/compat/select.lua'
    package: crud.select
  - function: init
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/select.lua'
    package: crud.select
  - function: pairs
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/select/compat/select.lua'
    package: crud.select.compat.select
  - function: call
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/select/compat/select.lua'
    package: crud.select.compat.select
  - function: init
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/select.lua'
    package: crud.select.compat.select
  - function: exists
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/compat.lua'
    package: crud.common.compat
  - function: require
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/compat.lua'
    package: crud.common.compat
  - function: check_select_safety
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/select/compat/common.lua'
    package: crud.select.compat.common
  - function: call
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/truncate.lua'
    package: crud.truncate
  - function: init
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/truncate.lua'
    package: crud.truncate
  - function: extract_key
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: table_count
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: is_version_ge
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: is_field_not_found
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: tarantool_supports_jsonpath_indexes
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: merge_primary_key_parts
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: format_replicaset_error
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: get_tarantool_version
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: flatten
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: get_space_format
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: enrich_field_names_with_cmp_key
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: tarantool_supports_fieldpaths
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: get_version_suffix_weight
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: tarantool_supports_uuids
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: fetch_latest_metadata_for_select
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: get_self_vshard_replicaset
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: cut_rows
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: flatten_obj_reload
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: get_version_suffix
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: tarantool_version_at_least
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: fetch_latest_metadata_when_map_storages
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: get_spaces
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: invert_tarantool_iter
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: reverse_inplace
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: fetch_latest_metadata_when_single_storage
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: is_cartridge_hotreload_supported
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: get_vshard_router_instance
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: convert_operations
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: init_storage_call
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: tarantool_supports_external_merger
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: add_intermediate_nullable_fields
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: storage_info_on_storage
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: storage_info
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: update_storage_call_error_description
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: list_slice
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: cut_objects
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: list_extend
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: tarantool_has_builtin_merger
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: is_uuid
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: format_result
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: get_format_fieldno_map
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: get_storage_call
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: get_fields_format
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: merge_options
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: get_index_fieldno_map
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: get_bucket_id_fieldno
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: unflatten_rows
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: is_version_in_range
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: get_space
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: check_name_isident
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: unflatten
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: tarantool_supports_netbox_skip_header_option
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua'
    package: crud.common.utils
  - function: get
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/map_call_cases/batch_upsert_iter.lua'
    package: crud.common.map_call_cases.batch_upsert_iter
  - function: new
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/map_call_cases/batch_upsert_iter.lua'
    package: crud.common.map_call_cases.batch_upsert_iter
  - function: new_readview
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/select/merger.lua'
    package: crud.select.merger
  - function: new
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/select/merger.lua'
    package: crud.select.merger
  - function: tuples
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/replace_many.lua'
    package: crud.replace_many
  - function: init
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/replace_many.lua'
    package: crud.replace_many
  - function: objects
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/replace_many.lua'
    package: crud.replace_many
  - function: object
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/replace.lua'
    package: crud.replace
  - function: init
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/replace.lua'
    package: crud.replace
  - function: tuple
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/replace.lua'
    package: crud.replace
  - function: get
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/map_call_cases/batch_insert_iter.lua'
    package: crud.common.map_call_cases.batch_insert_iter
  - function: new
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/map_call_cases/batch_insert_iter.lua'
    package: crud.common.map_call_cases.batch_insert_iter
  - function: complement_batching_errors
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/batching_utils.lua'
    package: crud.common.batching_utils
  - function: construct_sharding_hash_mismatch_errors
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/batching_utils.lua'
    package: crud.common.batching_utils
  - function: tuples
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/insert_many.lua'
    package: crud.insert_many
  - function: init
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/insert_many.lua'
    package: crud.insert_many
  - function: objects
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/insert_many.lua'
    package: crud.insert_many
  - function: init
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/readview.lua'
    package: crud.readview
  - function: new
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/readview.lua'
    package: crud.readview
  - function: update_cache
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/sharding_key.lua'
    package: crud.common.sharding_key
  - function: key_get_bucket_id
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/sharding.lua'
    package: crud.common.sharding
  - function: tuple_set_and_return_bucket_id
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/sharding.lua'
    package: crud.common.sharding
  - function: tuple_get_bucket_id
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/sharding.lua'
    package: crud.common.sharding
  - function: get
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/map_call_cases/base_postprocessor.lua'
    package: crud.common.map_call_cases.base_postprocessor
  - function: collect
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/map_call_cases/base_postprocessor.lua'
    package: crud.common.map_call_cases.base_postprocessor
  - function: new
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/map_call_cases/base_postprocessor.lua'
    package: crud.common.map_call_cases.base_postprocessor
  - function: filter_tuples_fields
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/schema.lua'
    package: crud.common.schema
  - function: wrap_box_space_func_result
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/schema.lua'
    package: crud.common.schema
  - function: wrap_func_result
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/schema.lua'
    package: crud.common.schema
  - function: reload_schema
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/schema.lua'
    package: crud.common.schema
  - function: get_normalized_space_schema
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/schema.lua'
    package: crud.common.schema
  - function: batching_result_needs_reload
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/schema.lua'
    package: crud.common.schema
  - function: result_needs_reload
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/schema.lua'
    package: crud.common.schema
  - function: wrap_func_reload
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/schema.lua'
    package: crud.common.schema
  - function: truncate_row_trailing_fields
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/schema.lua'
    package: crud.common.schema
  - function: filter_obj_fields
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/schema.lua'
    package: crud.common.schema
  - function: has_next
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/map_call_cases/base_iter.lua'
    package: crud.common.map_call_cases.base_iter
  - function: get
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/map_call_cases/base_iter.lua'
    package: crud.common.map_call_cases.base_iter
  - function: new
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/map_call_cases/base_iter.lua'
    package: crud.common.map_call_cases.base_iter
  - function: is_default
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/collations.lua'
    package: crud.common.collations
  - function: is_unicode
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/collations.lua'
    package: crud.common.collations
  - function: get
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/collations.lua'
    package: crud.common.collations
  - function: gen_func
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/compare/filters.lua'
    package: crud.compare.filters
  - function: compute_hash
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/sharding/utils.lua'
    package: crud.common.sharding.utils
  - function: extract_sharding_func_def
    source: '@/mnt/d/Projects/tarantool_pep/.rocks/share/tarantool/crud/common/sharding/utils.lua'
    package: crud.common.sharding.utils
...

[DifferentialOrange]

Thank you for the insights! It seems that all of your files are from the same source, so the assumption about two crud installations on the server is wrong.

On the other hand, rock contents itself is seem to be broken. You have crud/common/sharding.lua file, which was a part of the rock for crud 0.10.0, but was moved to crud/common/sharding/init.lua in crud 0.11.0. On the other hand, you have files from newer crud releases. So you have both old and new version files in your .rocks directory (and miss some new version files), which is the reason why everything falls apart. I'm not sure what could be the reason: your .rocks could have been broken manually, or it could be the issue with luarocks, tarantoolctl or cartridge. But it is most likely that artifact build is the reason. Can you rebuild your application artifact from scratch after cleaning your local .rocks? (I don't know what your packaging pipeline is.) If it won't fix your issue, please provide information on how you build your application artifact.

[seet61]

Our ci/cd process use cartridge pack for every build at new clean directory. But after packeging rpm it copy to destination server and install it via yum.

May be it happend because yum not remove existing direcoty aready installed project?

[DifferentialOrange]

It may be, but it's still weird that you not only have crud/common/sharding.lua, but also miss crud/common/sharding/init.lua. Can you explore the contents of your rpm artifact to check whether the issue starts on its build or after unpacking?

[seet61]

# cartridge pack rpm --version=1.2.41
   • Packing tarantool_pep into rpm
   • Temporary directory is set to /root/.cartridge/tmp/pack-a69iz6eykn
   • Initialize application dir
   • Build application in /root/.cartridge/tmp/pack-a69iz6eykn/package-files/usr/share/tarantool/tarantool_pep
   • Running `cartridge.pre-build`
   • Running `tarantoolctl rocks make`
   • Application was successfully built
   • Running `cartridge.post-build`
   • Generate VERSION file
   • Generate VERSION.lua file
   • Initialize systemd dir
   • Initialize tmpfiles dir
   • Created result RPM package: /var/tmp/tarantool_pep/tarantool_pep-1.2.41-1.x86_64.rpm
   • Application was successfully packed

# rpm -qlp /var/tmp/tarantool_pep/tarantool_pep-1.2.41-1.x86_64.rpm | grep 'crud'
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/cartridge/roles/crud-router.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/cartridge/roles/crud-storage.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/borders.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/cfg.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/common
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/common/batching_utils.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/common/call.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/common/collations.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/common/compat.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/common/const.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/common/dev_checks.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/common/map_call_cases
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/common/map_call_cases/base_iter.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/common/map_call_cases/base_postprocessor.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/common/map_call_cases/batch_insert_iter.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/common/map_call_cases/batch_postprocessor.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/common/map_call_cases/batch_upsert_iter.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/common/schema.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/common/sharding
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/common/sharding/init.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/common/sharding/router_metadata_cache.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/common/sharding/sharding_func.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/common/sharding/sharding_key.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/common/sharding/sharding_metadata.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/common/sharding/storage_metadata_cache.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/common/sharding/utils.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/common/sharding_func.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/common/sharding_key.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/common/stash.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/common/utils.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/compare
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/compare/comparators.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/compare/conditions.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/compare/filters.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/compare/keydef.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/compare/plan.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/compare/type_comparators.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/count.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/delete.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/get.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/insert.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/insert_many.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/len.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/ratelimit.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/readview.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/replace.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/replace_many.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/schema.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/select
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/select.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/select/compat
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/select/compat/common.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/select/compat/select.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/select/compat/select_old.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/select/executor.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/select/iterator.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/select/merger.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/stats
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/stats/init.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/stats/local_registry.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/stats/metrics_registry.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/stats/operation.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/stats/registry_utils.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/truncate.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/update.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/upsert.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/upsert_many.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/crud/version.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/rocks/crud
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/rocks/crud/1.4.2-1
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/rocks/crud/1.4.2-1/crud-1.4.2-1.rockspec
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/rocks/crud/1.4.2-1/doc
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/rocks/crud/1.4.2-1/doc/dev
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/rocks/crud/1.4.2-1/doc/dev/schema.md
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/rocks/crud/1.4.2-1/doc/pairs.md
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/rocks/crud/1.4.2-1/doc/playground.lua
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/rocks/crud/1.4.2-1/doc/select.md
/usr/share/tarantool/tarantool_pep/.rocks/share/tarantool/rocks/crud/1.4.2-1/rock_manifest

[DifferentialOrange]

Everything seems fine here, so it is likely that issue happens on the installation stage.

[seet61]

But if ci/cd remove /usr/share/tarantool/tarantool_pep/.rocks/ directory. everething will be ok. Could you add this instruction for rpm at pack?

or may be you have another idea to solve this problem?

[DifferentialOrange]

But if ci/cd remove /usr/share/tarantool/tarantool_pep/.rocks/ directory. everething will be ok. Could you add this instruction for rpm at pack?

or may be you have another idea to solve this problem?

I had failed to reproduce installation issue with cartridge-cli 2.12.10, Fedora 36 and dnf. Please, provide info on your toolchain.

Create application

cartridge create --name=myapp
cd ./myapp

Create v1 package

git diff
diff --git a/myapp-scm-1.rockspec b/myapp-scm-1.rockspec
index a186c50..1e923b1 100644
--- a/myapp-scm-1.rockspec
+++ b/myapp-scm-1.rockspec
@@ -12,6 +12,7 @@ dependencies = {
     'metrics == 1.0.0-1',
     'cartridge-metrics-role == 0.1.1-1',
     'cartridge-cli-extensions == 1.1.1-1',
+    'crud == 0.10.0-1',
 }
 build = {
     type = 'none';

cartridge pack rpm --version=0.1.0

Create v2 package

git diff
diff --git a/myapp-scm-1.rockspec b/myapp-scm-1.rockspec
index a186c50..1e923b1 100644
--- a/myapp-scm-1.rockspec
+++ b/myapp-scm-1.rockspec
@@ -12,6 +12,7 @@ dependencies = {
     'metrics == 1.0.0-1',
     'cartridge-metrics-role == 0.1.1-1',
     'cartridge-cli-extensions == 1.1.1-1',
+    'crud == 1.4.2-1',
 }
 build = {
     type = 'none';

cartridge pack rpm --version=0.2.0

Check v1 package crud 0.10.0 files are here, crud 1.4.2 are not

rpm -qlp ./myapp-0.1.0-1.x86_64.rpm | grep ".rocks/share/tarantool/crud/common/sharding/init.lua"
rpm -qlp ./myapp-0.1.0-1.x86_64.rpm | grep ".rocks/share/tarantool/crud/common/sharding.lua"
/usr/share/tarantool/myapp/.rocks/share/tarantool/crud/common/sharding.lua

Check v2 package crud 1.4.2 files are here, crud 0.10.0 are not

rpm -qlp ./myapp-0.2.0-1.x86_64.rpm | grep ".rocks/share/tarantool/crud/common/sharding/init.lua"
/usr/share/tarantool/myapp/.rocks/share/tarantool/crud/common/sharding/init.lua
rpm -qlp ./myapp-0.2.0-1.x86_64.rpm | grep ".rocks/share/tarantool/crud/common/sharding.lua"

Store artifacts

cp ./myapp-0.* ../artifacts/

Start a dev container

docker run -it -v `pwd`/artifacts:/artifacts fedora:36

Prepare for installation

curl -L https://tarantool.io/release/2/installer.sh | bash

Install v1 package

dnf install /artifacts/myapp-0.1.0-1.x86_64.rpm -y

Check v1 installation crud 0.10.0 files are here, crud 1.4.2 are not

[root@8d7fb5160fd8 /]# ls /usr/share/tarantool/myapp/.rocks/share/tarantool/crud/common/sharding.lua 
/usr/share/tarantool/myapp/.rocks/share/tarantool/crud/common/sharding.lua
[root@8d7fb5160fd8 /]# ls /usr/share/tarantool/myapp/.rocks/share/tarantool/crud/common/sharding/init.lua 
ls: cannot access '/usr/share/tarantool/myapp/.rocks/share/tarantool/crud/common/sharding/init.lua': No such file or directory

Install v2 package

dnf install /artifacts/myapp-0.2.0-1.x86_64.rpm -y

Check v2 installation crud 1.4.2 files are here, crud 0.10.0 are not

[root@8d7fb5160fd8 /]# ls /usr/share/tarantool/myapp/.rocks/share/tarantool/crud/common/sharding.lua 
ls: cannot access '/usr/share/tarantool/myapp/.rocks/share/tarantool/crud/common/sharding.lua': No such file or directory
[root@8d7fb5160fd8 /]# ls /usr/share/tarantool/myapp/.rocks/share/tarantool/crud/common/sharding/init.lua 
/usr/share/tarantool/myapp/.rocks/share/tarantool/crud/common/sharding/init.lua

[DifferentialOrange]

Will close this one soon if no further information will be provided.