provider fails to broadcast a bid for orders with mixed `count`

[andy108369]

The provider encounters a capabilities mismatch error when attempting to broadcast a bid for an order request where the count for one service is higher than the count for another service.

Scenarios

"no bid" here means provider received capabilities mismatch error

scenario 1 - bids (2 in total) :green_circle:

service foo, count 1 service bar, count 1

scenario 2 - no bid (3 in total) :boom:

service foo, count 1 service bar, count 2

scenario 3 - bids (4 in total) :green_circle:

service foo, count 2 service bar, count 2

scenario 4 - no bid (5 in total) :boom:

service foo, count 2 service bar, count 3

scenario 5 - bids (6 in total) :green_circle:

service foo, count 3 service bar, count 3

scenario 6 - bids (8 in total) :green_circle:

service foo, count 4 service bar, count 4

scenario 7 - no bids (9 in total) :boom:

service foo, count 5 service bar, count 4

scenario 8 - bids (10 in total) :green_circle:

service foo, count 5 service bar, count 5

scenario 9 - no bids (4 in total, but 3 services now) :boom:

service foo, count 1 service bar, count 2 service baz, count 1

scenario 10 - no bids (5 in total, 3 services) :boom:

service foo, count 1 service bar, count 2 service baz, count 2

scenario 11 - no bids (6 in total, 3 services) :green_circle:

service foo, count 2 service bar, count 2 service baz, count 2

scenario 12 - bids (3 in total, 3 services) :green_circle:

service foo, count 1 service bar, count 1 service baz, count 1

scenario 13 - bids (4 in total, 4 services) :green_circle:

service foo, count 1 service bar, count 1 service baz, count 1 service qux, count 1

scenario 14 - no bids (5 in total, 4 services) :boom:

service foo, count 2 service bar, count 1 service baz, count 1 service qux, count 1

scenario 15 - no bids (6 in total, 4 services) :boom:

service foo, count 2 service bar, count 2 service baz, count 1 service qux, count 1

scenario 16 - no bids (7 in total, 4 services) :boom:

service foo, count 2 service bar, count 2 service baz, count 2 service qux, count 1

scenario 17 - bids (8 in total, 4 services) :green_circle:

service foo, count 2 service bar, count 2 service baz, count 2 service qux, count 2

SW

provider & client both 0.4.8 akash node 0.28.2

provider log: capabilities mismatch

service foo, count 1 service bar, count 2

I[2023-11-20|18:03:25.755] order detected                               module=bidengine-service cmp=provider order=order/akash1h24fljt7p0nh82cq0za0uhsct3sfwsfu9w3c9h/13768245/1/1
I[2023-11-20|18:03:25.757] group fetched                                module=bidengine-order cmp=provider order=akash1h24fljt7p0nh82cq0za0uhsct3sfwsfu9w3c9h/13768245/1/1
I[2023-11-20|18:03:25.758] requesting reservation                       module=bidengine-order cmp=provider order=akash1h24fljt7p0nh82cq0za0uhsct3sfwsfu9w3c9h/13768245/1/1
D[2023-11-20|18:03:25.758] reservation requested                        module=provider-cluster cmp=provider cmp=service cmp=inventory-service order=akash1h24fljt7p0nh82cq0za0uhsct3sfwsfu9w3c9h/13768245/1/1 resources[{resource:{id:1,cpu:{units:{val:1000}},memory:{size:{val:1073741824}},storage:[{name:default,size:{val:1073741824}}],gpu:{units:{val:0}},endpoints:[{kind:1,sequence_number:0}]},count:1,price:{denom:uakt,amount:1000000.000000000000000000}},{resource:{id:2,cpu:{units:{val:1000}},memory:{size:{val:1073741824}},storage:[{name:default,size:{val:1073741824}}],gpu:{units:{val:0}},endpoints:[{kind:1,sequence_number:0}]},count:2,price:{denom:uakt,amount:1000000.000000000000000000}}]=(MISSING)
D[2023-11-20|18:03:25.758] reservation count                            module=provider-cluster cmp=provider cmp=service cmp=inventory-service cnt=1
I[2023-11-20|18:03:25.758] Reservation fulfilled                        module=bidengine-order cmp=provider order=akash1h24fljt7p0nh82cq0za0uhsct3sfwsfu9w3c9h/13768245/1/1
D[2023-11-20|18:03:26.418] submitting fulfillment                       module=bidengine-order cmp=provider order=akash1h24fljt7p0nh82cq0za0uhsct3sfwsfu9w3c9h/13768245/1/1 price=9.617504000000000000uakt
E[2023-11-20|18:03:26.422] transaction broadcast failed                 cmp=provider cmp=client/broadcaster err="rpc error: code = Unknown desc = rpc error: code = Unknown desc = failed to execute message; message index: 0: capabilities mismatch [cosmos/cosmos-sdk@v0.45.16/baseapp/baseapp.go:781] With gas wanted: '0' and gas used: '62826' : unknown request"
E[2023-11-20|18:03:26.422] unable to broadcast messages                 cmp=provider cmp=client/broadcaster error="rpc error: code = Unknown desc = rpc error: code = Unknown desc = failed to execute message; message index: 0: capabilities mismatch [cosmos/cosmos-sdk@v0.45.16/baseapp/baseapp.go:781] With gas wanted: '0' and gas used: '62826' : unknown request"
E[2023-11-20|18:03:26.422] bid failed                                   module=bidengine-order cmp=provider order=akash1h24fljt7p0nh82cq0za0uhsct3sfwsfu9w3c9h/13768245/1/1 err="rpc error: code = Unknown desc = rpc error: code = Unknown desc = failed to execute message; message index: 0: capabilities mismatch [cosmos/cosmos-sdk@v0.45.16/baseapp/baseapp.go:781] With gas wanted: '0' and gas used: '62826' : unknown request"
I[2023-11-20|18:03:26.422] shutting down                                module=bidengine-order cmp=provider order=akash1h24fljt7p0nh82cq0za0uhsct3sfwsfu9w3c9h/13768245/1/1
D[2023-11-20|18:03:26.422] unreserving reservation                      module=bidengine-order cmp=provider order=akash1h24fljt7p0nh82cq0za0uhsct3sfwsfu9w3c9h/13768245/1/1
D[2023-11-20|18:03:26.422] unreserving capacity                         module=provider-cluster cmp=provider cmp=service cmp=inventory-service order=akash1h24fljt7p0nh82cq0za0uhsct3sfwsfu9w3c9h/13768245/1/1
I[2023-11-20|18:03:26.422] attempting to removing reservation           module=provider-cluster cmp=provider cmp=service cmp=inventory-service order=akash1h24fljt7p0nh82cq0za0uhsct3sfwsfu9w3c9h/13768245/1/1
I[2023-11-20|18:03:26.422] removing reservation                         module=provider-cluster cmp=provider cmp=service cmp=inventory-service order=akash1h24fljt7p0nh82cq0za0uhsct3sfwsfu9w3c9h/13768245/1/1
I[2023-11-20|18:03:26.422] unreserve capacity complete                  module=provider-cluster cmp=provider cmp=service cmp=inventory-service order=akash1h24fljt7p0nh82cq0za0uhsct3sfwsfu9w3c9h/13768245/1/1
D[2023-11-20|18:03:26.422] reservation count                            module=provider-cluster cmp=provider cmp=service cmp=inventory-service cnt=0

SDL

version: '2.0'

services:
  foo:
    image: ubuntu:22.04
    expose:
      - port: 80
        as: 80
        to: 
          - global: true

  bar:
    image: ubuntu:22.04
    expose:
      - port: 80
        as: 80
        to:
          - global: true

profiles:
  compute:
    foo:
      resources:
        cpu:
          units: 1
        memory:
          size: 1Gi
        storage:
          size: 1Gi

    bar:
      resources:
        cpu:
          units: 1
        memory:
          size: 1Gi
        storage:
          size: 1Gi

  placement:
    westcoast:
      pricing:
        foo:
          denom: uakt
          amount: 1000000
        bar:
          denom: uakt
          amount: 1000000

deployment:
  foo:
    westcoast:
      profile: foo
      count: 1
  bar:
    westcoast:
      profile: bar
      count: 2

[troian]

are those deployments created via cloudmos or cli?

[andy108369]

are those deployments created via cloudmos or cli?

CLI - provider-services 0.4.8 (binary from the GH releases page)

[brewsterdrinkwater]

November 21st, 2023:

This requires a network upgrade to Mainnet 9. Upgrade only affects node.

Proposal can be found here: https://www.mintscan.io/akash/proposals/238

[brewsterdrinkwater]

Post Akash Mainnet 9, this has been thoroughly tested, and has been solved.