despiegk commented 2 years ago

STORY: dedicated nodes

support deployment of dedicated nodes, different pricing

specs to be completed

remarks

will be implemented first using #1155 (not sure yet)

Story issues

xmonader commented 2 years ago

@muhamadazmy please provide the specs for this story

muhamadazmy commented 2 years ago

Needed better definition of a dedicated node. From what I understand from https://github.com/threefoldtech/home/issues/1155 it means a user can reserve an entire node then use it exclusively to deploy solutions for other customers. If that is the case we can implement this as follows:

EDIT: Once a node is dedicated, there is a fixed charge billed to the tenant regardless of deployed workloads

A new tfgrid contract type can be added (RentContract)
When a user create a RentContract against a node, the grid validate that there are no other active contracts on that node on the creation.
Once a RentContract is created, the grid can only accept contracts on this node from the owner.
The node then changes it's behavior according as follows:
- Only deployments contracts from the same twin as the RentContract are accepted (already the grid will do the same check)

Capacity reporting behavior changes as follows:

Node send reports to RentContract.
- The idea is that the RentContract will ignore most reports except (maybe) for the network consumption. It's up to the RentContract to decide what to include in the bill, this can be fixed rent value, percent of the reported consumed capacity, and networking consumption
- Notice that deployment still has to be deployed via a NodeContract (what we have now for deployments) but they won't receive reports anymore. The are still needed for the following:
- Billing for reserved public IPs (these are not part of the rent contract and are per deplolyment)
- Cancellation of a deployment by cancelling the deployment contract.
- Consistency with current system

Required changes

TFGrid (assuming we using the 2nd suggestion)

Add RentContract type. Provide extrinsic for creation, cancellation and consumption reporting
How to configure promotions, is it set per farm or a global policy via TF ?
Rent contract should also pay the farmer for the (rent) even without having any workload, since when the node is reserved it can't be used by other users
consumption reported to the RentContract. The contract can then add the network consumption to the bill

ZOS

If the node receives a grid event that it has been rented. It needs to does the following:
- Never accept deployments from anyone but the renter (owner of the RentContract)
- On RentContract cancellation the node revert to normal behavior
- When an active rent contract in place, the node reports capacity to the RentContract, not deployments

Note, a NodeContract (deployment contract) will still create a billing statement for PublicIPs reservations since these are not included in the consumption reports anyway.

sasha-astiadi commented 2 years ago

@despiegk need to approve the spec to unblock this

DylanVerstraete commented 2 years ago

As Azmy proposes, a seperate contract type (RentContract) sounds like a very good idea.

Using a RentContract we can identity which nodes are being used as a Dedicated Node and easily plug that into the current billing module. Any subsequent NodeContract deployed on a node where a rentContract is active (and the same user is creating the nodeContracts) can be excluded from billing (apart from public ip and network usage).

Without the definition of a rent contract it's impossible to bill the user for reserving the entire node.

Proposed by Kristof in a call this morning:

"have a field on the node object that can mark the node as being dedicated by a user, from that moment on, the user's deployments on that node are free"

Issues with this approach

We cannot bill the user every X hours for marking the node as dedicated because there is no notion of "contract". The billing module on chain relies heavily on having a contract in a "billing loop" (this being either a nodeContract or nameContract).
When a user creates a nodeContract on his dedicated node, a lot of if-checks will need to be added in the current billing module code.

sasha-astiadi commented 2 years ago

@muhamadazmy please comment here why blocked

despiegk commented 2 years ago

ok for me

On Fri, Feb 25, 2022 at 7:46 PM sasha-astiadi @.***> wrote:

@muhamadazmy https://github.com/muhamadazmy please comment here why blocked

— Reply to this email directly, view it on GitHub https://github.com/threefoldtech/home/issues/1156#issuecomment-1050967254, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABN6FVC2JRVLXYS4OSW6FGLU46P4RANCNFSM5NWL2G2A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you were mentioned.Message ID: @.***>

--

Kristof De Spiegeleer +971525609014 +201206927877 +32 475405474 skype: despiegk

xmonader commented 2 years ago

Given this was pending approval, and Azmy is off this week: it won't meet this feb deadline

sasha-astiadi commented 2 years ago

@muhamadazmy please make sure the documentation is provided

sasha-astiadi commented 2 years ago

@DylanVerstraete please add todos on this issue thank you

xmonader commented 2 years ago

One major problem I can see, is people being able to provision small workloads e.g -redis vm- on the nodes preventing them from being rented out. So, I can put a very high spec machine meant to be rented out, but yet I still don't have any way to mark it to be used only as dedicated. The other problem, if we allowed people to flag their nodes as dedicated (most likely everyone will do so)

Suggestion for going forward and yet still support marking nodes:

As a measure not to protect the grid: the DAO can change the state of all dedicated nodes to free in case e.g 100% of the available nodes on the grid are dedicated
We can allow farmers only to mark up to 50% of their nodes as dedicated

delandtj commented 2 years ago

The other problem, if we allowed people to flag their nodes as dedicated (most likely everyone will do so) O_o why would you think that everyone will do so ? Dedicated != in use. so setting it as dedicated, doesn't mean that it is in use. We still need to see what 'in use' would mean, but these 2 terms need to be disconnected. A Farmer would only set a node to 'dedicated' if in some way or another he has some arrangement with a user to allocate a node for a certain usecase. Either way, if a farmer sets it to 'dedicated' he would do so before any workloads would be installed.

DylanVerstraete commented 2 years ago

A Farmer would only set a node to 'dedicated' if in some way or another he has some arrangement with a user to allocate a node for a certain usecase.

How would you define this arrangement? Also, if any farmer can mark their node as deprecated they can benefit from the fact that nobody can even use their node unless somebody actually rents their dedicated node.

Honestly, there is no elegant solution to this problem. If we allow everyone to mark their node as dedicated we will have no grid for users who just want to deploy small things instead of always renting a full machine.

On the other hand if we let every node to be "fully rented" then someone can block a node from being rented by deploying a small thing on that node. But what is the actual chance of this happening?

DylanVerstraete commented 2 years ago

Also, what does it matter if there is running small deployments on a node? cpu and mem resources are shared anyway?

muhamadazmy commented 2 years ago

@delandtj they will mark it as dedicated because they are rewarded for providing the capacity, not for it to be used. Which means if nobody uses it i will save on power and network. Hence it's better for me if i provide a lot of capacity that is not used.

LeeSmet commented 2 years ago

Giving farmers the option to mark nodes as dedicated is a bad idea imo. The issue is that there is no capacity planning whatsoever (on the farmer side), and this is supposedly by design. So we can't suddenly add a feature which starts doing that, as that is a horrible hack which will blow up in our face. Adding such would mean we need to think about capacity planning from the farmer side as a whole. For instance, farmers will want to differentiate between compute nodes and storage nodes, and only allow certain workloads to run on one but not the other.

Honestly I don't even think there is a problem. If we have a high spec machine and someone rents a small vm on that, that means you are left with a high spec machine which still has a lot of available resources, and anyone can still deploy stuff on it. To be frank, I don't even see the point of renting a whole node at this point. The main things would be overprovisioned cpu cores and shared IO access. But that is something we can solve with cpu pinning and I/O bandwidth limiting rather easily I think. Also, renting a node incentivizes a user to put all his workloads on a single node, while it makes more sense to put your workloads on different nodes (in the same farm), to have some resilience.

rkhamis commented 2 years ago

I think both sides of the argument are very valid. Simply making the concept of dedicated nodes available with no control could indeed result on small workloads deployed on a high spec machine, rendering it unrentable as a dedicated node. However, allowing farmers to mark their nodes as dedicated-able could end up with the grid having all/most of its nodes marked and with nowhere to deploy non-dedicated workloads.

So in interest of keeping the grid healthy and providing dedicated nodes to users whose usecases prefer a dedicated node, how about we allow farmers to only mark up to a specific percentage (for instance 50%) of their nodes as dedicated? that way, we insure that the grid will always have dedicated and none-dedicated nodes?

DylanVerstraete commented 2 years ago

how about we allow farmers to only mark up to a specific percentage (for instance 50%) of their nodes as dedicated? that way, we insure that the grid will always have dedicated and none-dedicated nodes?

This is not possible to do on tfchain. There is no storage that can be queried that returns all the nodes for a specific farmer. If this needs to be implemented then tfchain needs to loop over the entire nodes storage map and check if the number of nodes that the farmer already marked exceeds some sort of limit. I would strongly advise to not do this.

What if we for example allow nodes to be fully rented when their used capacity is below 10%? So if there is something small running there we can ignore that?

sasha-astiadi commented 2 years ago

@despiegk please give a feedback

rkhamis commented 2 years ago

After discussions with the engineers, the team has agreed that not marking nodes is the way to go for these reasons:

having no capacity planning on farmers’ side. Marking breaks the design and introduces a lot of impact on the grid
concerns about none-dedicated nodes availability if all/most farmers start marking their nodes
discussions with @muhamadazmy on possibly excluding network workloads from marking a node as unavailable for renting

rkhamis commented 2 years ago

Per our last discussion with @despiegk, the council will be marking some farms as dedicated. Logged in users in the admin portal will be able to see a list of all available nodes for renting and would be able though the portal to reserve/unreserve the nodes.

Issue for the portal is here: https://github.com/threefoldtech/tfchain_portal/issues/58

xmonader commented 2 years ago

despiegk commented 2 years ago

pricing is wrong please fix

its testnet so even more wrong

check the calculator

K.

On Mon, Apr 18, 2022 at 2:22 PM xmonader @.***> wrote:

[image: image] https://user-images.githubusercontent.com/64129/163795052-00ef0827-aa89-498b-bcb2-ce5baf34fba0.png

— Reply to this email directly, view it on GitHub https://github.com/threefoldtech/home/issues/1156#issuecomment-1101292957, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABN6FVB2HJ6W6YY63IUL6B3VFUZVVANCNFSM5NWL2G2A . You are receiving this because you were mentioned.Message ID: @.***>

--

Kristof De Spiegeleer +971525609014 +201206927877 +32 475405474 skype: despiegk

threefoldtech / home