jenkinsci / lockable-resources-plugin

Lock resources against concurrent use
https://plugins.jenkins.io/lockable-resources
MIT License
87 stars 185 forks source link

Any simple way to visualize the queue? #275

Closed guwenqing closed 1 year ago

guwenqing commented 2 years ago

Hi,

We have some highly competitive resources in our testing. We have been asked to do a queue estimation for the jobs that are waiting. Do we have a simple way to get the current queued list?

Thanks!

jimklimov commented 2 years ago

Hello, I think some of my farms would benefit from that too (knowing which resources are hotly contested and which are only used in bursts and then dormant, to distribute compute power better for shorter wall-clock times spent by jobs).

That said, I am not aware of any current easy to use methods; PRs welcome :) The LockableResourcesManager class tracks the link(s) between queueId's (Long numbers) and candidate resources that would satisfy that particular queue. This tracking can probably be inverted to report the resource instances waited-for and the length of the queue for each.

I think that data may need certain understanding, e.g. if resources A and B satisfy job J, and both are not available, they may get reported with a queue pressure of 1 each. Then one of them becomes available and re-assigned to that queued job, and both get a queue pressure of zero.

jimklimov commented 2 years ago

Also note that the $JENKINS_URL/lockable-resources page should have a table at the bottom listing the (persistently?) defined resources and the amount of items per requestable label available currently. I think this is part of the base features, but might be just in our private builds, gotta check. This table would be a good place to report the resources/labels that are not available currently and have a queue pressure on them.

In any case, especially for (persistent or ephemeral) resources requested by name, the main table itself is a good place to report the pressure for individual items.

Abhishekverma2002 commented 2 years ago

I want to work on this. Please tell me what I have to do and where to?

mPokornyETM commented 2 years ago

@Abhishekverma2002 are you still interested?

Vandit1604 commented 1 year ago

Hey I want to work on this issue. Is it currently assigned to someone? if not tell me what I have to do and where ?

mPokornyETM commented 1 year ago

@Vandit1604 Hi, nice to meet you. No this issue is not assigned at the moment. You can work on that as well.

Vandit1604 commented 1 year ago

@mPokornyETM hi i would love to work on this issue can you tell me what i have to do and where

mPokornyETM commented 1 year ago

if not tell me what I have to do and where ?

Has you any experiences with Jenkins?

Vandit1604 commented 1 year ago

Yes i have learned how to use jenkins so i could contribute to the project as i went through the README.md of this plugin its about using resources like printers and computers that can be used in builds but speaking of computers what would be the difference between computers used as resource and computers used as agent nodes

mPokornyETM commented 1 year ago

@mPokornyETM hi i would love to work on this issue can you tell me what i have to do and where

My proposal is very easy. Create new page history

There shall be one table like

Resource State Timestamp Reason
ResourceABC FREE 3. Jan 2022 Lock released on resource [ResourceABC-Debian] by <liknToJob/Run>
ResourceABC Locked 3. Jan 2022 Locked by <liknToJob/Run>
ResourceABC Waiting 2. Jan 2022 [ResourceABC] is locked by <liknToJob/Run>, waiting...
ResourceABC Trying to acquire lock 1. Jan 2022 Trying to acquire lock on [ResourceABC]

When you use data tables it will be fine. There are some features implemented from scratch like: filtering, ordering, max lines per page ... And you must not care about that.

To get all this history stuff, will be possible to use loggers (I hope) In that case you does not need to care about

I know it is not directly this issue, but when we combine it with #393 and we can close both.

mPokornyETM commented 1 year ago

Yes i have learned how to use jenkins so i could contribute to the project as i went through the README.md of this plugin its about using resources like printers and computers that can be used in builds but speaking of computers what would be the difference between computers used as resource and computers used as agent nodes

That the reason, why we need more documentation. Looks here https://github.com/jenkinsci/lockable-resources-plugin/pull/403/files#diff-97da7bf492a2ccae1ba961bfc01d8027619ebfa6392aa5d210b0124d61534585 It shall explain it to you. This plugin has nothing to do with Jenkins nodes as well. But it is often combined with Jenkins nodes. When you create 1 node with 10 executors, Jenkins will do there 10 parallel tasks as well. But when you need allow 10 executors only for one job ? In this case helps this plugin to you. You will create 1 resource (probably with the same name as the node) and lock it before you allocate the node

lock('my_node_1') {
  node('my_node_1') {
     // do our parallel tasks here
  }
}

So you can start our multi-branch project as well and there is no conflict, because Job1 allocate resource here lock('my_node_1') { and the next one job will wait, until the close is finished. Maybe I shall also describe this scenario in the documentation

Vandit1604 commented 1 year ago

I'm currently using this plugin to understand it more deeply and going through the link you provided. can you elaborate more on Waiting state of a resource and Trying to acquire lock state. Does waiting state means resource is not locked but more than one controllers are trying to lock it and Trying to acquire lock state is the state when only one controller is trying to lock the resource

mPokornyETM commented 1 year ago

lock() step has nothing to do with node() step. So to clean your mind, forgot nodes now. A example. You has one PLC where is running some simulator. You has many jobs, there shall communicate with this PLC. To eliminate multiple access there you can lock this plc with the step lock().

lock('PLC') {
  echo 'My PLC is locked, start the test now'
}

That means waiting means waiting on next free resource. Maybe this explain it better https://github.com/jenkinsci/lockable-resources-plugin/pull/403/files#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5:~:text=A%20resource%20is,and%20node%20labels.

Vandit1604 commented 1 year ago

@mPokornyETM i implemented the tabs from the bootstrap site but it's not working as tabs when i click them and i also tried creating new page but it gives Stapler request error 500 also can you give me some information about in which frameworkn of java this webapp is created so i can understand the file structure better i referred the jelly documentation but im not in clear after going through them can you suggest me some docs on how to add a new page and adding link