search

leeej84 / -DEPRECATED---PowerScale

Replacement for Citrix SmartScale
GNU General Public License v3.0
16 stars 11 forks source link

Machines being left in maintenance mode until the following morning #5

Closed leeej84 closed 4 years ago

leeej84 commented 4 years ago

There is an issue where machines are being left in maintenance mode until the following morning. This means machines are not being put back into service until the exact time that the business starts.

leeej84 commented 4 years ago

Log file and config file stored in OneDrive for review. Review will take place next week.

Wausert commented 4 years ago

Hi Lee,

I have the same problem. I set the script to draining not forcefully logoff users. The script correctly sets the maintenance mode on the servers but after that it "forgets" the servers need to be shutdown when all users have logged off. It seems the script cannot see machines in maintenance mode. Also the servers are not placed in production anymore the next day.

PS: Great work!!!

leeej84 commented 4 years ago

Apologies I've not got back to you on this, I've actually not been a position to proceed with any development or bug fixes, just to confirm; you got the latest branch? If so I really need to go on this. What should happen is that the machine is shutdown and on the way out of the shutdown process it should be taken out of maintenance mode.

leeej84 commented 4 years ago

@Wausert - I've spent some time trying to reproduce your problem and I'm afraid I'm not seeing it. Are you able to send me your config file and log and highlight the period when this is happening?

It doesn't matter if I am forcing or not forcing users off, either way it seems to work okay for me. Once the machines are powered off they are taken out of maintenance mode.

Something like this in the logs: VERBOSE: Setting XDSHNP-03.ctxlab.local maintenance mode On VERBOSE: Found 0 user sessions on XDSHNP-03.ctxlab.local VERBOSE: No sessions found on XDSHNP-03.ctxlab.local, shutting down VERBOSE: Machine action for ctxlab\XDSHNP-03 - Shutdown immediately VERBOSE: Setting XDSHNP-02.ctxlab.local maintenance mode On VERBOSE: Found 0 user sessions on XDSHNP-02.ctxlab.local VERBOSE: No sessions found on XDSHNP-02.ctxlab.local, shutting down VERBOSE: Machine action for ctxlab\XDSHNP-02 - Shutdown immediately VERBOSE: Setting XDSHNP-03.ctxlab.local maintenance mode Off VERBOSE: Setting XDSHNP-02.ctxlab.local maintenance mode Off

leeej84 commented 4 years ago

@Wausert - Just as i typed that, I found a small typo, although I cannot reproduce your issue doesn't mean its not there. I've just uploaded the typo change, if you just drop the most up to date Decision Making.ps1 script over the top of your current one you should be okay.

sgoldsmith71 commented 4 years ago

Hi Lee I was actually getting the same issue with this release so I rolled back to the previous version. I don’t know if it makes a difference but we are using session based scaling Steve

Sent from Outlookhttps://aka.ms/qtex0l on iPhone

From: Leee Jeffries notifications@github.com Sent: Saturday, January 25, 2020 8:13:13 AM To: leeej84/PowerScale PowerScale@noreply.github.com Cc: Subscribed subscribed@noreply.github.com Subject: Re: [leeej84/PowerScale] Machines being left in maintenance mode until the following morning (#5)

@Wauserthttps://github.com/Wausert - I've spent some time trying to reproduce your problem and I'm afraid I'm not seeing it. Are you able to send me your config file and log and highlight the period when this is happening?

It doesn't matter if I am forcing or not forcing users off, either way it seems to work okay for me. Once the machines are powered off they are taken out of maintenance mode.

Something like this in the logs: VERBOSE: Setting XDSHNP-03.ctxlab.local maintenance mode On VERBOSE: Found 0 user sessions on XDSHNP-03.ctxlab.local VERBOSE: No sessions found on XDSHNP-03.ctxlab.local, shutting down VERBOSE: Machine action for ctxlab\XDSHNP-03 - Shutdown immediately VERBOSE: Setting XDSHNP-02.ctxlab.local maintenance mode On VERBOSE: Found 0 user sessions on XDSHNP-02.ctxlab.local VERBOSE: No sessions found on XDSHNP-02.ctxlab.local, shutting down VERBOSE: Machine action for ctxlab\XDSHNP-02 - Shutdown immediately VERBOSE: Setting XDSHNP-03.ctxlab.local maintenance mode Off VERBOSE: Setting XDSHNP-02.ctxlab.local maintenance mode Off

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/leeej84/PowerScale/issues/5?email_source=notifications&email_token=AMIWFBXVM5CSYVQECIT4W43Q7NKWHA5CNFSM4JNRPW2KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJ4DSLY#issuecomment-578304303, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AMIWFBUYCQXKASCSLW5ZJMDQ7NKWHANCNFSM4JNRPW2A.

Disclaimer

The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful.

This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast Ltd.

leeej84 commented 4 years ago

Thanks Steve. That shouldn't make a difference no. That being said I'd need to see log files and config to be 100 sure. I don't suppose you could test the latest Steve?

Kind Regards

Leee Jeffries

From: sgoldsmith71 notifications@github.com Sent: Friday, January 24, 2020 9:49:20 PM To: leeej84/PowerScale PowerScale@noreply.github.com Cc: Leee Jeffries leee.jeffries@leeejeffries.com; Author author@noreply.github.com Subject: Re: [leeej84/PowerScale] Machines being left in maintenance mode until the following morning (#5)

Hi Lee I was actually getting the same issue with this release so I rolled back to the previous version. I don’t know if it makes a difference but we are using session based scaling Steve

Sent from Outlookhttps://aka.ms/qtex0l on iPhone

From: Leee Jeffries notifications@github.com Sent: Saturday, January 25, 2020 8:13:13 AM To: leeej84/PowerScale PowerScale@noreply.github.com Cc: Subscribed subscribed@noreply.github.com Subject: Re: [leeej84/PowerScale] Machines being left in maintenance mode until the following morning (#5)

@Wauserthttps://github.com/Wausert - I've spent some time trying to reproduce your problem and I'm afraid I'm not seeing it. Are you able to send me your config file and log and highlight the period when this is happening?

It doesn't matter if I am forcing or not forcing users off, either way it seems to work okay for me. Once the machines are powered off they are taken out of maintenance mode.

Something like this in the logs: VERBOSE: Setting XDSHNP-03.ctxlab.local maintenance mode On VERBOSE: Found 0 user sessions on XDSHNP-03.ctxlab.local VERBOSE: No sessions found on XDSHNP-03.ctxlab.local, shutting down VERBOSE: Machine action for ctxlab\XDSHNP-03 - Shutdown immediately VERBOSE: Setting XDSHNP-02.ctxlab.local maintenance mode On VERBOSE: Found 0 user sessions on XDSHNP-02.ctxlab.local VERBOSE: No sessions found on XDSHNP-02.ctxlab.local, shutting down VERBOSE: Machine action for ctxlab\XDSHNP-02 - Shutdown immediately VERBOSE: Setting XDSHNP-03.ctxlab.local maintenance mode Off VERBOSE: Setting XDSHNP-02.ctxlab.local maintenance mode Off

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/leeej84/PowerScale/issues/5?email_source=notifications&email_token=AMIWFBXVM5CSYVQECIT4W43Q7NKWHA5CNFSM4JNRPW2KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJ4DSLY#issuecomment-578304303, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AMIWFBUYCQXKASCSLW5ZJMDQ7NKWHANCNFSM4JNRPW2A.

Disclaimer

The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful.

This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast Ltd.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/leeej84/PowerScale/issues/5?email_source=notifications&email_token=AIAFOEF35CFXDSGUOSJNODLQ7NO6BA5CNFSM4JNRPW2KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJ4GIRA#issuecomment-578315332, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIAFOEG3ZWCZWCV7WGIKQATQ7NO6BANCNFSM4JNRPW2A.

sgoldsmith71 commented 4 years ago

No worries Lee, I am away for a couple of days but can get onto it later in the weekend Steve

Sent from Outlookhttps://aka.ms/qtex0l on iPhone

From: Leee Jeffries notifications@github.com Sent: Saturday, January 25, 2020 8:52:50 AM To: leeej84/PowerScale PowerScale@noreply.github.com Cc: Stephen Goldsmith SGoldsmith@CarterNewell.com; Comment comment@noreply.github.com Subject: Re: [leeej84/PowerScale] Machines being left in maintenance mode until the following morning (#5)

Thanks Steve. That shouldn't make a difference no. That being said I'd need to see log files and config to be 100 sure. I don't suppose you could test the latest Steve?

Kind Regards

Leee Jeffries

From: sgoldsmith71 notifications@github.com Sent: Friday, January 24, 2020 9:49:20 PM To: leeej84/PowerScale PowerScale@noreply.github.com Cc: Leee Jeffries leee.jeffries@leeejeffries.com; Author author@noreply.github.com Subject: Re: [leeej84/PowerScale] Machines being left in maintenance mode until the following morning (#5)

Hi Lee I was actually getting the same issue with this release so I rolled back to the previous version. I don’t know if it makes a difference but we are using session based scaling Steve

Sent from Outlookhttps://aka.ms/qtex0l<https://aka.ms/qtex0l> on iPhone

From: Leee Jeffries notifications@github.com Sent: Saturday, January 25, 2020 8:13:13 AM To: leeej84/PowerScale PowerScale@noreply.github.com Cc: Subscribed subscribed@noreply.github.com Subject: Re: [leeej84/PowerScale] Machines being left in maintenance mode until the following morning (#5)

@Wauserthttps://github.com/Wausert<https://github.com/Wausert> - I've spent some time trying to reproduce your problem and I'm afraid I'm not seeing it. Are you able to send me your config file and log and highlight the period when this is happening?

It doesn't matter if I am forcing or not forcing users off, either way it seems to work okay for me. Once the machines are powered off they are taken out of maintenance mode.

Something like this in the logs: VERBOSE: Setting XDSHNP-03.ctxlab.local maintenance mode On VERBOSE: Found 0 user sessions on XDSHNP-03.ctxlab.local VERBOSE: No sessions found on XDSHNP-03.ctxlab.local, shutting down VERBOSE: Machine action for ctxlab\XDSHNP-03 - Shutdown immediately VERBOSE: Setting XDSHNP-02.ctxlab.local maintenance mode On VERBOSE: Found 0 user sessions on XDSHNP-02.ctxlab.local VERBOSE: No sessions found on XDSHNP-02.ctxlab.local, shutting down VERBOSE: Machine action for ctxlab\XDSHNP-02 - Shutdown immediately VERBOSE: Setting XDSHNP-03.ctxlab.local maintenance mode Off VERBOSE: Setting XDSHNP-02.ctxlab.local maintenance mode Off

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/leeej84/PowerScale/issues/5?email_source=notifications&email_token=AMIWFBXVM5CSYVQECIT4W43Q7NKWHA5CNFSM4JNRPW2KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJ4DSLY#issuecomment-578304303<https://github.com/leeej84/PowerScale/issues/5?email_source=notifications&email_token=AMIWFBXVM5CSYVQECIT4W43Q7NKWHA5CNFSM4JNRPW2KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJ4DSLY#issuecomment-578304303>, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AMIWFBUYCQXKASCSLW5ZJMDQ7NKWHANCNFSM4JNRPW2A<https://github.com/notifications/unsubscribe-auth/AMIWFBUYCQXKASCSLW5ZJMDQ7NKWHANCNFSM4JNRPW2A>.

Disclaimer

The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful.

This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast Ltd.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/leeej84/PowerScale/issues/5?email_source=notifications&email_token=AIAFOEF35CFXDSGUOSJNODLQ7NO6BA5CNFSM4JNRPW2KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJ4GIRA#issuecomment-578315332<https://github.com/leeej84/PowerScale/issues/5?email_source=notifications&email_token=AIAFOEF35CFXDSGUOSJNODLQ7NO6BA5CNFSM4JNRPW2KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJ4GIRA#issuecomment-578315332>, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIAFOEG3ZWCZWCV7WGIKQATQ7NO6BANCNFSM4JNRPW2A<https://github.com/notifications/unsubscribe-auth/AIAFOEG3ZWCZWCV7WGIKQATQ7NO6BANCNFSM4JNRPW2A>.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/leeej84/PowerScale/issues/5?email_source=notifications&email_token=AMIWFBVVDYX35GUNKW4IGATQ7NPLFA5CNFSM4JNRPW2KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJ4GQIQ#issuecomment-578316322, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AMIWFBROW2DJCYUQDSJ7OCTQ7NPLFANCNFSM4JNRPW2A.

Disclaimer

The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful.

This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast Ltd.

leeej84 commented 4 years ago

Nice one Steve, thanks man.

Wausert commented 4 years ago

@Wausert - I've spent some time trying to reproduce your problem and I'm afraid I'm not seeing it. Are you able to send me your config file and log and highlight the period when this is happening?

It doesn't matter if I am forcing or not forcing users off, either way it seems to work okay for me. Once the machines are powered off they are taken out of maintenance mode.

Something like this in the logs: VERBOSE: Setting XDSHNP-03.ctxlab.local maintenance mode On VERBOSE: Found 0 user sessions on XDSHNP-03.ctxlab.local VERBOSE: No sessions found on XDSHNP-03.ctxlab.local, shutting down VERBOSE: Machine action for ctxlab\XDSHNP-03 - Shutdown immediately VERBOSE: Setting XDSHNP-02.ctxlab.local maintenance mode On VERBOSE: Found 0 user sessions on XDSHNP-02.ctxlab.local VERBOSE: No sessions found on XDSHNP-02.ctxlab.local, shutting down VERBOSE: Machine action for ctxlab\XDSHNP-02 - Shutdown immediately VERBOSE: Setting XDSHNP-03.ctxlab.local maintenance mode Off VERBOSE: Setting XDSHNP-02.ctxlab.local maintenance mode Off

Hi Lee,

Sorry for the delay, I've been away... Thanks for the time to try to figure this out! I will send you my config file. Unfortunately the logs are already rotated.... Can create new ones if required.

Wausert

leeej84 commented 4 years ago

Working on this fix guys, I understand the issue. I now have a working fix for bringing machines out of maintenance mode, unfortunately machines in maintenance aren't actually considered in the general performance or logic, I essentially need to rework a fair amount I think. Its okay, I am thinking on it now. I'll fix it.

anwaraims commented 4 years ago

Hi Lee, I am currently testing the powerscale functionality in dev environment and keep getting below message saying I have 0 machines powered on and 0 machine powered off which is not the case as I definitely have machines that running, something that I am not doing right here?

VERBOSE: - VERBOSE: #######PowerScale script starting - Test mode value is True####### VERBOSE: Start Circular Log Management VERBOSE: It is currently outside working hours - performing machine analysis VERBOSE: The current number of powered on machines is 0 and the target is 2 - resulting action is to Startup 2 machines VERBOSE: It is currently inside working hours, machines are required to be started VERBOSE: There are 0 machine(s) currently switched on and registered, There are 0 machine(s) in maintenance mode and there are 0 machine(s) powered off VERBOSE: In total there are 0 machine(s) able to be placed into service. VERBOSE: The number of machines available is 0 and the number required is 2 WARNING: There are not enough machines available to service the request, working on the machines we can 0 1 2 3 4 5 6 7 8 VERBOSE: #######PowerScale script finishing####### VERBOSE: -

Thanks

leeej84 commented 4 years ago

@anwaraims - In your config file make sure you have the correct detection method selected and that the name is correct. Send me your config file and screenshot of your machine catalogs and delivery groups if you are in doubt.

anwaraims commented 4 years ago

Thanks for your quick reply Lee please see attached (I needed to rename CreatConfig.ps1 to .txt as upload wont support .ps1 file)

Create.Config.File.txt Delivery Group01 Delivery Group02 Machine Catalog

leeej84 commented 4 years ago

Okay, your issue is here:

$machineDetection = "WEUCCDCTXD"

Change this line to:

$machineDetection = "prefix"

FYI, you may want to raise a seperate issue, just because this one is for a seperate issue.

anwaraims commented 4 years ago

Thanks changing it to "prefix" has worked.

I will raise a separate issue for any further problem I may get once again many thanks :)

leeej84 commented 4 years ago

Hi All, download the latest release, this is now fixed. I wont close this yet, I'll wait for testing feedback. No rush.

anwaraims commented 4 years ago

Thanks Lee great work!!!

Wausert commented 4 years ago

Hi Leee, Thanks! However, I get a Powershell error with this version:

PS C:\PowerScale> & '.\Decision Making.ps1' C:\PowerScale Exception calling "ParseExact" with "3" argument(s): "String was not recognized as a valid DateTime." At C:\PowerScale\Decision Making.ps1:118 char:74

VERBOSE: - VERBOSE: #######PowerScale script starting - Test mode value is True#######

and this one:

VERBOSE: 30 - Overall Session Index Average VERBOSE: 0.75 - Overall Session Count Average VERBOSE: Start Circular Log Management IsWeekDay : Cannot process argument transformation on parameter 'date'. Cannot convert null to type "System.DateTime". At C:\PowerScale\Decision Making.ps1:1559 char:23

VERBOSE: The current number of powered on machines is 4 and the target is 1 - resulting action is to Shutdown 3 machines

leeej84 commented 4 years ago

Hi All, apologies. I removed the maintenance release parameter from the whole script except where the time object is generated, the script errored here and from then on did not know what time it was. Fixed in a new 2.4 release.

https://github.com/leeej84/PowerScale/releases/tag/2.4

Drop these over what you have and you'll be up and running with the latest fixes.

What do you guys think about adding a cost saving value in the graphing? So measuring how many machines are switched off and then overall how much you are saving at each script run? I'll then create a total for the entire period on the right?

Wausert commented 4 years ago

Hi All, apologies. I removed the maintenance release parameter from the whole script except where the time object is generated, the script errored here and from then on did not know what time it was. Fixed in a new 2.4 release.

https://github.com/leeej84/PowerScale/releases/tag/2.4

Drop these over what you have and you'll be up and running with the latest fixes.

What do you guys think about adding a cost saving value in the graphing? So measuring how many machines are switched off and then overall how much you are saving at each script run? I'll then create a total for the entire period on the right?

I can confirm this fixes the powershell script error. Great work. Cost saving graph is nice to have. If you can get an estimate saving per day/month/week makes customers happy :)