Enabling easier API wide password management

[ju2wheels]

Hi all, im working on a tool to support internal IBM ITCS104 security requirements for changing our passwords every 90 days and have a need to locate and change passwords within SoftLayer to update them to match the servers.

Opening this for discussion/thoughts as there is a lot of supporting stuff that would be needed.

I plan on adding the actual tool into vagrant-softlayer as a standalone tool since there isnt an existing cli tool one in this project but hope it may serve as a reference for enhancements to the softlayer-python cli.

Changes/Additions needed to enable mass password updates for various types with a select everything and filter down approach:

Account
    sl_dynamic_attr evaultMasterUsers
      - self.service.getEvaultMasterUsers -> AccountPassword
    sl_dynamic_attr users
      - self.service.getUsers -> [ UserCustomer ]
    sl_dynamic_attr networkMessageDeliveryAccounts
      - self.service.getNetworkMessageDeliveryAccounts -> [ NetworkMessageDelivery ]
    sl_dynamic_attr virtual_disk_images
      - self.service.getVirtualDiskImages -> [ VirtualDiskImage ]

AccountPassword
    sl_attr local_props*
    self.find_password_for_network_storage(sl_client, net_storage_obj_filter => nil)
      - SoftLayer_Account::getNetworkStorage -> SoftLayer_Network_Storage::getAccountPassword -> [ NetworkStorage ]
    self.find_webcc_password_for_network_storage(sl_client, net_storage_obj_filter => nil)
      - SoftLayer_Account::getNetworkStorage -> SoftLayer_Network_Storage::getWebccAccount -> [ NetworkStorage ]
    def password=(password)
      - self.service.editObject password

BareMetalServer
    sl_dynamic_attr softwareComponents
      - self.service.getSoftwareComponents -> [ SoftwareComponent ]
    sl_dynamic_attr remoteManagementAccounts
      - self.service.getRemoteManageAccounts -> [ Hash ]

NetworkMessageDelivery
    sl_attr local_props*
    def password=(password)
      - self.service.editObject password

NetworkServiceResource
    sl_attr local_props*
    - Is apiPassword in this editable?

NetworkStorage
    sl_attr local_props*
    sl_dynamic_attr accountPassword
       - self.service.getAccountPassword -> AccountPassword
       - whats the difference between NetworkStorage.password and NetworkStorage.accountPassword other than type?
    sl_dynamic_attr credentials
       - self.service.getCredentials -> [ NetworkStorageCredential ]
    sl_dynamic_attr serviceResource
       - self.service.getServiceResource -> [ NetworkServiceResource ]
    sl_dynamic_attr webccAccount
       - self.service.getWebccAccount -> AccountPassword
    def password=(password)
       - This one updates self.password using self.service.editObject, not the credential
    def update_credential_password(username, password)
       - self.service.editCredential(username,password)

NetworkStorageCredential
    sl_attr local_props*
    self.find_network_storage_credentials(sl_client, :network_storage_obj_filter => nil)
       - SoftLayer_Account::getNetworkStorage -> SoftLayer_Network_Storage::getCredentials -> [ NetworkStorageCredential ]

Server
    sl_dynamic_attr softwareComponents
      - self.service.getSoftwareComponents -> [ SoftwareComponent ]

SoftwareComponent
    sl_attr local_props*
    self.find_software_on_servers(sl_client, :hw_obj_filter => nil, software_comp_obj_filter => nil)
       - SoftLayer_Account::getHardware -> SoftLayer_Hardware::getSoftwareComponents -> [ SoftwareComponent ]
    self.find_software_on_network_hardware(sl_client, :hw_obj_filter => nil, software_comp_obj_filter => nil)
       - SoftLayer_Account::getNetworkHardware -> SoftLayer_Hardware::getSoftwareComponents -> [ SoftwareComponent ]
    self.find_software_on_virtual_guests(sl_client, :vguest_obj_filter => nil, software_comp_obj_filter => nil)
       - SoftLayer_Account::getVirtualGuests -> SoftLayer_Virtual_Guest::getSoftwareComponents -> [ SoftwareComponent ]
    sl_dynamic_attr passwords
       - self.service.getPasswords -> [ SoftwareComponentPassword ]
    def has_user_password?(user)
       - self.passwords.map{|pw| pw.username}.include?(user)

SoftLayerSoftwareComponentPassword
    sl_attr local_props*
    self.find_password_for_applicationDeliveryControllers(sl_client, :app_dev_cont_obj_filter=>@@app_dev_cont_obj_filter)
      - SoftLayer_Account::getApplicationDeliveryControllers -> SoftLayer_Network_Application_Delivery_Controller::getPassword -> [ SoftwareComponentPassword ]
      - Use object filter to allow filtering app dev controller by dc or other properties
      - as with nw fw cred, if this is returned by one of network hardware SoftwareComponent then we just need to filter that instead
    self.find_password_for_networkVLANFirewallCredentials(sl_client, :hw_fw_obj_filter=>nil, :net_vlan_obj_filter=>nil)
      - SoftLayer_Account::getNetworkHardware -> SoftLayer_Hardware::getNetworkVlans -> SoftLayer_Network_Vlan::getNetworkVlanFirewall -> SoftLayer_Network_Vlan_Firewall::getManagementCredentials -> [ SoftwareComponentPassword ]
      - Use object filters to filter by dc/fqdn of hw_firewall and vlan fqdn
      - may need to filter on dedicatedFirewallFlag in Network_Vlan or filter the hardware by function and get fw hw only to avoid dup entries?
      - not clear if this one is even necessary or correct, it may be part of the SoftwareComponent pw's returned by SoftwareComponent.software_on_network_hardware and we just have to filter that instead
    def password=(password)
      - self.service.editObject password
    @@app_dev_cont_object_filter = advancedModeFlag == true

UserCustomer
    sl_attr local_props*
    sl_dynamic_attr externalBindings
      - self.service.getExternalBindings -> [ UserCustomerExternalBinding ]

UserCustomerExternalBinding
    sl_attr local_props*
    def password=(password)
      - self.service.editObject password #This does not exist, is it read only? SoftLayer_User_Customer only has add/remove binding but expects external binding object (is that hash?)...
      - If this is read only probably will not add this or UserCustomer at all

VirtualDiskImage
    sl_attr local_props*
    sl_dynamic_attr softwareReferences
      - self.service.getSoftwareReferences -> [ VirtualDiskImageSoftware ]

VirtualDiskImageSoftware
    sl_attr local_props*
    slattr passwords
      - self.passwords
    sl_attr diskImage
      - self.diskImage
    has_user_password?(user)
      - self.passwords.map{|pw| pw.username}.include?(user)
    def update_user_password(username, password)
      - softlayer_service[:VirtualDiskImage].object_with_id(diskImage.id).editObject self.id, username, password

Tentative example CLI interface the above would support:

sl pw-man user <portal|forum|vpn> <username> <password>
sl pw-man user reset <username> <password> <newPassword> [<securityQuestionId|securityQuestionString> <securityQuestionAnswer>]
sl pw-man user lost <username> <useremail>
sl pw-man user update_lost <username> <key> <password> [<securityAnswers>]
sl pw-man user list-sec-quest <username>
sl pw-man user list-lost-sec-quest <username> <key>
sl pw-man user set-sec-quest <?>

• sl pw-man software [--add-user] [sw_filter] [server_filter] [username] password

  if no user provided set for all present

--add-user: Adds user if it does not exist, skips pw update otherswise if user provided and not found

server_filter: --servers_only --virtual_guest_only --network_hardware_only --dc DCNAME,... --hostname HOSTNAME,... --domain DOMAIN,...

sw_filter: #Filter on properties of SoftLayer_Software_Description::getAllObjects --software-id id,... --software-description descr --software-name name,...

• sl pw-man software --list #id,longDescription,name,requiredUser of SoftLayer_Software_Description
• sl pw-man net-app-dev-controller [net_app_dev_cont_filter] password

net_app_dev_cont_filter: --net-app-dev-cont-id id,... --net-app-dev-cont-name name,...

• sl pw-man net-vlan-fw-cred [hw_fw_filter] [net_vlan_filter] password hw_fw_filter: --fw-dc DCNAME,... --fw-hostname HOSTNAME,... --fw-domain DOMAIN,...

net_vlan_filter: unknown what to filter on if its even necessary

• sl pw-man net-vlan-fw-cred --list #hw_fw id,domain,hostname, vlan_fw cred sl pw-man remote-user [--add-user] [hw_server_filter] username password hw_server_filter: --dc DCNAME,... --hostname HOSTNAME,... --domain DOMAIN,...
• sl pw-man network-message-delivery username password
• sl pw-man network-storage [server_filter] [net_storage_filter] password
• sl pw-man network-storage-cred [server_filter] [net_storage_filter] username password

[SLsthompson]

Perhaps we should get on a group chat/phone call with the Python guys and all get on the same page. Would you like to do that?

[SLsthompson]

Since there is so much here... it might be wise to break the problem down into chunks that can be separate issues (and implemented separately… with this one being the "master issue").

[ju2wheels]

Hi Scott, sure I can join a chat or group call. Im not sure what the other contacts are or if you have a internal ST group so Ill just drop you an email so we can sort out scheduling.

Ill break these out into a few more issues, ill be starting with the SoftwareComponent password stuff first as this will give us the largest up front benefit.

[ju2wheels]

Ref branch ju2wheels/softlayer-ruby/tree/sl_pw_man_find_mod for questions:

1. I dont have any of these elements in my current test account (but my customers might and could use some clarification) or could not get editing to work:
   • Are UserCustomerExternalBinding's password editable through User_Customer add/remove binding methods or editObject method. Are there any examples for this?
   • Are apiPassword in NetworkServiceResource's attached to NetworkStorage editable through Network_Storage.editObject service or how do these become attached to a Network_Storage instance in general?
   • Are VirtualDiskImageSoftware passwords editable? I tried to use editObject from VirtulDiskImage to add some but neither added them nor threw any exception. If its editable what is the expected structure of the passwords array, what are they used for and what would normally add them?
   • Are remote management account passwords editable? I tried to edit through BareMetalService.service.editObject but it didnt change it nor throw an exception.
   • I dont have any application delivery controllers in my test account, can we verify if advancedModeFlag property is expected as true/false or 0/1 for object filtering?

2. In working with NetworkStorage and its associated credentials I noticed that there are password requirements that need to be met that are not documented and the exception thrown depending on whether you use editObject or the associated service method for changing passwords is different.

   Its not clear what those password requirements are in the doc, nor is it clear when a NetworkStorage instanced is allowed to have no password and no storage credentials associated with it. In my testing, I had pre-existing network storage devices in production with no password and no storage credentials that I couldnt try setting a password on as they are in use and a newly created ISCSI instance that had its NetworkStorage password equal to the first default NetworkStorageCredential instance that it refused to let me remove.

   If the NetworkStorage devices that I have with no password/credentials allows itself to have its password set or credentials added, then how do we undo this if we cannot remove the last remaining credential instance or send nil/empty string as a password?

3. How would you like the commits organized for the final PR? I can break it out/flatten commits into different branches/PRs as needed.

SL Ticket # 13966982

[SLsthompson]

For the external bindings, we currently support three different ones (Symantec VIP, Azure Multi-Factor Authentication (formerly Phone Factor) and Google Authenticator. To my knowledge none of the three makes use of the password field of the External Binding class. I think setting or checking the password on an external binding would be a moot operation.

[SLsthompson]

I've sent an email to the Network Storage folks to see if they could address your questions in that area.

[SLsthompson]

OK I got your questions picked up by the right folks to start handling them. I'll keep tracking to see if we can get some movement on them.

[SLsthompson]

If the NetworkStorage devices that I have with no password/credentials allows itself to have its password set or credentials added, then how do we undo this if we cannot remove the last remaining credential instance or send nil/empty string as a password?

If I understand your question, and the response I got, It seems that you are running into work-in-progress and its causing you some difficulty. The "Legacy iSCSI" storage and the "Consistent Performance" iSCSI storage in use different mechanisms to handle their credentials. The legacy storage mechanisms do not use SoftLayer_Network_Storage_Credential to store their credentials information which is why you don't see that object attached to them. The consistent performance ones DO have the credentials object attached, and that object is required (as I understand it).

[ju2wheels]

I think setting or checking the password on an external binding would be a moot operation.

OK, ill drop that one if its not editable, will leave the class for it there since its already implemented.

The "Legacy iSCSI" storage and the "Consistent Performance" iSCSI storage in use different mechanisms to handle their credentials. The legacy storage mechanisms do not use SoftLayer_Network_Storage_Credential to store their credentials information which is why you don't see that object attached to them. The consistent performance ones DO have the credentials object attached, and that object is required (as I understand it).

When I created the test iSCSI through the portal it automatically attached a Network_Storage_Credential to it and the password in Network_Storage is actually a reference to that instance so changing either one updated both.

The confusion I had was when I tried to remove that credential it wouldnt work (which is ok), but when I looked at some of the other iSCSI (which may be legacy, im not sure) they have neither a set password in Network_Storage nor any Network_Storage_Credential instances. I just want to ensure if the legacy behaves differently when it comes to setting the password that the implementation allows for it (right now the wrapper method denies using nil/empty string as password but if the Legacy allows no password and no credentials as a state but can still have its password set, how do we undo it?).

Ill email you an example dump of my network storage, it may make the question clearer.

[matttrach]

If you have legacy storage objects with nasType starts with 'ISCSI' that do not have storage credentials, then that is probably a bug, I would suggest opening a ticket. Some legacy objects (like NAS) are not tied to the storage credential system yet, and some standard objects (like Object Storage, Evault) have their own way of handling credentials as well at the moment.

[ju2wheels]

For the case of ISCI Consistent Performance they have an attached Host and to change the password you need to change it in the attached hardware.

This was in part of the response I got from the SL Support ticket, its more these type edge cases I was looking for if its correct.

I looked at what I had on the portal vs what Im getting from the API and figured out that the ones im seeing with no password/credentials are Consistent Performance and the other legacy ones do have either password or credentials, but they all have the same nasType of 'ISCSI' and are otherwise indistinguishable type wise.

For reference, this is what is see:

>> net_stor = SoftLayer::Account.account_for_client(sl_client).service.object_mask(SoftLayer::NetworkStorage.default_object_mask).getNetworkStorage.map{|net_storage| SoftLayer::NetworkStorage.new(sl_client, net_storage)}

>> net_stor.each{|stor| puts stor.type; puts stor.username; puts stor.password.inspect; puts stor.credentials.map{|cred| [cred.username, cred.password] }.inspect;puts}
NAS
IBM278181-1
"redacted"
[]

ISCSI
IBMI278181-1
"redacted"
[]

ISCSI
IBMI278181-5
"redacted"
[["IBMI278181-5", "redacted"]]

ISCSI
IBMI278181-6
"redacted"
[["IBMI278181-6", "redacted"]]

LOCKBOX
IBMLB278181-3
"redacted"
[]

LOCKBOX
IBMLB278181-4
"redacted"
[]

LOCKBOX
IBMLB278181-5
"redacted"
[]

NAS
IBMN278181-1
"redacted"
[]

NAS
IBMN278181-2
"redacted"
[]

HUB
IBMOS278181-2
nil
[]

ISCSI
IBM01SL278181-2
nil
[]

ISCSI
IBM01SL278181-3
nil
[]

NAS_CONTAINER
IBM01SVC278181_2
nil
[]

NAS_CONTAINER
IBM01SVC278181_3
nil
[]

NAS
IBM01SV278181_2
nil
[]

The first three ISCSI are legacy, the last two Consistent Performance (per portal). Is the username the proper way to differentiate (IBMI vs IBM01SL)?

[SLsthompson]

In the case of Consistent Performance the user name can be changed as the "name" field on the "allowedHost". For the password, you call SoftLayer_Network_Storage_Allowed_Host::setCredentialPassword.

I will look further into the way the portal figures out the storage type.

[SLsthompson]

I started looking at the question for SoftLayer_Network_Application_Delivery_Controller and in particular the advancedModeFlag. I didn't get very far... it looks like the posted documentation may be very wrong on this one too.

[SLsthompson]

For the SoftLayer_Virtual_Disk_Image_Software... do you have any of these on your account? I've sent a question about them, but haven't heard back yet.

[SLsthompson]

Are remote management account passwords editable? I tried to edit through BareMetalService.service.editObject but it didnt change it nor throw an exception.

For the Remote Management users, my understanding is that there is not a way to update the passwords for remote management users. This appears to be "by design". If you want to change the ACTUAL password for a remote management user then you would have to do so through the IPMI tool chain. Those passwords would not be reflected through the SoftLayer Portal and there doesn't appear to be a way to update the portal so it could display that change :-(.

[ju2wheels]

SoftLayer_Virtual_Disk_Image_Software... do you have any of these on your account?

Yes, they appear to be references to Software_Component/Software_Component_Password but the passwords listed dont match what I see in SoftwareComponents.

For example, if it is internal links to Software_Component then I would have expected these two things to produce the same results, but it doesnt:

el_sw = SoftLayer::Software.find_software_on_virtual_servers(:client=>sl_client, :name => "EL")
puts el_sw.map{|sw| sw.passwords.map{|pw| { :username => pw.username, :password => pw.password}}.inspect }

virt_disks = sl_client[:Account].getVirtualDiskImages.map{|virt_disk| SoftLayer::VirtualDiskImage.new(sl_client, virt_disk)}
virt_disks.each{|virt_disk| virt_disk.software.select {|sw| sw.name == "EL" }.each {|sw| puts sw.passwords.inspect} }

I get the same number of password entries for both but only the Software entries have properly populated passwords and the VirtualDiskImageSoftware passwords are returning more empty passwords than what is actually in SoftwareComponentPasswords (shows 3 of 37 vs 37 of 37 for SoftwareComponentPasswords).

If its editable only from SoftwareComponentPasswords then ill skip adding editing for VirtualDiskImageSoftware but I still think something is weird there.