Provide a way to automate the initial configuration of a Selenium Grid Extras node

[charlescapps]

Currently, AFAIK, the only way to register a new node to the Hub is to execute "java -jar selenium-grid-extras.jar" and manually answer the questions asked.

You have to go through this process for SGE to create a directory for the node on the Hub.

It would be awesome if there a command-line option such as "-DregisterNodeWithHub=true" _which would register the node with the hub, even if the selenium_grid_extrasconfig.json file already exists.

That way, it wouldn't be necessary to go through all of the prompts, and we could easily automate the process of deploying new SGE nodes.

[smccarthy]

I am not sure if this is helpful, but what I have started to do in the past, was to have the selenium_grid_extras_config.json, hub json config, node json config already created, and just push that to the new node (using chef). I would also push the SGE jar file as well. This way, it looks like the questions have already been answered. My config files were all templates in my chef cookbook.

@charlescapps , would that solve your problem?

[charlescapps]

I don't think that works, because there's still the matter of creating the config folder on the Hub. I'm using the option to centralize configs on the Hub. Maybe if I manually created this config folder based on the node hostname, the node would download the latest configs automatically...I'm not sure if that works.

Tl; dr - it would be highly useful if we could do the ordinary node registration process withour manually going through the prompts. On Apr 8, 2015 5:58 PM, "Shawn McCarthy" notifications@github.com wrote:

I am not sure if this is helpful, but what I have started to do in the past, was to have the selenium_grid_extras_config.json, hub json config, node json config already created, and just push that to the new node (using chef). I would also push the SGE jar file as well. This way, it looks like the questions have already been answered. My config files were all templates in my chef cookbook.

@charlescapps https://github.com/charlescapps , would that solve your problem?

— Reply to this email directly or view it on GitHub https://github.com/groupon/Selenium-Grid-Extras/issues/111#issuecomment-91081226 .

[chdavi]

Why not just use the rest endpoint to update the configuration files on the hub? That's what it does when you answer the questions.

I created templates for my nodes, override a few variables, then push the changed configs up via the rest endpoint in powershell.

That way, there are no prompts involved.

I can post the code if you are interested.

-Chris

On Apr 8, 2015, at 10:08 PM, Charles Capps notifications@github.com wrote:

I don't think that works, because there's still the matter of creating the config folder on the Hub. I'm using the option to centralize configs on the Hub. Maybe if I manually created this config folder based on the node hostname, the node would download the latest configs automatically...I'm not sure if that works.

Tl; dr - it would be highly useful if we could do the ordinary node registration process withour manually going through the prompts. On Apr 8, 2015 5:58 PM, "Shawn McCarthy" notifications@github.com wrote:

I am not sure if this is helpful, but what I have started to do in the past, was to have the selenium_grid_extras_config.json, hub json config, node json config already created, and just push that to the new node (using chef). I would also push the SGE jar file as well. This way, it looks like the questions have already been answered. My config files were all templates in my chef cookbook.

@charlescapps https://github.com/charlescapps , would that solve your problem?

— Reply to this email directly or view it on GitHub https://github.com/groupon/Selenium-Grid-Extras/issues/111#issuecomment-91081226 .

— Reply to this email directly or view it on GitHub.

[smccarthy]

@chdavi I would be interested in seeing it.

[charlescapps]

Sure, posting the code would be great. I haven't looked at the code closely for this project to see how it was implemented. I didn't even realize it was just doing an HTTP request.

We could probably write a powershell script to do the HTTP request (since our nodes are Windows nodes.)

~Charles Capps

On Wed, Apr 8, 2015 at 9:43 PM, Christopher Davis notifications@github.com wrote:

Why not just use the rest endpoint to update the configuration files on the hub? That's what it does when you answer the questions.

I created templates for my nodes, override a few variables, then push the changed configs up via the rest endpoint in powershell.

That way, there are no prompts involved.

I can post the code if you are interested.

-Chris

On Apr 8, 2015, at 10:08 PM, Charles Capps notifications@github.com wrote:

I don't think that works, because there's still the matter of creating the config folder on the Hub. I'm using the option to centralize configs on the Hub. Maybe if I manually created this config folder based on the node hostname, the node would download the latest configs automatically...I'm not sure if that works.

Tl; dr - it would be highly useful if we could do the ordinary node registration process withour manually going through the prompts. On Apr 8, 2015 5:58 PM, "Shawn McCarthy" notifications@github.com wrote:

I am not sure if this is helpful, but what I have started to do in the past, was to have the selenium_grid_extras_config.json, hub json config, node json config already created, and just push that to the new node (using chef). I would also push the SGE jar file as well. This way, it looks like the questions have already been answered. My config files were all templates in my chef cookbook.

@charlescapps https://github.com/charlescapps , would that solve your problem?

— Reply to this email directly or view it on GitHub < https://github.com/groupon/Selenium-Grid-Extras/issues/111#issuecomment-91081226

.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub https://github.com/groupon/Selenium-Grid-Extras/issues/111#issuecomment-91114145 .

[chdavi]

@charlescapps You are in luck, all my nodes are windows nodes so I have got most of what you need already in powershell. I am writing chocolatey packages for the node install and the configs. Each configuration package is responsible for calling the hub and updating it with the new configuration.

Utility Functions

function Process-Configs {
param(      
      [string] $hub = $(throw "-hub is required.")
    )

    $HUB_TOKEN = "@@HUB@@"
    $HOSTNAME_TOKEN = "@@HOSTNAME@@"    
    $node = $env:COMPUTERNAME

    $fileNames = Get-ChildItem "C:\selenium" -Recurse  | Where-Object {$_.Name -match ".*json|.*txt"} | select -expand fullname
    foreach ($filename in $filenames) 
    {
        Write-Debug "Replacing tokens in $filename"
        (Get-Content $fileName) -replace $HUB_TOKEN, $hub -replace $HOSTNAME_TOKEN, $node | Set-Content $fileName  
    }
}

function Get-Arguments {
    <#
    .SYNOPSIS
    Parse the chaocolateyPackageParameters into a dictionary

    .DESCRIPTION
    Parse the chaocolateyPackageParameters into a dictionary

    .OUTPUTS
    The argument dictionary
    #>
    $arguments = @{}
    $packageParameters = $env:chocolateyPackageParameters
    $splitParams = $packageParameters.Split(",")
    foreach($p in $splitParams) {
        $param = $p.Split("=")
        $arguments.Add($param[0], $param[1])        
    }
    return $arguments
}

function Upload-Config {
    <#
    .SYNOPSIS
    Uploads a config to the selenium server

    .DESCRIPTION
    This will take a configuration file and upload it to the selenium hub

    .PARAMETER Path
    The path to the file to upload

    .PARAMETER Name
    The name of the configuration file

    .PARAMETER hub
    The url of the hub to upload the file to

    .EXAMPLE
    Upload-Config '__PATH__' '__NAME__' '__HUB__' 

    .OUTPUTS
    None
    #>
    param(
      [string] $path = $(throw "-path is required."),
      [string] $name = $(throw "-name is required."),
      [string] $hub = $(throw "-hub is required.")
    )        

    $content = encodeBase64(Get-Content $path)    
    $content
    $URI = "http://" + $hub + ":3000/update_node_config?node=$env:COMPUTERNAME&filename=$name&content=$content"
    Write-Host "Uploading the configuration file: $name to $URI"
    $r = Invoke-WebRequest -Method Get -Uri $URI
    Write-Debug "Invoke-WebRequest"
    Write-Debug $r
}

function encodeBase64($content) {
<#
    .SYNOPSIS
    Encodes a string to its base64 equivalent

    .DESCRIPTION
    This will take a string and base64 encode it

    .PARAMETER Path
    The path to the file to upload

    .PARAMETER Content
    The string content to encode

    .OUTPUTS
    The encoded string
    #>
    $content = [System.Text.Encoding]::UTF8.GetBytes($content)
    $content = [System.Convert]::ToBase64String($content)
    return $content
}

Example

if($arguments.ContainsKey("hub")) {
    $hub = $arguments["hub"]
}

Process-Configs $hub
Upload-Config "c:\selenium\node_5555.json" "node_5555.json" $hub

[smccarthy]

@charlescapps Is this still an "issue"? If so, I can mark this as an enhancement. Also, I started to create a chef cookbook for selenium-grid-extras (https://github.com/smccarthy/chef-selenium-grid-extras ). I am new to chef and I am sure there are some improvements needed for it.

[charlescapps]

This is still an issue for our automation. Chef sounds cool, but I don't really want to invest the time in it right now. It would be a huge cross-department project.

Since we're uploading configs to the Hub, the nice thing is that we don't need Chef or Puppet or anything like that. We can just use a simple script that updates all the node_5555.json files in the configs/ directory on the Hub.

So, yep, it would still be useful if there was a command line flag we could pass in to setup the Node initially in "non-interactive" mode. We don't even care what the initial config is (as long as upload to hub = true) since we can manage it on the Hub, we just need a non-interactive way to create the folder on the configs/ directory without the Hub having any knowledge of what all the nodes are until the nodes run SGE.

As the other user suggests, we could have a PowerShell script, but...why the extra complication? I'd rather not maintain a script that depends on REST endpoints that could change over time.

[chdavi]

@charlescapps @smccarthy Just thought I would mention that I no longer use the REST endpoints and upload my configs, and instead use Docker, Vagrant, VM's & Puppet to maintain the configs on the nodes instead of the hub.

This is much cleaner, you can spin up/tear down the machines in a few minutes, and give you the benefit of your grid being in source control. You can also scale your grid much easier, just add a yaml config file for it and vagrant up you new windows testing VM. Once linked clones are finished in vagrant it will take a fraction of the total disk space of a normal machine.

Docker is even easier. docker-compose scale firefox=15 will give you more firefox capabilities in just a few seconds. Trashing the linux part of the grid and restarting it takes only a few seconds.

[carlosmmelo]

do you have a repo for the docker conf? @chdavi