PXD-310 ⁃ Network: Load Balance the squid proxy

[skattar1406]

➤ Trevar Simmons commented:

This includes HA-ifying.

[skattar1406]

➤ Yajing Tang commented:

we have commons that use this for ftp and sftp proxying too. Need to test and make sure it still work for this two cases

[skattar1406]

➤ Ray Powell commented:

Are you sure about the SFTP part ? I dont think you can route SFTP( FTP like cmds over SSH) through squid. Can do FTPS (SSL enabled FTP)

Raymond Powellrpowell1@uchicago.edu Office #: 1-773-834-2376 Director of Cloud Operations for Center for Data Intensive Science at the University of Chicago

On Thu, May 10, 2018 at 4:06 PM, Yajing Tang notifications@github.com wrote:

we have commons that use this for ftp and sftp proxying too. Need to test and make sure it still work for this two cases

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/uc-cdis/cloud-automation/issues/250#issuecomment-388186070, or mute the thread https://github.com/notifications/unsubscribe-auth/ABRsi2yWtkjH7Vy3ekD5ueZh2v5jgL0Pks5txKvagaJpZM4TKm7U .

[skattar1406]

➤ Yajing Tang commented:

you can? as long as it has the ssh key added to squid

[skattar1406]

➤ Yajing Tang commented:

also, to clarify the ticket, does this also implies the HA-ified squid is moved to csoc? One thing to keep in mind is we still want our automation easily configurable later to allow a Commons to be spin up without CSOC ( eg: a env var for the automation script to tell it do terraform for the Commons that has bastion and squid within the Commons)

[skattar1406]

➤ rarya commented:

The idea is to have a centralized NLB arrangement for the squid proxy in CSOC for all commons.I am working on terraform scripting/testing up the endpoint service in CSOC and the endpoint in the commons VPC to access the squid service from commons to the CSOC. If we want to keep the squid on each commons or the code to spin one around when needed; I believe it is something we can discuss and decide on how we want to do that.

[skattar1406]

➤ Yajing Tang commented:

yeah it's cool to have the central load balanced squid. Just want to make sure that it's all configurable and not hardcoding csoc specific ids to the Commons gen3 automation so that it's easy for us to support gen3 without csoc too.

[skattar1406]

➤ rarya commented:

Tested file download from the test SFTP server (sftp.planx-pla.net); Works fine via the squid-nlb setup.