search

CCSI-Toolset / FOQUS

FOQUS: Framework for Optimization and Quantification of Uncertainty and Surrogates
https://foqus.readthedocs.io
Other
46 stars 54 forks source link

FOQUS Image Builder #1005

Closed boverhof closed 1 year ago

boverhof commented 2 years ago

Descriptors

Create a new AMI for FOQUS Cloud using Cloud init

Details

boverhof commented 2 years ago

Unattended Install Help.pdf

boverhof commented 2 years ago

UserData

<PowerShell>
$ErrorActionPreference = "Stop"
$DESKTOP = "$home\Desktop"
$BUCKET_NAME = "foqus2-cloud-files"
Start-Transcript -Path "$DESKTOP\foqus_cloud_ami_setup.log"
net user Administrator CHANGE_THE_PASSWORD
Uninstall-WindowsFeature -Name Windows-Defender

function InstallSQLExpress2014() {
  Write-Host "==== Install SQL Express 2014"
  cd $DESKTOP\$BUCKET_NAME
  Read-S3Object -BucketName foqus2-cloud-files -Key aspen_unintended_install/aspen_v12/3rdparty/sqlexpress_x64_2014_sp2.exe -File SQLEXPR_x64_ENU.exe
  Read-S3Object -BucketName  $BUCKET_NAME -Key config/SQLSERVER/SQL_2014_configuration.ini -File SQL_2014_configuration.ini;
  mkdir SQLEXPR_2014
  Write-Host "  extract SQL Express 2014"
  .\SQLEXPR_x64_ENU.exe /q /x:C:\Users\Administrator\Desktop\$BUCKET_NAME\SQLEXPR_2014
  cd SQLEXPR_2014
  sleep 5
  Write-Host "  install using configuration"
  .\SETUP.EXE /ConfigurationFile=C:\Users\Administrator\Desktop\$BUCKET_NAME\SQL_2014_configuration.ini
  Write-Host "  finished SQL Express 2014"
}

function InstallAspen12() {
  Write-Host "==== Install Aspen v12"
  cd $DESKTOP
  Read-S3Object -BucketName  $BUCKET_NAME -Key aspen_unintended_install/aspen_v12/Aspen12Installer.zip -File Aspen12Installer.zip
  Read-S3Object -BucketName  $BUCKET_NAME -Key aspen_unintended_install/aspen_v12/Aspen12UnattendedInstaller.zip -File Aspen12UnattendedInstaller.zip
  Expand-Archive Aspen12UnattendedInstaller.zip
  Expand-Archive Aspen12Installer.zip
  cd Aspen12UnattendedInstaller
  copy '.\aspenONE Engineering V12.xml' $DESKTOP
  Write-Host "  running unattended installation"
  & '.\aspenONE Engineering V12.bat'
  Write-Host "  finished unattended installation"
}

function InstallFOQUS() {
  Write-Host "==== Install FOQUS"
  conda install -y -c anaconda git
  cd $DESKTOP
  git clone https://github.com/boverhof/FOQUS.git  --single-branch --branch="boverhof_fix_optimization_01.18.2022"
  cd FOQUS;
  pip install -r requirements.txt
  Write-Host "  finished FOQUS"
}
function InstallSQLCompact() {
  Write-Host "==== Install SQLCompact"
  cd $DESKTOP\$BUCKET_NAME
  Read-S3Object -BucketName  $BUCKET_NAME -Key SSCERuntime_x64-ENU.exe -File SSCERuntime_x64-ENU.exe;
  .\SSCERuntime_x64-ENU.exe /package /quiet  /l C:\Users\Administrator\Desktop\SSCERuntime_x64.log
  Write-Host "  finished SQLCompact"
}

function InitializeBucketFolder() {
  Write-Host "==== Install Bucket Folder"
  cd $DESKTOP
  mkdir $BUCKET_NAME
  cd $BUCKET_NAME
  Read-S3Object -BucketName  $BUCKET_NAME -Key CompactView_1.4.16.0_Installer.exe -File CompactView_1.4.16.0_Installer.exe;
  Write-Host "  finished bucket"
}

Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://community.chocolatey.org/install.ps1'))
choco install -y  anaconda3
C:\tools\Anaconda3\Scripts\conda.exe create --name foqus -y  python=3.8

$P1 =
"C:\tools\Anaconda3\envs\foqus;C:\tools\Anaconda3\envs\foqus\Library\mingw-w64\bin;C:\tools\Anaconda3\envs\foqus\Library\usr\bin;C:\tools\Anaconda3\envs\foqus\Library\bin;C:\tools\Anaconda3\envs\foqus\Scripts;C:\tools\Anaconda3\envs\foqus\bin;C:\tools\Anaconda3\envs\foqus\condabin"
$P2 = "C:\tools\Anaconda3;C:\tools\Anaconda3\Library\mingw-w64\bin;C:\tools\Anaconda3\Library\usr\bin;C:\tools\Anaconda3\Library\bin;C:\tools\Anaconda3\Scripts;C:\tools\Anaconda3\bin;C:\tools\Anaconda3\condabin"

$P3 = "C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0;C:\Program Files\Amazon\cfn-bootstrap;C:\ProgramData\chocolatey\bin;C:\Users\Administrator\AppData\Local\Microsoft\WindowsApps"

[Environment]::SetEnvironmentVariable('path', "$P1;$P2;$P3", 'Process');
[Environment]::SetEnvironmentVariable('path', "$P1;$P2;$P3", 'Machine');

InitializeBucketFolder
InstallFOQUS
InstallSQLExpress2014
InstallAspen12
InstallSQLCompact
</PowerShell>
dghoshal-lbl commented 2 years ago

Got this error while installing FOQUS through the user data script:

ERROR: Could not find a version that satisfies the requirement PyQt5==5.13 (from ccsi-foqus) (from versions: 5.12.3, 5.1
4.0, 5.14.1, 5.14.2, 5.15.0, 5.15.1, 5.15.2, 5.15.3, 5.15.4, 5.15.5, 5.15.6)
ERROR: No matching distribution found for PyQt5==5.13
  finished FOQUS
dghoshal-lbl commented 2 years ago

Got this error when using expand-archive, though manually unpacking succeeds:

PS C:\Users\Administrator\Desktop> Expand-Archive -Force Aspen12Installer.zip
Remove-Item : Cannot find path 'C:\Users\Administrator\Desktop\Aspen12Installer\Aspen12Installer\aspenonesuite\Aspen
Basic Engineering V12 (64bit)\' because it does not exist.
At
C:\Windows\system32\WindowsPowerShell\v1.0\Modules\Microsoft.PowerShell.Archive\Microsoft.PowerShell.Archive.psm1:411
char:46
+ ...                 $expandedItems | % { Remove-Item $_ -Force -Recurse }
+                                          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (C:\Users\Admini...ng V12 (64bit)\:String) [Remove-Item], ItemNotFoundEx
   ception
    + FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.RemoveItemCommand
dghoshal-lbl commented 2 years ago

@boverhof : I am getting three types of errors while trying to do the unattended installation of Aspen on a Windows AMI: i) The first error is an http connection failed error. This happens only when the script is executed through user_data field at launch time of an instance.

Chocolatey installed 1/1 packages.
 See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).
Collecting package metadata (current_repodata.json): failed

CondaHTTPError: HTTP 000 CONNECTION FAILED for url <https://repo.anaconda.com/pkgs/main/win-64/current_repodata.json>
Elapsed: -

An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.

The error goes away when I retry the step from within the instance, once launched.

ii) The second error is related to Aspen installation. I get numerous application errors (example screenshot attached) while trying to install Aspen. If I ignore these errors, Aspen eventually gets installed, but does not start. I wonder if you have seen any of these errors?

Screen Shot 2022-02-04 at 2 02 30 PM

I am also unable to provide the license server information anywhere during the unattended installation, and I am figuring it out.

iii) The third one is a minor one, where SQL express installation hangs indefinitely. Happens occasionally.

lbianchi-lbl commented 2 years ago

This could mean that PyQt 5.13 has been dropped from the PyPI repository, so we should look into this to make sure the same error doesn't happen in other environments.

boverhof commented 1 year ago