Install UPP on cluster and create Imagebuilder recip

fossell commented 2 years ago

This is a sub-issue of #34 .

Describe the Sub-Issue

First need to make sure UPP can be installed and built by ImageBuilder

Time Estimate

3 days

Define the Metadata

Assignee

[ ] Select engineer(s) or no engineer required
[x] Select scientist(s) or no scientist required

Labels

[x] Select component(s)
[x] Select priority

Projects and Milestone

[x] Select Project
[x] Select Milestone as the next official version or Backlog of Development Ideas

hahnd commented 2 years ago

The ImageBuilder file is a bit hidden, so just wanted to point it out.

Not sure if you already have a strategy for this task, but it helps me to start the cluster head node, build it there manually, and then add to the ImageBuilder file once I know it works.

fossell commented 2 years ago

The ImageBuilder file is a bit hidden, so just wanted to point it out.

Not sure if you already have a strategy for this task, but it helps me to start the cluster head node, build it there manually, and then add to the ImageBuilder file once I know it works.

@hahnd - I was actually just working on that this morning. I did get UPP installed successfully, I had to install jasper libs too and just did a sudo yum -y install jasper and that seemed to do the trick. And I had to set some paths to libpng, zlib too in the configure file.

This is my guess at what the UPP build commands in cf_imagebuilder_wrf_intel.yaml should be. But I also need to figure out what env vars need to be set or how to easily modify the configure.upp file on the fly to get the proper include and lib paths set for libpng and zlib.

- name: build-upp   
   action: ExecuteBash
   inputs:
     commands:
      - source /etc/bashrc
      - cd /home/ec2-user
      - wget https://dtcenter.org/sites/default/files/code/DTC_upp_v4.0.1.tar.gz
      - tar zxvf DTC_upp_v4.0.1.tar.gz
      - cd UPPV4.0.1
      - ./clean
      - ./configure << EOF
      - 4
      - EOF
      - ./compile 2>&1 | tee build.log
      - cd /home/ec2-user
      - chown -R ec2-user.ec2-user UPPV4.0.1

fossell commented 2 years ago

@hahnd - Ok I see the imagebuilder file that jasper should be included. Which makes sense because it's required for WPS. Let me take a look again and see why I couldn't find it.

hahnd commented 2 years ago

I am not familiar with UPP, but in the AWS cluster environment, you should have an environment variable: ${JASPER}, so you can reference ${JASPER}/include or ${JASPER}/lib as needed.

fossell commented 2 years ago

I deleted the cluster and created a new, I do see jasper in /opt/jasper now... I'm not sure what happened before. I'm able to add setting of JASPERLIB and JASPERINC and everything looks to be building successfully. So the updated build commands are below. I will test this rebuilding the package now that the manual installation steps on the head node are working well.

- name: build-upp   
   action: ExecuteBash
   inputs:
     commands:
      - source /etc/bashrc
      - cd /home/ec2-user
      - wget https://dtcenter.org/sites/default/files/code/DTC_upp_v4.0.1.tar.gz
      - tar zxvf DTC_upp_v4.0.1.tar.gz
      - cd UPPV4.0.1
      - export JASPERLIB="-L${LIBPNG}/lib -L${ZLIB}/lib -L${JASPER}/lib -ljasper -lpng -lz" 
      - export JASPERINC="-I${LIBPNG}/include -I${ZLIB}/lib -I${JASPER}/include"
      - ./clean
      - ./configure << EOF
      - 4
      - EOF
      - ./compile  2>&1 | tee build.log
      - cd /home/ec2-user
      - chown -R ec2-user.ec2-user UPPV4.0.1

NCAR / wrfcloud