search

UCSB-VRL / bisqueUCSB

An ML Platform for storing, analyzing, and visualizing your data all in the browser.
https://bisque2.ece.ucsb.edu
Other
35 stars 10 forks source link

TurboGears==2.2.1 #10

Closed amilworks closed 2 years ago

amilworks commented 4 years ago

TurboGears 2.2.1

This is the second upgrade in the process. If you have any questions, tag @amilworks on here. Current Version: TurboGears2==2.2.0 Upgrade Version: TurboGears2==2.2.1

As you are going through, check off what Step you completed to make it easier for us to help, and for you to feel accomplished.

Getting Started

Step 1. Clone the Repo

GOAL

git clone -b tg-upgrade --single-branch https://github.com/UCSB-VRL/bisqueUCSB.git

cd tg-upgrade and you should see a folder structure like this

.
├── boot
├── builder
├── COPYRIGHT
├── DOCKER-BUILD_LOG.md
├── Dockerfile.caffe.xenial
├── DOCKER-RUN_LOG.md
├── entry.sh
├── LICENSE
├── Makefile
├── modules
├── README.md
├── requirements.txt
├── run-bisque.sh
├── source
├── sources.list
├── start-bisque.sh
└── virtualenv.sh

4 directories, 13 files

Step 2. Verify Everything Works

GOAL

You have two choices to verify that the repo is working.

Option 1. Pull the Latest Docker Build (Recommended)

The repo is rebuilt and pushed automatically after a successful push/merge on GitHub. Download the latest image by running the following command:

docker pull amilworks/bisque-tg2-upgrade:git

You can run this image to verify BisQue is working and able to run locally on your system by running:

docker run -itp 8080:8080  --name bqdev-tg2 amilworks/bisque-tg2-upgrade:git

You should see BisQue on http://localhost:8080/ in your browser. If not, paste the logs below and we will take a look.

Option 2. Build the Image

Building the image from scratch is what you will end up doing throughout the upgrade process. Luckily, we automated it to where you run a Makefile.

make bisque05-caffe-xenial

If you vim Makefile, you will see it's a series of basic commands. I suggest you Edit this file and --tag the image something different and update the version you are upgrading instead of amilworks/bisque05-caffe-xenial:flour-tg2.2. You can leave it but it's kind of long. Once the build is done, verify that BisQue comes up on your local system just as above:

docker run -itp 8080:8080  --name bqdev-tg2 amilworks/bisque-tg2-upgrade:git

You should see BisQue on http://localhost:8080/ in your browser. If not, paste both the Build and Run Logs below and we will take a look.

Step 3. Make Changes/Break Stuff

GOAL

Create a new branch Creating a new branch will enable you to make changes, nuke your own branch over and over again, and maintain a clean copy of the original branch when you need to start fresh for the millionth time. 

git checkout -b [name_of_your_new_branch]

Open source/requirements.txt with your favorite editor (hopefully Vim). Change the version to the one you are assigned, and save the file.

Here is the link to the line you will update.

TurboGears2==2.2   # <--- Update Version
TurboMail==3.0.3
WebError==0.10.3
WebFlash==0.1a9
WebHelpers==1.3
WebOb==1.1.1       # <--- Potential Update Needed, Check Build Logs

Push to GitHub You made changes and you want to have version control. Push your new branch and changes to GitHub by running:

git push -u origin  [name_of_your_new_branch]

Step 4. Build and Run!

GOAL

Once you made your changes, make sure you cd .. out of source/ and are back to the tree view displayed above. You should see a file called Makefile. You can Edit this file as mentioned in Option 2. or simply run the following command to build the image with your updated TurboGears2. 

make bisque05-caffe-xenial

While its building, make sure that there are no errors during the pip installs. Typically errors are skipped over in favor of a successful build. I know, right?

Next, once the build is finished, run the image and verify BisQue comes up on your local system:

docker run -itp 8080:8080  --name bqdev-tg2 amilworks/bisque-tg2-upgrade:git

NOTE: There might be other packages that need updating when upgrading to different TurboGears2 versions. Sometimes there will be an error raised with the version required, or it simply will not run and you will have to do a logsdump here and we will have to sift through it together to track the failed step(s).

If everything is successful, push your branch to GitHub if you have not already, file a pull request, and pat yourself on the back. Job well done. We will review it and test it on our production-like setup and give feedback if any issues arise.

Post Upgrade

Use this issue to document any issues you faced during the upgrade process. Log dumps can be thrown here as well. Try to use the code syntax for large blocks of code so we can easily read through it. Otherwise, go crazy.

ASMIftekhar commented 4 years ago

I think this is not a serious issue but when I see bisque on my local browser it shows an error but the site comes up and all :

Error Requested resource does not exist: /services

ASMIftekhar commented 4 years ago

I cant push local update to the repo, remote: Permission to UCSB-VRL/bisqueUCSB.git denied to ASMIftekhar.

ASMIftekhar commented 4 years ago

Getting this error after changing TG version and running the make file:

Step 7/23 : RUN wget -q https://bitbucket.org/dimin/bioimageconvert/downloads/$IMGCNV.tar.gz && tar xf $IMGCNV.tar.gz && mv $IMGCNV/imgcnv /usr/bin && mv $IMGCNV/libimgcnv.so /usr/lib/x86_64-linux-gnu/ && rm -rf $IMGCNV && apt-get install -y --no-install-recommends libswscale-ffmpeg3 libfftw3-3 libgdcm2.6 libavcodec-ffmpeg56 libavformat-ffmpeg56 libavutil-ffmpeg54 libhdf5-cpp-11 ---> Running in 8b6c936f4981 The command '/bin/sh -c wget -q https://bitbucket.org/dimin/bioimageconvert/downloads/$IMGCNV.tar.gz && tar xf $IMGCNV.tar.gz && mv $IMGCNV/imgcnv /usr/bin && mv $IMGCNV/libimgcnv.so /usr/lib/x86_64-linux-gnu/ && rm -rf $IMGCNV && apt-get install -y --no-install-recommends libswscale-ffmpeg3 libfftw3-3 libgdcm2.6 libavcodec-ffmpeg56 libavformat-ffmpeg56 libavutil-ffmpeg54 libhdf5-cpp-11' returned a non-zero code: 8 Makefile:9: recipe for target 'bisque05-caffe-xenial' failed make: *** [bisque05-caffe-xenial] Error 8

ASMIftekhar commented 3 years ago

Changed TurboGear from 2.1.5 to 2.2.1. Make got well but running the container gives me the following error:

ASMIftekhar commented 3 years ago

So I changed web0b==1.0.8 to 1.1.1 and build again. The built went well but after running the docker container I got this error. This means something got broken due to the update of web0b. Any way to track this? image

amilworks commented 3 years ago

So its saying that the Turbogears version you have installed needs WebOb to be upgraded. Nice work!

Requirement.parse('WebOb==1.1.1'), set(['TurboGears2']))
amilworks commented 3 years ago

Can you try upgrading the requests package?

ASMIftekhar commented 3 years ago

To which version?

ASMIftekhar commented 3 years ago

This comment is for @mcever @FeiX-OvO and @ivanfarevalo. You first need to build the container by running: make bisque05-caffe-xenial Then docker run command should be: docker run -itp 8080:8080 --name bqdev-tg2 amilworks/bisque05-caffe-xenial:flour-tg2.2

amilworks commented 3 years ago

So I tried to find a specific/relevant version, but couldn't find anything that was definitive like "Use this version for WebOb==1.1.1". So my suggestion is naive, let's try the latest since it supports Python 2.7. We will try this extreme, then we can try the opposite extreme of the next version bump from our current one.

pip install requests==2.25.0
ASMIftekhar commented 3 years ago

Didnt work, it broke some other package name MonekeyPatch. Are you sure Web0b is not working because of requests. Then may be we can try randomly different requests versions.

amilworks commented 3 years ago

Yeah, so you're right. WebOb actually doesn't have any dependencies, from what I can see, except for the Python Standard library. That's pretty crazy. This will probably require a little more digging because from the looks of it, nothing was Raised to say "upgrade package X to v2.x". Here are the changes from the commit log for turbogears2==2.2. I highlighted one in bold that might be the reason for the /services issue but I am not completely sure.

@satish1901 Do you have any ideas?

github-actions[bot] commented 2 years ago

Stale issue message