search

oracle / docker-images

Official source of container configurations, images, and examples for Oracle products and projects
https://developer.oracle.com/use-cases/#containers
Universal Permissive License v1.0
6.53k stars 5.42k forks source link

./buildExtensions.sh OPATCHAUTO-72156: Patch location not accessible or empty. #1820

Closed xtender closed 3 years ago

xtender commented 3 years ago

I'm trying to upgrade an image with 19.3 up to 19.9: I've put p31771877_190000_Linux-x86-64.zip into patches directory and ran ./buildExtensions.sh -a -b oracle/database:19.3.0-ee -t oracle/database:19.9.0-ee

and looks like it can't upgrade because of the following errors in step 5:

OPATCHAUTO-72156: Patch location not accessible or empty.
OPATCHAUTO-72156: Patch location supplied cannot be accessed or no content found.
OPATCHAUTO-72156: Please provide a correct patch location.

Full log:

extensions]# ./buildExtensions.sh -a -b oracle/database:19.3.0-ee -t oracle/database:19.9.0-ee
==========================
DOCKER info:
Client:
 Debug Mode: false

Server:
 Containers: 8
  Running: 8
  Paused: 0
  Stopped: 0
 Images: 73
 Server Version: 19.03.8
 Storage Driver: overlay2
  Backing Filesystem: <unknown>
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 7ad184331fa3e55e52b890ea95e65ba581ae3429
 runc version: dc9208a3303feef5b3839f4323d9beb36df0a9dd
 init version: fec3683
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 3.10.0-1062.12.1.el7.x86_64
 Operating System: CentOS Linux 7 (Core)
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 31.13GiB
 Name: orasql.org
 ID: BSQE:QM2V:GIUD:VT7Q:M5YC:JXTE:ZELX:RPBP:7II6:MQ4I:XJ3S:3BBI
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled
==========================
Building extension patching...
Sending build context to Docker daemon  1.425GB
Step 1/17 : ARG BASE_IMAGE=oracle/database:19.3.0-ee
Step 2/17 : FROM ${BASE_IMAGE} as patching
 ---> d58acd761923
Step 3/17 : ENV HOST_PATCH_DIR="patches"     PATCH_DIR=/opt/install/patches     PATCH_DB_BINARIES_FILE="patchDBBinaries.sh"
 ---> Running in f95d9dc720eb
Removing intermediate container f95d9dc720eb
 ---> f038ee29e866
Step 4/17 : COPY --chown=oracle:dba $HOST_PATCH_DIR $PATCH_DB_BINARIES_FILE $PATCH_DIR/
 ---> f1cfbd6cee72
Step 5/17 : RUN chmod ug+x $PATCH_DIR/*.sh &&     sync &&     $PATCH_DIR/$PATCH_DB_BINARIES_FILE &&     rm -rf $PATCH_DIR/* $ORACLE_HOME/.patch_storage
 ---> Running in 7c62ede93814
Unzipping /opt/install/patches/release_update/p31771877_190000_Linux-x86-64.zip
No one-offs to be installed.
Applying Release Update: 31771877
Running: /opt/oracle/product/19c/dbhome_1/OPatch/opatchauto apply -binary -oh /opt/oracle/product/19c/dbhome_1 /opt/install/patches/31771877 -target_type rac_database
OPATCHAUTO-72156: Patch location not accessible or empty.
OPATCHAUTO-72156: Patch location supplied cannot be accessed or no content found.
OPATCHAUTO-72156: Please provide a correct patch location.

 perl patching initializaion failed with error code 255.
Removing intermediate container 7c62ede93814
 ---> c836abe09170
Step 6/17 : FROM ${BASE_IMAGE}
 ---> d58acd761923
Step 7/17 : ENV DATAPATCH_FILE="runDatapatch.sh"     LSPATCHES_FILE="savePatchSummary.sh"
 ---> Running in 60d1b6ddeb34
Removing intermediate container 60d1b6ddeb34
 ---> 6b51da9eee55
Step 8/17 : ARG EXTENSION_NAME="patching"
 ---> Running in ef866981e637
Removing intermediate container ef866981e637
 ---> 5ab8cf0b6df8
Step 9/17 : COPY --chown=oracle:dba --from=patching $ORACLE_BASE $ORACLE_BASE
 ---> 1f8d04ea0722
Step 10/17 : COPY --chown=oracle:dba $DATAPATCH_FILE $ORACLE_BASE/scripts/startup
 ---> 57aa70f683b2
Step 11/17 : RUN chmod ug+x $ORACLE_BASE/scripts/startup/*.sh && sync
 ---> Running in 80cc5f5d1a08
Removing intermediate container 80cc5f5d1a08
 ---> 318f4c44fdc9
Step 12/17 : COPY --chown=oracle:dba $LSPATCHES_FILE $ORACLE_BASE/scripts/setup
 ---> 84a62aab6e35
Step 13/17 : RUN chmod ug+x $ORACLE_BASE/scripts/setup/*.sh && sync
 ---> Running in d2d158d13cbb
Removing intermediate container d2d158d13cbb
 ---> 7e180a577290
Step 14/17 : RUN mv "$ORACLE_BASE/$RUN_FILE" "$ORACLE_BASE/$RUN_FILE.$EXTENSION_NAME"
 ---> Running in d1deaf0b1cd7
Removing intermediate container d1deaf0b1cd7
 ---> 01a02238813a
Step 15/17 : COPY  --chown=oracle:dba $RUN_FILE $ORACLE_BASE/
 ---> f4f659568da8
Step 16/17 : RUN chmod ug+x $ORACLE_BASE/*.sh && sync
 ---> Running in 8d7d23d9bf4e
Removing intermediate container 8d7d23d9bf4e
 ---> 4612c6b74d3b
Step 17/17 : RUN echo -e "\n. $ORACLE_BASE/$RUN_FILE.$EXTENSION_NAME" >> "$ORACLE_BASE/$RUN_FILE"
 ---> Running in c526e16605e4
Removing intermediate container c526e16605e4
 ---> a1ae34f88f48
Successfully built a1ae34f88f48
xtender commented 3 years ago

I've added a couple of ls -la before echo "Running: $cmd"; in patchDBBinaries.sh 

    cmd="${ORACLE_HOME}/OPatch/opatchauto apply -binary -oh $ORACLE_HOME ${PATCH_DIR}/${ru_patch}";
    echo "ls: ";
    ls -la "${PATCH_DIR}";
    ls -la "${PATCH_DIR}/${ru_patch}";
    echo "Running: $cmd";

and it showed:

ls:
total 500
drwxr-xr-x 1 oracle dba        4096 Dec 28 12:46 .
drwxr-xr-x 1 root   root       4096 Dec 28 12:45 ..
drwxr-xr-x 5 oracle oinstall   4096 Oct  8 04:34 31771877
-rw-rw-r-- 1 oracle oinstall 475361 Oct 20 19:54 PatchSearch.xml
drwxr-xr-x 2 oracle dba        4096 Oct  1 09:36 one_offs
-rwxr-xr-- 1 oracle dba        2020 Dec 28 12:44 patchDBBinaries.sh
drwxr-xr-x 2 oracle dba        4096 Dec 25 12:17 release_update
total 100
drwxr-xr-x  5 oracle oinstall  4096 Oct  8 04:34 .
drwxr-xr-x  1 oracle dba       4096 Dec 28 12:46 ..
-rw-r--r--  1 oracle oinstall 73448 Oct  8 04:36 README.html
-rw-r--r--  1 oracle oinstall    21 Oct  8 04:36 README.txt
drwxr-x---  3 oracle oinstall  4096 Oct  8 04:36 custom
drwxr-x---  3 oracle oinstall  4096 Oct  8 04:36 etc
drwxr-x--- 39 oracle oinstall  4096 Oct  8 04:34 files
Running: /opt/oracle/product/19c/dbhome_1/OPatch/opatchauto apply -binary -oh /opt/oracle/product/19c/dbhome_1 /opt/install/patches/31771877
OPATCHAUTO-72156: Patch location not accessible or empty.
OPATCHAUTO-72156: Patch location supplied cannot be accessed or no content found.
OPATCHAUTO-72156: Please provide a correct patch location.
xtender commented 3 years ago

Even with newer OPatch 12.2.0.1.23:

Unzipping /opt/install/patches/release_update/p31771877_190000_Linux-x86-64.zip
/opt/install/patches/one_offs/p6880880_190000_Linux-x86-64.zip
Removing directory /opt/oracle/product/19c/dbhome_1/OPatch
Unzipping OPatch archive /opt/install/patches/one_offs/p6880880_190000_Linux-x86-64.zip to /opt/oracle/product/19c/dbhome_1
Applying Release Update: 31771877 in /opt/install/patches/31771877
Running: cd /opt/install/patches/31771877 && pwd && echo  && ls -la && /opt/oracle/product/19c/dbhome_1/OPatch/opatchauto apply /opt/install/patches/31771877 -binary -oh /opt/oracle/product/19c/dbhome_1 -target_type rac_database
/opt/install/patches/31771877

total 100
drwxr-xr-x  5 oracle oinstall  4096 Oct  8 04:34 .
drwxr-xr-x  1 oracle dba       4096 Dec 30 13:23 ..
-rw-r--r--  1 oracle oinstall 73448 Oct  8 04:36 README.html
-rw-r--r--  1 oracle oinstall    21 Oct  8 04:36 README.txt
drwxr-x---  3 oracle oinstall  4096 Oct  8 04:36 custom
drwxr-x---  3 oracle oinstall  4096 Oct  8 04:36 etc
drwxr-x--- 39 oracle oinstall  4096 Oct  8 04:34 files
OPATCHAUTO-72156: Patch location not accessible or empty.
OPATCHAUTO-72156: Patch location supplied cannot be accessed or no content found.
OPATCHAUTO-72156: Please provide a correct patch location.

 perl patching initializaion failed with error code 255.
rishabh20 commented 3 years ago

Please copy the patch inside extensions/patching/patches/release_update directory and retry. That is what is mentioned in the patching readme as well.

xtender commented 3 years ago

@rishabh20 that's exactly what I did, and you can see it in the log above: 

Unzipping /opt/install/patches/release_update/p31771877_190000_Linux-x86-64.zip
rishabh20 commented 3 years ago

@xtender What is the size of the patch?

xtender commented 3 years ago

@rishabh20 1424692091 bytes:

[root@orasql release_update]# pwd
/home/xtender/github-oracle/docker-images/OracleDatabase/SingleInstance/extensions/patching/patches/release_update
[root@orasql release_update]# ls -l
total 1391308
-rw-rw-r-- 1 oracle 54322 1424692091 Dec 23 20:59 p31771877_190000_Linux-x86-64.zip
rishabh20 commented 3 years ago

@xtender The patch looks okay. I tried following the same steps on my machine and it went through fine. As of now, I don't know what could be the issue here. Please explain how u built the base 19.3 image.

xtender commented 3 years ago

@rishabh20 again, docker builds it, but oracle in resulted image returns still 19.3. Are you sure that your final oracle shows 19.9 in v$instance?

select version_full from v$instance;

VERSION_FULL
-----------------
19.3.0.0.0
xtender commented 3 years ago

Please explain how u built the base 19.3 image.

It was created successfully using ./buildContainerImage.sh from (docker-images/OracleDatabase/SingleInstance/dockerfiles/19.3.0/) from this repo as per instrunction:

./buildContainerImage.sh -v 19.3.0 -e
theBNT commented 3 years ago

I can confirm that i reproduced this issue on current git latest via

the resulting image claims its still 19.3.0 and the patches dont seem to be applied.

When trying via the one_offs p32067171_190000_Linux-x86-64.zip and p32218454_190000_Linux-x86-64.zip (and opatch update p6880880_190000_Linux-x86-64.zip) in /data/docker-images/OracleDatabase/SingleInstance/extensions/patching/patches/one_offs i get the following error during the build:

==Following patches FAILED in apply:

Patch: /opt/install/patches/32218454
Log: /opt/oracle/product/19c/dbhome_1/cfgtoollogs/opatchauto/core/opatch/opatch2021-03-31_19-10-51PM_1.log
Reason: Failed during Analysis: CheckPatchApplicable Failed, [ Prerequisite Status: FAILED, Prerequisite output:
The details are:

Patch 32218454:
Jar Action: Destination File "/opt/oracle/product/19c/dbhome_1/sqldeveloper/sqldeveloper/lib/osdt_cert.jar" does not exists or is not writeable
'oracle.dbdev, 19.0.0.0.0': Cannot update file '/opt/oracle/product/19c/dbhome_1/sqldeveloper/sqldeveloper/lib/osdt_cert.jar' with 'oracle/security/crypto/cert/GeneralName$Type.class'

Used files are:

> ls -la p*
-rw------- 1 user user  125586988 Mar 30 00:42 p32067171_190000_Linux-x86-64.zip
-rw------- 1 user user 1495300727 Mar 30 00:41 p32218454_190000_Linux-x86-64.zip
-rw------- 1 user user 3042388690 Mar 29 12:28 p32226239_190000_Linux-x86-64.zip
-rw------- 1 user user  120761121 Mar 29 11:48 p6880880_190000_Linux-x86-64.zip
> md5sum p*
d58d6a9cd276838648bfda323a8f6119  p32067171_190000_Linux-x86-64.zip
add92eb9abca0f67593ae08e128bc3d4  p32218454_190000_Linux-x86-64.zip
732ba3134a302320930a22676dff82f4  p32226239_190000_Linux-x86-64.zip
9672129ff86a7bc4eddc36097b1ba2ff  p6880880_190000_Linux-x86-64.zip
abhisbyk commented 3 years ago

@theBNT The issue that you are facing can be resolved by passing additional build argument (while creating 19.3 docker image) to make SLIMMING env var false. By default SLIMMING is true. If SLIMMING env var is true some unnecessary components are removed from the image and sqldeveloper is one of them (causing the issue in this case).

Example build command: ./buildContainerImage.sh -i -e -v 19.3.0 -o '--build-arg SLIMMING=false'

abhisbyk commented 3 years ago

@xtender I am not able to reproduce this issue on my machine. I used OPatch 12.2.0.1.24 and I am able to build extended image without any error.

My resulted image is showing 19.9.0.0.0 in v$instance

SQL> select version_full from v$instance;

VERSION_FULL
-----------------
19.9.0.0.0

Not sure what issue is causing this error. Can you pls check logs in $ORACLE_HOME/cfgtoollogs/opatchauto/ directory for some additional information this error ?

abhisbyk commented 3 years ago

@xtender Also can you pls try buildling extension with the 19.3 base image with SLIMMING false as I explained above ?

theBNT commented 3 years ago

@abhisbyk thank you so much, this was indeed the issue and i also found it described only here. Patching now works for me!

The logical followup question is now: how to produce slim(er) patched images? Since all layers still exit, it doesnt make sense to delete the files after patching. The resulting image is 17GB! Docker export/import is not an option because all the metadata is gone. Docker squashing is still in experimental. Will this be addressed by PR #1666 or will patching not work there as well? #1265 is somewhat related.

abhisbyk commented 3 years ago

@theBNT I guess currently patching will not work with the slim image as per the solution suggested by PR #1666. The reason is that cleanup.sh script introduced in the PR will mandatorily remove the components like sqldeveloper, apex etc (as in our case happens when SLIMMING is set). There is no option for not removing these components in that PR, hence, the same error that you were having initially will come.

But the idea of that PR (using compressed RMAN backup for creating the database, while the slimmer docker image only have binaries required) is interesting. Thorough testing is required to check whether this will be able to reduce the size of patched image.

abhisbyk commented 3 years ago

@xtender Any update on the issue that you are facing ?

Closing this issue for now as not able to replicate it on the local development environment.