search

openssl / web

www.openssl.org
51 stars 77 forks source link

How to build the openssl website? #477

Open hwhsu1231 opened 3 months ago

hwhsu1231 commented 3 months ago

Problem Description

Hello, OpenSSL Maintainance Team.

After reading the README.md of this repository, I have so many questions about how to build the OpenSSL website:

  1. Where are build_doc.sh and build_web.sh?
  2. How to install OpenSSL::Query? (since its hyperlink is invalid)
  3. Where is omc/data repository?
  4. How to prepare those CHECKOUTS?

About the 4th question, except omc/data, does it mean that I should run the following commands:

git clone --depth=1 https://github.com/openssl/web.git
cd web
git clone --depth=1 https://github.com/openssl/general-policies.git general-policies
git clone --depth=1 https://github.com/openssl/technical-policies.git technical-policies
git clone --depth=1 --branch=master https://github.com/openssl/openssl.git openssl
git clone --depth=1 --branch=openssl-3.1 https://github.com/openssl/openssl.git openssl-3.1
git clone --depth=1 --branch=openssl-3.0 https://github.com/openssl/openssl.git openssl-3.0
git clone --depth=1 --branch=OpenSSL_1_1_1-stable https://github.com/openssl/openssl.git openssl-1.1.1-stable
ls -l
Click to expand the log ```bash hwhsu1231@vb-kubuntu:~/Repo/testing$ git clone --depth=1 https://github.com/openssl/web.git Cloning into 'web'... remote: Enumerating objects: 184, done. remote: Counting objects: 100% (184/184), done. remote: Compressing objects: 100% (176/176), done. remote: Total 184 (delta 9), reused 119 (delta 3), pack-reused 0 Receiving objects: 100% (184/184), 6.46 MiB | 3.05 MiB/s, done. Resolving deltas: 100% (9/9), done. hwhsu1231@vb-kubuntu:~/Repo/testing$ cd web hwhsu1231@vb-kubuntu:~/Repo/testing/web$ git clone --depth=1 https://github.com/openssl/general-policies.git general-policies Cloning into 'general-policies'... remote: Enumerating objects: 60, done. remote: Counting objects: 100% (60/60), done. remote: Compressing objects: 100% (53/53), done. remote: Total 60 (delta 6), reused 35 (delta 5), pack-reused 0 Receiving objects: 100% (60/60), 46.53 KiB | 952.00 KiB/s, done. Resolving deltas: 100% (6/6), done. hwhsu1231@vb-kubuntu:~/Repo/testing/web$ git clone --depth=1 https://github.com/openssl/technical-policies.git technical-policies Cloning into 'technical-policies'... remote: Enumerating objects: 78, done. remote: Counting objects: 100% (78/78), done. remote: Compressing objects: 100% (62/62), done. remote: Total 78 (delta 16), reused 60 (delta 14), pack-reused 0 Receiving objects: 100% (78/78), 46.15 KiB | 926.00 KiB/s, done. Resolving deltas: 100% (16/16), done. hwhsu1231@vb-kubuntu:~/Repo/testing/web$ git clone --depth=1 --branch=master https://github.com/openssl/openssl.git openssl Cloning into 'openssl'... remote: Enumerating objects: 5509, done. remote: Counting objects: 100% (5509/5509), done. remote: Compressing objects: 100% (4841/4841), done. remote: Total 5509 (delta 1039), reused 2121 (delta 606), pack-reused 0 Receiving objects: 100% (5509/5509), 19.18 MiB | 2.98 MiB/s, done. Resolving deltas: 100% (1039/1039), done. hwhsu1231@vb-kubuntu:~/Repo/testing/web$ git clone --depth=1 --branch=openssl-3.1 https://github.com/openssl/openssl.git openssl-3.1 Cloning into 'openssl-3.1'... remote: Enumerating objects: 24626, done. remote: Counting objects: 100% (24626/24626), done. remote: Compressing objects: 100% (19675/19675), done. remote: Total 24626 (delta 1906), reused 21023 (delta 1495), pack-reused 0 Receiving objects: 100% (24626/24626), 22.35 MiB | 3.23 MiB/s, done. Resolving deltas: 100% (1906/1906), done. hwhsu1231@vb-kubuntu:~/Repo/testing/web$ git clone --depth=1 --branch=openssl-3.0 https://github.com/openssl/openssl.git openssl-3.0 Cloning into 'openssl-3.0'... remote: Enumerating objects: 24500, done. remote: Counting objects: 100% (24500/24500), done. remote: Compressing objects: 100% (19566/19566), done. remote: Total 24500 (delta 1871), reused 21040 (delta 1481), pack-reused 0 Receiving objects: 100% (24500/24500), 21.98 MiB | 3.00 MiB/s, done. Resolving deltas: 100% (1871/1871), done. hwhsu1231@vb-kubuntu:~/Repo/testing/web$ git clone --depth=1 --branch=OpenSSL_1_1_1-stable https://github.com/openssl/openssl.git openssl-1.1.1-stable Cloning into 'openssl-1.1.1-stable'... remote: Enumerating objects: 18408, done. remote: Counting objects: 100% (18408/18408), done. remote: Compressing objects: 100% (15196/15196), done. remote: Total 18408 (delta 963), reused 15616 (delta 679), pack-reused 0 Receiving objects: 100% (18408/18408), 14.55 MiB | 3.03 MiB/s, done. Resolving deltas: 100% (963/963), done. hwhsu1231@vb-kubuntu:~/Repo/testing/web$ ls -l total 128 -rw-rw-r-- 1 hwhsu1231 hwhsu1231 4808 3月 28 15:32 about.md drwxrwxr-x 2 hwhsu1231 hwhsu1231 4096 3月 28 15:32 bin drwxrwxr-x 2 hwhsu1231 hwhsu1231 4096 3月 28 15:32 community -rw-rw-r-- 1 hwhsu1231 hwhsu1231 529 3月 28 15:32 dirdata.yaml drwxrwxr-x 3 hwhsu1231 hwhsu1231 4096 3月 28 15:32 docs -rw-rw-r-- 1 hwhsu1231 hwhsu1231 356 3月 28 15:32 err404.md -rw-rw-r-- 1 hwhsu1231 hwhsu1231 6518 3月 28 15:32 favicon.ico drwxrwxr-x 7 hwhsu1231 hwhsu1231 4096 3月 28 15:32 general-policies drwxrwxr-x 3 hwhsu1231 hwhsu1231 4096 3月 28 15:32 img drwxrwxr-x 3 hwhsu1231 hwhsu1231 4096 3月 28 15:32 inc -rw-rw-r-- 1 hwhsu1231 hwhsu1231 1337 3月 28 15:32 index.md.tt -rw-rw-r-- 1 hwhsu1231 hwhsu1231 25135 3月 28 15:32 Makefile drwxrwxr-x 2 hwhsu1231 hwhsu1231 4096 3月 28 15:32 news drwxrwxr-x 33 hwhsu1231 hwhsu1231 4096 3月 28 15:33 openssl drwxrwxr-x 24 hwhsu1231 hwhsu1231 4096 3月 28 15:34 openssl-1.1.1-stable drwxrwxr-x 26 hwhsu1231 hwhsu1231 4096 3月 28 15:33 openssl-3.0 drwxrwxr-x 30 hwhsu1231 hwhsu1231 4096 3月 28 15:33 openssl-3.1 drwxrwxr-x 4 hwhsu1231 hwhsu1231 4096 3月 28 15:32 policies -rw-rw-r-- 1 hwhsu1231 hwhsu1231 3653 3月 28 15:32 README.md -rw-rw-r-- 1 hwhsu1231 hwhsu1231 2249 3月 28 15:32 roadmap.md -rw-rw-r-- 1 hwhsu1231 hwhsu1231 104 3月 28 15:32 robots.txt drwxrwxr-x 4 hwhsu1231 hwhsu1231 4096 3月 28 15:32 source drwxrwxr-x 2 hwhsu1231 hwhsu1231 4096 3月 28 15:32 support drwxrwxr-x 6 hwhsu1231 hwhsu1231 4096 3月 28 15:32 technical-policies hwhsu1231@vb-kubuntu:~/Repo/testing/web$ ```

And the run the following make command:

make CHECKOUTS=$(pwd)

Correct?

levitte commented 3 months ago
  1. Those build scripts are internal to our infrastructure. They essentially run make, but with specific targets that only builds the simple web files, or that only builds the docs.
  2. The link to OpenSSL::Query is incorrect, it should be https://github.com/openssl/tools/tree/master/OpenSSL-Query
  3. omc/data is an internal repository, and should not be to build. OpenSSL::Query does the job of actually getting most of the the necessary data. newsflash.txt is an exception... for now, I suggest doing mkdir data; touch data/newsflash.txt.
  4. You got the 4th question right.
hwhsu1231 commented 3 months ago

Hello, @levitte

After preparing the prerequisites, I tried to build the openssl website with the following commands:

git clone --depth=1 https://github.com/openssl/web.git
cd web
git clone --depth=1 https://github.com/openssl/general-policies.git general-policies
git clone --depth=1 https://github.com/openssl/technical-policies.git technical-policies
git clone --depth=1 --branch=master https://github.com/openssl/openssl.git openssl
git clone --depth=1 --branch=openssl-3.1 https://github.com/openssl/openssl.git openssl-3.1
git clone --depth=1 --branch=openssl-3.0 https://github.com/openssl/openssl.git openssl-3.0
git clone --depth=1 --branch=OpenSSL_1_1_1-stable https://github.com/openssl/openssl.git openssl-1.1.1-stable
mkdir data; touch data/newsflash.txt
make CHECKOUTS=$(pwd)

However, it stoped at an error message:

make: *** No rule to make target '/home/hwhsu1231/Repo/testing/web/openssl-3.3/NEWS.md', needed by 'news/openssl-3.3-notes.inc'.  Stop.
Click to expand the log ```bash hwhsu1231@vb-kubuntu:~/Repo/testing$ git clone --depth=1 https://github.com/openssl/web.git Cloning into 'web'... remote: Enumerating objects: 184, done. remote: Counting objects: 100% (184/184), done. remote: Compressing objects: 100% (176/176), done. remote: Total 184 (delta 9), reused 119 (delta 3), pack-reused 0 Receiving objects: 100% (184/184), 6.46 MiB | 3.04 MiB/s, done. Resolving deltas: 100% (9/9), done. hwhsu1231@vb-kubuntu:~/Repo/testing$ cd web hwhsu1231@vb-kubuntu:~/Repo/testing/web$ git clone --depth=1 https://github.com/openssl/general-policies.git general-policies Cloning into 'general-policies'... remote: Enumerating objects: 60, done. remote: Counting objects: 100% (60/60), done. remote: Compressing objects: 100% (53/53), done. remote: Total 60 (delta 6), reused 35 (delta 5), pack-reused 0 Receiving objects: 100% (60/60), 46.53 KiB | 916.00 KiB/s, done. Resolving deltas: 100% (6/6), done. hwhsu1231@vb-kubuntu:~/Repo/testing/web$ git clone --depth=1 https://github.com/openssl/technical-policies.git technical-policies Cloning into 'technical-policies'... remote: Enumerating objects: 78, done. remote: Counting objects: 100% (78/78), done. remote: Compressing objects: 100% (62/62), done. remote: Total 78 (delta 16), reused 60 (delta 14), pack-reused 0 Receiving objects: 100% (78/78), 46.15 KiB | 964.00 KiB/s, done. Resolving deltas: 100% (16/16), done. hwhsu1231@vb-kubuntu:~/Repo/testing/web$ git clone --depth=1 --branch=master https://github.com/openssl/openssl.git openssl Cloning into 'openssl'... remote: Enumerating objects: 5509, done. remote: Counting objects: 100% (5509/5509), done. remote: Compressing objects: 100% (4841/4841), done. remote: Total 5509 (delta 1039), reused 2121 (delta 606), pack-reused 0 Receiving objects: 100% (5509/5509), 19.18 MiB | 3.28 MiB/s, done. Resolving deltas: 100% (1039/1039), done. hwhsu1231@vb-kubuntu:~/Repo/testing/web$ git clone --depth=1 --branch=openssl-3.1 https://github.com/openssl/openssl.git openssl-3.1 Cloning into 'openssl-3.1'... remote: Enumerating objects: 24626, done. remote: Counting objects: 100% (24626/24626), done. remote: Compressing objects: 100% (19546/19546), done. remote: Total 24626 (delta 1936), reused 22089 (delta 1624), pack-reused 0 Receiving objects: 100% (24626/24626), 22.36 MiB | 3.23 MiB/s, done. Resolving deltas: 100% (1936/1936), done. hwhsu1231@vb-kubuntu:~/Repo/testing/web$ git clone --depth=1 --branch=openssl-3.0 https://github.com/openssl/openssl.git openssl-3.0 Cloning into 'openssl-3.0'... remote: Enumerating objects: 24500, done. remote: Counting objects: 100% (24500/24500), done. remote: Compressing objects: 100% (19566/19566), done. remote: Total 24500 (delta 1871), reused 21040 (delta 1481), pack-reused 0 Receiving objects: 100% (24500/24500), 21.98 MiB | 3.21 MiB/s, done. Resolving deltas: 100% (1871/1871), done. hwhsu1231@vb-kubuntu:~/Repo/testing/web$ git clone --depth=1 --branch=OpenSSL_1_1_1-stable https://github.com/openssl/openssl.git openssl-1.1.1-stable Cloning into 'openssl-1.1.1-stable'... remote: Enumerating objects: 18408, done. remote: Counting objects: 100% (18408/18408), done. remote: Compressing objects: 100% (15196/15196), done. remote: Total 18408 (delta 963), reused 15616 (delta 679), pack-reused 0 Receiving objects: 100% (18408/18408), 14.55 MiB | 3.03 MiB/s, done. Resolving deltas: 100% (963/963), done. hwhsu1231@vb-kubuntu:~/Repo/testing/web$ mkdir data; touch data/newsflash.txt hwhsu1231@vb-kubuntu:~/Repo/testing/web$ make CHECKOUTS=$(pwd) /bin/sh: 1: cd: can't cd to /home/hwhsu1231/Repo/testing/web/data/secadv/ /bin/sh: 1: cd: can't cd to /home/hwhsu1231/Repo/testing/web/data/vulnerabilities-json/ /bin/sh: 1: cd: can't cd to /home/hwhsu1231/Repo/testing/web/data/secadv/ /bin/sh: 1: cd: can't cd to /home/hwhsu1231/Repo/testing/web/data/vulnerabilities-json/ pandoc --print-default-template=html5 \ | ./bin/mk-pandoc-template html5 > inc/pandoc-template.html5 ./bin/md-to-html5 about.md ./bin/md-to-html5 err404.md ./bin/md-to-html5 roadmap.md sed news/newsflash.inc \ -e '/^#/d' \ -e 's@^@@' \ -e 's@: @@' \ -e 's@$@@' head -7 news/newsflash.inc >newsflash.inc ./bin/from-tt index.md.tt ./bin/md-to-html5 index.md ./bin/md-to-html5 community/binaries.md ./bin/md-to-html5 community/conduct.md ./bin/md-to-html5 community/contacts.md ./bin/md-to-html5 community/getting-started.md ./bin/md-to-html5 community/index.md ./bin/md-to-html5 community/mailinglists.md ./bin/md-to-html5 community/mission.md wget -q https://api.openssl.org/0/Group/commit/Members ./bin/mk-committers community/committers.inc ./bin/from-tt community/committers.md.tt ./bin/md-to-html5 community/committers.md ./bin/mk-omc -n -l -t 'OMC Alumni' omc-alumni omc-emeritus > community/omc-alumni.inc ./bin/from-tt community/omc-alumni.md.tt ./bin/md-to-html5 community/omc-alumni.md ./bin/mk-omc -n -e -l -p -t 'OMC Members' omc omc-inactive > community/omc.inc ./bin/from-tt community/omc.md.tt ./bin/md-to-html5 community/omc.md ./bin/mk-omc -n -p -t 'OTC Members' otc otc-inactive > community/otc.inc Use of uninitialized value in concatenation (.) or string at /usr/local/share/perl/5.34.0/OpenSSL/Query/PersonREST.pm line 75. Use of uninitialized value in concatenation (.) or string at /usr/local/share/perl/5.34.0/OpenSSL/Query/PersonREST.pm line 75. Use of uninitialized value in concatenation (.) or string at /usr/local/share/perl/5.34.0/OpenSSL/Query/PersonREST.pm line 75. Use of uninitialized value in concatenation (.) or string at /usr/local/share/perl/5.34.0/OpenSSL/Query/PersonREST.pm line 75. Use of uninitialized value in concatenation (.) or string at /usr/local/share/perl/5.34.0/OpenSSL/Query/PersonREST.pm line 75. Use of uninitialized value in concatenation (.) or string at /usr/local/share/perl/5.34.0/OpenSSL/Query/PersonREST.pm line 75. Use of uninitialized value in concatenation (.) or string at /usr/local/share/perl/5.34.0/OpenSSL/Query/PersonREST.pm line 75. Use of uninitialized value in concatenation (.) or string at /usr/local/share/perl/5.34.0/OpenSSL/Query/PersonREST.pm line 75. ./bin/from-tt community/otc.md.tt ./bin/md-to-html5 community/otc.md make: *** No rule to make target '/home/hwhsu1231/Repo/testing/web/openssl-3.3/NEWS.md', needed by 'news/openssl-3.3-notes.inc'. Stop. hwhsu1231@vb-kubuntu:~/Repo/testing/web$ ```
levitte commented 3 months ago

Ah, you need these too:

git clone --depth=1 --branch=openssl-3.2 https://github.com/openssl/openssl.git openssl-3.2
git clone --depth=1 --branch=openssl-3.3 https://github.com/openssl/openssl.git openssl-3.3
hwhsu1231 commented 3 months ago

Hello, @levitte

Ah, you need these too:

git clone --depth=1 --branch=openssl-3.2 https://github.com/openssl/openssl.git openssl-3.2
git clone --depth=1 --branch=openssl-3.3 https://github.com/openssl/openssl.git openssl-3.3

This time, this error showed up:

make: *** No rule to make target '/home/hwhsu1231/Repo/testing/web/data/vulnerabilities-json/.github/workflows/doc_manual.yml', needed by 'news/secjson/.github/workflows/doc_manual.yml'.  Stop.
Click to expand the log ```bash hwhsu1231@vb-kubuntu:~/Repo/testing/web$ git clone --depth=1 --branch=openssl-3.2 https://github.com/openssl/openssl.git openssl-3.2 Cloning into 'openssl-3.2'... remote: Enumerating objects: 5386, done. remote: Counting objects: 100% (5386/5386), done. remote: Compressing objects: 100% (4725/4725), done. remote: Total 5386 (delta 1010), reused 2078 (delta 599), pack-reused 0 Receiving objects: 100% (5386/5386), 18.88 MiB | 2.57 MiB/s, done. Resolving deltas: 100% (1010/1010), done. hwhsu1231@vb-kubuntu:~/Repo/testing/web$ git clone --depth=1 --branch=openssl-3.3 https://github.com/openssl/openssl.git openssl-3.3 Cloning into 'openssl-3.3'... remote: Enumerating objects: 5509, done. remote: Counting objects: 100% (5509/5509), done. remote: Compressing objects: 100% (4839/4839), done. remote: Total 5509 (delta 1038), reused 2080 (delta 608), pack-reused 0 Receiving objects: 100% (5509/5509), 19.19 MiB | 2.74 MiB/s, done. Resolving deltas: 100% (1038/1038), done. hwhsu1231@vb-kubuntu:~/Repo/testing/web$ make CHECKOUTS=$(pwd) /bin/sh: 1: cd: can't cd to /home/hwhsu1231/Repo/testing/web/data/secadv/ /bin/sh: 1: cd: can't cd to /home/hwhsu1231/Repo/testing/web/data/vulnerabilities-json/ /bin/sh: 1: cd: can't cd to /home/hwhsu1231/Repo/testing/web/data/secadv/ /bin/sh: 1: cd: can't cd to /home/hwhsu1231/Repo/testing/web/data/vulnerabilities-json/ wget -q https://api.openssl.org/0/Group/commit/Members ./bin/mk-committers community/committers.inc ./bin/from-tt community/committers.md.tt ./bin/md-to-html5 community/committers.md ./bin/mk-omc -n -l -t 'OMC Alumni' omc-alumni omc-emeritus > community/omc-alumni.inc ./bin/from-tt community/omc-alumni.md.tt ./bin/md-to-html5 community/omc-alumni.md ./bin/mk-omc -n -e -l -p -t 'OMC Members' omc omc-inactive > community/omc.inc ./bin/from-tt community/omc.md.tt ./bin/md-to-html5 community/omc.md ./bin/mk-omc -n -p -t 'OTC Members' otc otc-inactive > community/otc.inc Use of uninitialized value in concatenation (.) or string at /usr/local/share/perl/5.34.0/OpenSSL/Query/PersonREST.pm line 75. Use of uninitialized value in concatenation (.) or string at /usr/local/share/perl/5.34.0/OpenSSL/Query/PersonREST.pm line 75. Use of uninitialized value in concatenation (.) or string at /usr/local/share/perl/5.34.0/OpenSSL/Query/PersonREST.pm line 75. Use of uninitialized value in concatenation (.) or string at /usr/local/share/perl/5.34.0/OpenSSL/Query/PersonREST.pm line 75. Use of uninitialized value in concatenation (.) or string at /usr/local/share/perl/5.34.0/OpenSSL/Query/PersonREST.pm line 75. Use of uninitialized value in concatenation (.) or string at /usr/local/share/perl/5.34.0/OpenSSL/Query/PersonREST.pm line 75. Use of uninitialized value in concatenation (.) or string at /usr/local/share/perl/5.34.0/OpenSSL/Query/PersonREST.pm line 75. Use of uninitialized value in concatenation (.) or string at /usr/local/share/perl/5.34.0/OpenSSL/Query/PersonREST.pm line 75. ./bin/from-tt community/otc.md.tt ./bin/md-to-html5 community/otc.md ./bin/mk-notes 3.3 < /home/hwhsu1231/Repo/testing/web/openssl-3.3/NEWS.md > news/openssl-3.3-notes.inc ./bin/from-tt -d news -i news/openssl-notes.md.tt -o news/openssl-3.3-notes.md release='3.3' ./bin/md-to-html5 news/openssl-3.3-notes.md ./bin/mk-notes 3.2 < /home/hwhsu1231/Repo/testing/web/openssl-3.2/NEWS.md > news/openssl-3.2-notes.inc ./bin/from-tt -d news -i news/openssl-notes.md.tt -o news/openssl-3.2-notes.md release='3.2' ./bin/md-to-html5 news/openssl-3.2-notes.md ./bin/mk-notes 3.1 < /home/hwhsu1231/Repo/testing/web/openssl-3.1/NEWS.md > news/openssl-3.1-notes.inc ./bin/from-tt -d news -i news/openssl-notes.md.tt -o news/openssl-3.1-notes.md release='3.1' ./bin/md-to-html5 news/openssl-3.1-notes.md ./bin/mk-notes 3.0 < /home/hwhsu1231/Repo/testing/web/openssl-3.0/NEWS.md > news/openssl-3.0-notes.inc ./bin/from-tt -d news -i news/openssl-notes.md.tt -o news/openssl-3.0-notes.md release='3.0' ./bin/md-to-html5 news/openssl-3.0-notes.md ./bin/md-to-html5 news/fips-cve.md ./bin/md-to-html5 news/index.md cp /home/hwhsu1231/Repo/testing/web/openssl/CHANGES.md news/changelog.txt (echo 'Table of contents'; sed -e '1,/^OpenSSL Releases$/d' < news/changelog.txt) \ | pandoc -t html5 -f commonmark | ./bin/post-process-html5 >news/changelog.inc ./bin/from-tt 'releases=3.3 3.2 3.1 3.0 ' news/changelog.md.tt cp /home/hwhsu1231/Repo/testing/web/openssl-3.3/CHANGES.md news/cl33.txt cp /home/hwhsu1231/Repo/testing/web/openssl-3.2/CHANGES.md news/cl32.txt cp /home/hwhsu1231/Repo/testing/web/openssl-3.1/CHANGES.md news/cl31.txt cp /home/hwhsu1231/Repo/testing/web/openssl-3.0/CHANGES.md news/cl30.txt ./bin/md-to-html5 news/changelog.md ./bin/from-tt news/newslog.md.tt ./bin/md-to-html5 news/newslog.md ./bin/from-tt news/pgpkey.md.tt ./bin/md-to-html5 news/pgpkey.md mkdir -p news/secjson make: *** No rule to make target '/home/hwhsu1231/Repo/testing/web/data/vulnerabilities-json/.github/workflows/doc_manual.yml', needed by 'news/secjson/.github/workflows/doc_manual.yml'. Stop. hwhsu1231@vb-kubuntu:~/Repo/testing/web$ ```