rm or formalize dealing with "general" KEYS files in our dist dir [LUCENE-5143]

asfimport commented 11 years ago

At some point in the past, we started creating a snapshots of KEYS (taken from the auto-generated data from id.apache.org) in the release dir of each release...

http://www.apache.org/dist/lucene/solr/4.4.0/KEYS http://www.apache.org/dist/lucene/java/4.4.0/KEYS http://archive.apache.org/dist/lucene/java/4.3.0/KEYS http://archive.apache.org/dist/lucene/solr/4.3.0/KEYS etc...

But we also still have some "general" KEYS files...

https://www.apache.org/dist/lucene/KEYS https://www.apache.org/dist/lucene/java/KEYS https://www.apache.org/dist/lucene/solr/KEYS

...which (as i discovered when i went to add my key to them today) are stale and don't seem to be getting updated.

I vaguely remember someone (rmuir?) explaining to me at one point the reason we started creating a fresh copy of KEYS in each release dir, but i no longer remember what they said, and i can't find any mention of a reason in any of the release docs, or in any sort of comment in buildAndPushRelease.py

we probably do one of the following:

remove these "general" KEYS files
add a disclaimer to the top of these files that they are legacy files for verifying old releases and are no longer used for new releases
ensure these files are up to date stop generating per-release KEYS file copies
update our release process to ensure that the general files get updated on each release as well

Migrated from LUCENE-5143 by Chris M. Hostetter (@hossman), resolved Sep 11 2018 Attachments: KEYS (versions: 4), LUCENE_5143_KEYS.patch, LUCENE-5143_READMEs.patch (versions: 3), LUCENE-5143.patch (versions: 4), LUCENE-5143-reopen-smoke.patch (versions: 2), verify.log, verify.sh (versions: 3) Linked issues:

SOLR-5085

asfimport commented 11 years ago

Chris M. Hostetter (@hossman) (migrated from JIRA)

At the moment, there is no special KEYS file for the solr ref guide...

https://www.apache.org/dist/lucene/solr/ref-guide/

...if we decided to delete these "general" KEYS files, then one should be added to that directory, and the doc process needs to ensure it's kept up to day (SOLR-5085)

asfimport commented 11 years ago

Uwe Schindler (@uschindler) (migrated from JIRA)

I did not know this issue, so sorry for the comment today relating to uploading the GPG key of @ctargett to https://id.apache.org (which she should do in any case).

In my opinion we should only have one non-stale KEYS file which is automatically kept up-to-date. An idea to solve that would be something like a symlink (not possible with subversion) or a .htaccess file that rewrites the URL to the one at the autogen-place.

Auto-updating is not a problem, as the ASF guidelines state that the older keys (after a key-change) should still be kept in the Apache LDAP, so if one person have multiple keys (e.g. outdated ones), all of them are kept in the KEYS file.

asfimport commented 7 years ago

Jan Høydahl (@janhoy) (migrated from JIRA)

In my opinion we should only have one non-stale KEYS file which is automatically kept up-to-date.

+1,

I.e. stop publishing KEYS file in the solr, java and version-specific folders. That's what e.g. Hadoop and Ant do, and what is proposed in https://www.apache.org/dev/release-signing.html#keys-policy

asfimport commented 7 years ago

Jan Høydahl (@janhoy) (migrated from JIRA)

To bring closure, I propose the following

We delete these: lucene/java/KEYS, lucene/solr/KEYS, and lucene/solr/KEYS.txt
We update https://archive.apache.org/dist/lucene/KEYS and make sure it contains all keys ever used for signing a release
We stop publishing KEYS to every release dir. Instead the RM will only need to make sure that his/her own key is in the file. Never remove keys, only add.
https://wiki.apache.org/lucene-java/ReleaseTodo should probably be updated to clarify

I think this is in-line with https://www.apache.org/dev/release-signing.html#keys-policy

asfimport commented 7 years ago

Jan Høydahl (@janhoy) (migrated from JIRA)

So I did some more research. I downloaded all the .asc signatures

wget -r -np -k -m -A asc,KEYS https://archive.apache.org/dist/lucene/

It's a big tree, including tika, nutch etc as well. Top of the tree:

├── archive.apache.org
│   └── dist
│       └── lucene
│           ├── KEYS
│           ├── hadoop
│           │   ├── KEYS
│           │   ├── hadoop-0.14.4
│           │   │   └── hadoop-0.14.4.tar.gz.asc
│           │   ├── hadoop-0.15.2
│           │   │   └── hadoop-0.15.2.tar.gz.asc
│           │   └── stable
│           │       └── hadoop-0.14.4.tar.gz.asc
│           ├── java
│           │   ├── 2.9.4
│           │   │   ├── lucene-2.9.4-src.tar.gz.asc
│           │   │   ├── lucene-2.9.4-src.zip.asc
│           │   │   ├── lucene-2.9.4.tar.gz.asc
│           │   │   └── lucene-2.9.4.zip.asc
│           │   ├── 3.0.3
│           │   │   ├── lucene-3.0.3-src.tar.gz.asc
│           │   │   ├── lucene-3.0.3-src.zip.asc
│           │   │   ├── lucene-3.0.3.tar.gz.asc
│           │   │   └── lucene-3.0.3.zip.asc

Next, I found all PGP keys ever used to sign a Lucene / Solr release, abusing the --verify flag without the tarball file:

for f in $(find archive.apache.org/dist/lucene/java/ -name *.asc); do gpg --verify $f /dev/null 2>> java.txt; done
cat java.txt |grep "key ID" | awk -e '{print $14;}' | sort | uniq > java-ids.txt
for f in $(find archive.apache.org/dist/lucene/solr/ -name *.asc); do echo $f; gpg --verify $f /dev/null 2>> solr.txt; done
cat solr.txt |grep "key ID" | awk -e '{print $14;}' | sort | uniq > solr-ids.txt

Now the java-ids.txt file contains all unique PGP key IDs used to sign Lucene releases, and the same for solr-ids.txt. Then I like to import all these keys to my keychain and list who have ever signed a release:

for proj in java solr; do echo "$proj-keys.txt"; for k in $(cat $proj.txt |grep "key ID" | awk -e '{print $14;}' | sort|uniq); do gpg --recv-keys 0x$k 2>&1 >/dev/null | grep "gpg: key " >> $proj-keys.txt ; done ; done
$ cat java-keys.txt 
gpg: key 0AFCEE7C: "Yonik Seeley <yonik@apache.org>" not changed
gpg: key 1EFAFD39: "Ryan Ernst <ryan@iernst.net>" not changed
gpg: key 13E57FFC: "Upayavira <uv@odoko.co.uk>" not changed
gpg: key 177050F6: "Shalin Shekhar Mangar (CODE SIGNING KEY) <shalin@apache.org>" not changed
gpg: key 1ED139E7: "Steven Alan Rowe (CODE SIGNING KEY) <sarowe@apache.org>" not changed
gpg: key 322D7ECA: "Robert Muir (Code Signing Key) <rmuir@apache.org>" not changed
gpg: key 3A95B305: "Christoph Goller (CODE SIGNING KEY) <goller@apache.org>" not changed
gpg: key 3FCFDB3E: "Noble Paul (CODE SIGNING KEY) <noble@apache.org>" not changed
gpg: key 6BD872A0: "Michael Busch (Lucene Committer) <buschmi@apache.org>" not changed
gpg: key 6BD872A0: "Michael Busch (Lucene Committer) <buschmi@apache.org>" not changed
gpg: key 6E68DA61: "Michael McCandless (CODE SIGNING KEY) <mikemccand@apache.org>" not changed
gpg: key 6FDB8105: "Jim Ferenczi (CODE SIGNING KEY) <jimczi@apache.org>" not changed
gpg: key 76BC6507: "Adrien Grand (CODE SIGNING KEY) <jpountz@apache.org>" not changed
gpg: key 78796AC8: "Mark Robert Miller (CODE SIGNING KEY) <markrmiller@apache.org>" not changed
gpg: key A3A13A7F: "Anshum Gupta <anshum@apache.org>" not changed
gpg: key A7239D59: "Doug Cutting (Lucene guy) <cutting@apache.org>" not changed
gpg: key A7239D59: "Doug Cutting (Lucene guy) <cutting@apache.org>" not changed
gpg: key E1EE085F: "Uwe Schindler (CODE SIGNING KEY) <uschindler@apache.org>" not changed
gpg: key EB0199F8: "Mark Robert Miller (CODE SIGNING KEY) <markrmiller@apache.org>" not changed
gpg: key ECA39416: "Simon Willnauer (Code Signing Key) <simonw@apache.org>" not changed
gpg: key F625308A: "Nicholas Walter Knize (CODE SIGNING KEY) <nknize@apache.org>" not changed
gpg: key F6ED44FC: "Timothy Potter <thelabdude@apache.org>" not changed
gpg: key FE045966: "Grant Ingersoll (CODE SIGNING KEY) <gsingers@apache.org>" not changed
$ cat solr-keys.txt 
gpg: key 0AFCEE7C: "Yonik Seeley <yonik@apache.org>" not changed
gpg: key 1EFAFD39: "Ryan Ernst <ryan@iernst.net>" not changed
gpg: key 13E57FFC: "Upayavira <uv@odoko.co.uk>" not changed
gpg: key 177050F6: "Shalin Shekhar Mangar (CODE SIGNING KEY) <shalin@apache.org>" not changed
gpg: key 1ED139E7: "Steven Alan Rowe (CODE SIGNING KEY) <sarowe@apache.org>" not changed
gpg: key 322D7ECA: "Robert Muir (Code Signing Key) <rmuir@apache.org>" not changed
gpg: key 3FCFDB3E: "Noble Paul (CODE SIGNING KEY) <noble@apache.org>" not changed
gpg: key 6E68DA61: "Michael McCandless (CODE SIGNING KEY) <mikemccand@apache.org>" not changed
gpg: key 6FDB8105: "Jim Ferenczi (CODE SIGNING KEY) <jimczi@apache.org>" not changed
gpg: key 76BC6507: "Adrien Grand (CODE SIGNING KEY) <jpountz@apache.org>" not changed
gpg: key 78796AC8: "Mark Robert Miller (CODE SIGNING KEY) <markrmiller@apache.org>" not changed
gpg: key 521A0277: "Cassandra Targett <casstargett@gmail.com>" not changed
gpg: key A3A13A7F: "Anshum Gupta <anshum@apache.org>" not changed
gpg: key A867E8B1: "Grant Ingersoll (CODE SIGNING KEY) <gsingers@apache.org>" not changed
gpg: key E1EE085F: "Uwe Schindler (CODE SIGNING KEY) <uschindler@apache.org>" not changed
gpg: key ECA39416: "Simon Willnauer (Code Signing Key) <simonw@apache.org>" not changed
gpg: key F625308A: "Nicholas Walter Knize (CODE SIGNING KEY) <nknize@apache.org>" not changed
gpg: key F6ED44FC: "Timothy Potter <thelabdude@apache.org>" not changed
gpg: key F8F58E19: "Chris "Hoss" Hostetter <hossman@apache.org>" not changed
gpg: key FE045966: "Grant Ingersoll (CODE SIGNING KEY) <gsingers@apache.org>" not changed

The overlap is not surprising, and you can see some differences related to pre-3.0 era.

We will now test if all the keys for historic RMs are present in the various KEYS files in the mirrors:

cp archive.apache.org/dist/lucene/KEYS KEYS-lucene.txt && cp archive.apache.org/dist/lucene/java/KEYS KEYS-java.txt && cp archive.apache.org/dist/lucene/solr/KEYS KEYS-solr.txt && cp archive.apache.org/dist/lucene/solr/6.4.1/KEYS KEYS-641.txt
for keys in KEYS-*.txt; do echo match_$keys; for id in $(cat *-ids.txt|sort|uniq); do echo "Looking for id $(cat *-keys.txt |sort|uniq| grep $id)"; grep -E "$id|$(echo $id|echo $id|awk -e '{print substr($_,0,4)" "substr($_,5,4);}')" $keys ; done >match_$keys ; done

The result is depressing. lucene/KEYS lack 14 keys, lucene/java/KEYS lack 12 keys, lucene/solr/KEYS lack 11 keys and lucene/*/6.4.1/KEYS lack 6 keys.

The missing keys in the latest 6.4.1 KEYS file (assuming it is auto generated from http://people.apache.org/keys/group/lucene.asc) are :

0AFCEE7C: "Yonik Seeley <yonik@apache.org>" (OLD KEY from 2007, probably not listed in LDAP anymore)
13E57FFC: "Upayavira <uv@odoko.co.uk>" (INVALID KEY IN LDAP, see https://people.apache.org/keys/committer/)
3A95B305: "Christoph Goller (CODE SIGNING KEY) <goller@apache.org>" (Not committer? - key not in LDAP)
521A0277: "Cassandra Targett <casstargett@gmail.com>" (INVALID KEY IN LDAP, see https://people.apache.org/keys/committer/)
A867E8B1: "Grant Ingersoll (CODE SIGNING KEY) <gsingers@apache.org>" (OLD KEY from 2008)
EB0199F8: "Mark Robert Miller (CODE SIGNING KEY) <markrmiller@apache.org>" (OLD KEY from 2009)

So if the goal is to have a KEYS file available from which one can verify any release, then we could simply generate one from IDs we know have been used by RMs:

for id in $(cat solr-ids.txt java-ids.txt|sort|uniq); do echo "Exporting id $id - $(grep $id lucene-solr-keys.txt)"; gpg --fingerprint $id >> KEYS_NEW; gpg --export -a $id >> KEYS ; done

And then one of the responsibilities of new RMs is to add their key to this one KEYS file.

asfimport commented 7 years ago

Jan Høydahl (@janhoy) (migrated from JIRA)

Or more automatic, we can make a static file OLD_KEYS from the key IDs 0AFCEE7C, 3A95B305, A867E8B1 and EB0199F8 which we concatenate with the current LDAP export as part of the release process. Ping @upayavira and @ctargett, you should update your info in http://id.apache.org with the correct 40-char PGP key fingerprint.

asfimport commented 7 years ago

Cassandra Targett (@ctargett) (migrated from JIRA)

Maybe or maybe not coincidentally, I got a similar request from Sebb to update my key to the fingerprint. I've done it now.

asfimport commented 7 years ago

Jan Høydahl (@janhoy) (migrated from JIRA)

Question: How does the auto-generated KEYS file end up in the version-specifig release dir? Cannot see anything about it in the ReleaseTodo?

asfimport commented 7 years ago

Ishan Chattopadhyaya (@chatman) (migrated from JIRA)

Question: How does the auto-generated KEYS file end up in the version-specifig release dir? Cannot see anything about it in the ReleaseTodo?

@janhoy, I was wondering the same. @sarowe informed me, "look at line 152 of dev-tools/scripts/buildAndPushRelease.py"

asfimport commented 7 years ago

Jan Høydahl (@janhoy) (migrated from JIRA)

Ah, thanks :)

I think we should stop copying this file to KEYS - see the last chapter of this page http://home.apache.org/keys/ - it says explicitly that it cannot be used as-is for a KEYS file since it will not contain all needed keys.

So if we agree that we do not need KEYS file for each version release, only one on top-level, then we can add to the Release TODO a bullet about checking that your own key is in the current KEYS file, and if not add it. It should be sufficient, since we only require keys of committers who have signed a release, so an RM who has done a release simply needs to verify that his current key is on the list.

asfimport commented 7 years ago

Jan Høydahl (@janhoy) (migrated from JIRA)

Attaching a patch against DIST site which updates the README.html files

Clone Lucene's README for Solr, remove old HEADER.html, so it will display below the download links, not above
For all three sub folders java, pylucene and solr, add a paragraph "Use a mirror" to encourage downloading from mirrors
Added logos to the README.html files (where to find a smaller official Solr logo online?)
Remove warning about tar for macOS, as this is not a problem anymore
Revise the "Signatures" paragraph to a "Signatures and hashes" paragraph also describing what .md5 and .sha1 files are for
Rewrite the Signatures paragraph
- It was still talking about binaries/ and source/ folders
- Make it clear that the .asc file must be downloaded from apache site
- Use x.y.z instead of random version number in example commands
- Changed link to the official apache dist to be HTTPS and write a note about always downloading KEYS file over HTTPS

There will be a separate git patch for buildAndPushRelease.py to stop publishing KEYS inside release folders.

asfimport commented 7 years ago

Jan Høydahl (@janhoy) (migrated from JIRA)

This patch fixes buildAndPushRelease.py so that it will not push KEYS inside versioned release folder anymore.

It also adds an automatic check that the committer's key used for signing is actually present on the KEYS file. It does this by fetching the KEYS file and then finding the key ID using a regex.

Question is if we should still publish java/KEYS and solr/KEYS, in which case we could discontinue also the root one) or if both these README's should refer to the common file in the dist root?

asfimport commented 7 years ago

Jan Høydahl (@janhoy) (migrated from JIRA)

So what do people think about the README.html changes?

I think perhaps we should use the KEYS files from the sub folders (solr/KEYS, java/KEYS, pylucene/KEYS) and remove the topmost one? Or should solr/lucene instead share the one topmost /KEYS file?

Regarding the buildAndPushRelease changes, I think there should also be an offline mode that will either skip verification or verify against a local dist checkout.

So the next step would be to commit these patches and then commit a consolidated KEYS file (or two identical in java/ and solr/).

asfimport commented 7 years ago

Jan Høydahl (@janhoy) (migrated from JIRA)

@hossman, @uschindler, @anshumg - any comments to my two patches in this issue?

asfimport commented 7 years ago

Uwe Schindler (@uschindler) (migrated from JIRA)

Hi, we just have to ensure that all old keys keep alive in the general keys file, also OLD keys of people that changed their keys. So in general never ever delete a key from Apache's id.apache.org system, unless you want to re-sign all artifacts you have published. If we can ensure this, I am fine with deleting the per-version files.

Maybe we should do some crosscheck: Collect all key IDs from all files on dist.apache.org and calculate a union of it. After that check that all keys are also present in the global file. If not, ask the committers to re-upload the old/outdated public keys from the old keys file to their id.apache.org accounts.

Uwe

asfimport commented 7 years ago

Jan Høydahl (@janhoy) (migrated from JIRA)

Updated README patch to also discuss .sha512 hash.

asfimport commented 6 years ago

Jan Høydahl (@janhoy) (migrated from JIRA)

Updated LUCENE-5143.patch with these changes:

Added option --local-keys <path-to-KEYS-file> as alternative to pulling online version
Moved key check code to separate function

To sum up the full proposal:

We will STOP publishing version specific KEYS files, instead there will be one top level https://archive.apache.org/dist/lucene/KEYS for both lucene and solr, which will be kept up to date with all historic RM key IDs. The addition of the KEYS file check in this patch, the buildAndPushRelease script will abort if RM's gpg key is not present in that KEYS file. Steps to roll out:

Apply this patch to buildAndPushRelease.py
Release version 7.4 (this is the first release without the version-specific KEYS file)
Apply the READMEs patch against dist svn
Delete lucene/KEYS and solr/KEYS since they are not in use
Consider documenting usage of KEYS file on Lucene and Solr's respective Download web pages

PS: The pylucene project may continue using their local KEYS file.

asfimport commented 6 years ago

Jan Høydahl (@janhoy) (migrated from JIRA)

Uploaded separate patch for new master KEYS file. It is intended to be the new KEYS file and only be updated on demand, typically when a new RM prepares a release. Added some documentation to the file as well as the historic missing keys.

asfimport commented 6 years ago

Jan Høydahl (@janhoy) (migrated from JIRA)

@uschindler, @hossman Have you had a look at this? I'm ready to commit this if we agree it's a Good Thing™

asfimport commented 6 years ago

Chris M. Hostetter (@hossman) (migrated from JIRA)

Jan: I have not reviewed in depth, but in concept i have no objections to consolidating to a single KEYS file and having the release scripts abort if the RM's keys are not found there.

(Given the importances of KEYS files though, I would personally suggest that we shouldn't change this process/files w/o at least one other PMC member besides yourself giving it a detailed review.)

asfimport commented 6 years ago

Uwe Schindler (@uschindler) (migrated from JIRA)

Jan: +1 to your analysis of all the key files and merging them together, that's really needed. We should also document this on the release wiki. I did not go through all keys, I trust your script that you haven't forgotten one. Maybe we should do some test:

delete your locally cached GPG keys folder (maybe as a "pristine" test user).
download one (small) file from all prior releases (one file of each) and the asc file. Maybe Maven central is good for this, too.
import the new keys file into the "pristine" user's GPG.
check all file signatures

asfimport commented 6 years ago

Jan Høydahl (@janhoy) (migrated from JIRA)

I put together a script verify.sh verify.sh that :

Lists all files in lucene and solr folder in archive
Downloads all src artifacts (the smallest) with .asc files
Downloads KEYS file from this JIRA and imports it in a clean gpg (Docker)
Verify all sigs and produce a report

First run with my first KEYS file on this JIRA gave this result:

SUMMARY
=======
Number of artifacts to check:      158
Number of artifacts checked :      158
Number of artifacts SUCCESS :      150
Number of artifacts FAILED  :        8

The 8 failed files were

lucene-2.2.0-src.tar.gz
lucene-2.3.0-src.tar.gz
lucene-2.3.1-src.tar.gz
lucene-2.3.2-src.tar.gz
lucene-6.4.0-src.tgz
lucene-7.3.0-src.tgz
solr-6.4.0-src.tgz
solr-7.3.0-src.tgz

There were three missing keys in my compiled KEYS file:

gpg: key 1F2123C36BD872A0: public key "Michael Busch (Lucene Committer) <buschmi@apache.org>" imported
gpg: key 707B7D9F6FDB8105: public key "Jim Ferenczi (CODE SIGNING KEY) <jimczi@apache.org>" imported
gpg: key 069E9741F3D97FD6: public key "Alan Woodward <alan.woodward@romseysoftware.co.uk>" imported

After adding them to the end of the KEYS file, all artifacts verify ok

SUMMARY
=======
Number of artifacts to check:      158
Number of artifacts checked :      158
Number of artifacts SUCCESS :      158
Number of artifacts FAILED  :        0

See complete verify.log in attachment, and new KEYS file attached.

The important thing is that the KEYS file can verify all releases. It may not contain all keys for the last added committers, but then they will get notified once they become RM the first time that they have to update KEYS file with their own PGP public key.

verify.sh

```sh #!/bin/sh # echo "Fetching lucene file list (NB: May get rate blocked by apache)" # lftp -e "set net:limit-rate 1000:1000; du -a >_lucene.txt;quit" https://archive.apache.org/dist/lucene/java/ # cat _lucene.txt |cut -f 2 |grep src|grep -vE "md5|sha1|zip" |grep -v archive >_lucene_files.txt # sed 's|^\.|https://archive.apache.org/dist/lucene/java|g' <_lucene_files.txt >_lucene_src_urls.txt # echo "Fetching solr file list (NB: May get rate blocked by apache)" # lftp -e "set net:limit-rate 1000:1000; du -a >solr.txt;quit" https://archive.apache.org/dist/lucene/solr/ # cat _solr.txt |cut -f 2 |grep src|grep -vE "md5|sha1|zip" |grep -v archive >_solr_files.txt # sed 's|^\.|https://archive.apache.org/dist/lucene/solr|g' <_solr_files.txt >_solr_src_urls.txt # echo "Downloading src artifacts with detached sigs" # wget -i _lucene_src_urls.txt # wget -i _solr_src_urls.txt # echo Downloading KEYS file from JIRA # rm KEYS >/dev/null 2>&1 # wget https://issues.apache.org/jira/secure/attachment/12917687/KEYS echo "Import KEYS in clean gpg (./gnupg) using Docker" rm -rf ./gnupg >/dev/null 2>&1 mkdir gnupg docker run --rm -it -v $(pwd)/gnupg:/root/.gnupg -v $(pwd):/lucene vladgh/gpg --import /lucene/KEYS echo "Verifying all artifacts" rm _SIG_FAIL.txt _SIG_FAIL.log _SIG_SUCCESS.txt >/dev/null 2>&1 ; touch _SIG_FAIL.txt _SIG_FAIL.log _SIG_SUCCESS.txt for artifact in $(ls *.asc); do docker run --rm -it -v $(pwd)/gnupg:/root/.gnupg -v $(pwd):/lucene vladgh/gpg --verify /lucene/$artifact >tmp.txt 2>&1 if [ ! $? -eq 0 ]; then echo $artifact >>_SIG_FAIL.txt echo "====================================================================" >>_SIG_FAIL.log cat tmp.txt >>_SIG_FAIL.log ; rm tmp.txt echo "$artifact [FAIL]" else echo $artifact >>_SIG_SUCCESS.txt echo "$artifact [OK]" fi done echo echo "SUMMARY" echo "=======" echo "Number of artifacts to check: $(ls *.asc |wc -l)" echo "Number of artifacts checked : $(cat _SIG_SUCCESS.txt _SIG_FAIL.txt |wc -l)" echo "Number of artifacts SUCCESS : $(cat _SIG_SUCCESS.txt |wc -l)" echo "Number of artifacts FAILED : $(cat _SIG_FAIL.txt |wc -l)" echo echo "FAILED artifacts:" cat _SIG_FAIL.log echo DONE ```

asfimport commented 6 years ago

Jan Høydahl (@janhoy) (migrated from JIRA)

Uploaded an updated KEYS file with Cao's GPG key.

Do you guys feel confident that the changes are safe (i.e. the various verifications proved above, including the latest verify.sh), or do you need other tests?

I can commit this now before 7.4.0

asfimport commented 6 years ago

Jan Høydahl (@janhoy) (migrated from JIRA)

SUMMARY
=======
Number of artifacts to check:      162
Number of artifacts checked :      162
Number of artifacts SUCCESS :      162
Number of artifacts FAILED  :        0

asfimport commented 6 years ago

Jan Høydahl (@janhoy) (migrated from JIRA)

Anyone else interested in getting this out the door in time for 7.5 release?

asfimport commented 6 years ago

Jan Høydahl (@janhoy) (migrated from JIRA)

Ping

asfimport commented 6 years ago

Jan Høydahl (@janhoy) (migrated from JIRA)

Starting to commit this for 7.5.0 release. Will do the following steps

Update KEYS file in dist repo
Update links on Lucene&Solr download pages to link to this one KEYS file
Delete java/KEYS and solr/KEYS since they are not in use
Commit changes to buildAndPushRelease.py and update RM checklist
Update various documentation

asfimport commented 6 years ago

ASF subversion and git services (migrated from JIRA)

Commit 982ee3931bc0bf41088b154d6acc9bdf31ec70d4 in lucene-solr's branch refs/heads/master from @janhoy https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=982ee39

LUCENE-5143: rm or formalize dealing with "general" KEYS files in our dist dir

asfimport commented 6 years ago

ASF subversion and git services (migrated from JIRA)

Commit 4d588fcd8151be308c3ecec134ddc2a6afb0531b in lucene-solr's branch refs/heads/branch_7x from @janhoy https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=4d588fc

LUCENE-5143: rm or formalize dealing with "general" KEYS files in our dist dir

(cherry picked from commit 982ee39)

asfimport commented 6 years ago

Jan Høydahl (@janhoy) (migrated from JIRA)

This is now updated, ready for the 7.5.0 release. Thanks everyone for reviewing

asfimport commented 6 years ago

Jan Høydahl (@janhoy) (migrated from JIRA)

Reopening since our build.xml files still contain a -dist-keys target which copies KEYS from http://home.apache.org/keys/group/lucene.asc and places it in the dist (package/) folder. This is not necessary since it will never be used.

Also, smokeTestRelease.py should not expect to find KEYS file in solr/ or lucene/, but instead use the main dist/lucene/KEYS file

asfimport commented 6 years ago

Jan Høydahl (@janhoy) (migrated from JIRA)

Attached patch LUCENE-5143-reopen-smoke.patch with changes to smokeTestRelease.py:

Pulls KEYS from https://archive.apache.org/dist/lucene/KEYS instead of <staging-folder>/{lucene,solr}
Barfs if KEYS file is present in <staging-folder>/{lucene,solr}
Optionally takes --local-keys <file> argument to use locally cached KEYS file

Also some changes to build-files. I believe this is dead code related to staging RCs to home.apache.org which is no longer in use but superseded by buildAndPushRelease.py? Please correct me if I'm wrong:

Removes targets stage, copy-to-stage and -dist-keys from lucene/build.xml
Removes targets copy-to-stage and -dist-keys from solr/build.xml
Removes macro copy-to-stage-macro from lucene/common-build.xml

None of this is tested. @jimczi - I'll try to test the smnoketester before the first 7.5 RC.

LUCENE-5143-reopen-smoke.patch

```diff diff --git a/dev-tools/doap/solr.rdf b/dev-tools/doap/solr.rdf index 108eee3ea9..c739fb7d5f 100644 --- a/dev-tools/doap/solr.rdf +++ b/dev-tools/doap/solr.rdf @@ -71,7 +71,7 @@ solr-7.4.0 2018-06-27 - 7.4.1 + 7.4.0 diff --git a/dev-tools/scripts/buildAndPushRelease.py b/dev-tools/scripts/buildAndPushRelease.py index 3694f1a8b0..f9d615d562 100644 --- a/dev-tools/scripts/buildAndPushRelease.py +++ b/dev-tools/scripts/buildAndPushRelease.py @@ -78,9 +78,12 @@ def getGitRev(): raise RuntimeError('git clone is dirty:\n\n%s' % status) branch = os.popen('git rev-parse --abbrev-ref HEAD').read().strip() command = 'git log origin/%s..' % branch - unpushedCommits = os.popen(command).read().strip() - if len(unpushedCommits) > 0: - raise RuntimeError('There are unpushed commits - "%s" output is:\n\n%s' % (command, unpushedCommits)) + p = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) + stdout, stderr = p.communicate() + if len(stdout.strip()) > 0: + raise RuntimeError('There are unpushed commits - "%s" output is:\n\n%s' % (command, stdout.decode('utf-8'))) + if len(stderr.strip()) > 0: + raise RuntimeError('Command "%s" failed:\n\n%s' % (command, stderr.decode('utf-8'))) print(' git clone is clean') return os.popen('git rev-parse HEAD').read().strip() @@ -271,14 +274,6 @@ def parse_config(): config.version = read_version(config.root) print('Building version: %s' % config.version) - if config.sign: - sys.stdout.flush() - import getpass - config.key_id = config.sign - config.key_password = getpass.getpass('Enter GPG keystore password: ') - else: - config.gpg_password = None - return config def check_cmdline_tools(): # Fail fast if there are cmdline tool problems @@ -313,15 +308,15 @@ def check_key_in_keys(gpgKeyID, local_keys): if len(gpgKeyID) > 40: gpgKeyID = gpgKeyID.replace(" ", "") if len(gpgKeyID) == 8: - re_to_match = r"^pub\s+\d+[DR]/%s " % gpgKeyID + gpgKeyID8Char = "%s %s" % (gpgKeyID[0:4], gpgKeyID[4:8]) + re_to_match = r"^pub .*\n\s+\w{4} \w{4} \w{4} \w{4} \w{4} \w{4} \w{4} \w{4} %s" % gpgKeyID8Char elif len(gpgKeyID) == 40: gpgKeyID40Char = "%s %s %s %s %s %s %s %s %s %s" % \ (gpgKeyID[0:4], gpgKeyID[4:8], gpgKeyID[8:12], gpgKeyID[12:16], gpgKeyID[16:20], gpgKeyID[20:24], gpgKeyID[24:28], gpgKeyID[28:32], gpgKeyID[32:36], gpgKeyID[36:]) - print("Generated id string %s" % gpgKeyID40Char) - re_to_match = r"^\s+Key fingerprint = %s$" % gpgKeyID40Char + re_to_match = r"^pub .*\n\s+%s" % gpgKeyID40Char else: - print('Invalid gpg key id format. Must be 8 byte short ID or 40 byte fingerprint, with or without 0x prefix.') + print('Invalid gpg key id format. Must be 8 byte short ID or 40 byte fingerprint, with or without 0x prefix, no spaces.') exit(2) if re.search(re_to_match, keysFileText, re.MULTILINE): print(' Found key %s in KEYS file at %s' % (gpgKeyID, keysFileLocation)) @@ -337,7 +332,14 @@ def main(): c = parse_config() - check_key_in_keys(c.key_id, c.local_keys) + if c.sign: + sys.stdout.flush() + c.key_id = c.sign + check_key_in_keys(c.key_id, c.local_keys) + import getpass + c.key_password = getpass.getpass('Enter GPG keystore password: ') + else: + c.gpg_password = None if c.prepare: rev = prepare(c.root, c.version, c.key_id, c.key_password) diff --git a/dev-tools/scripts/smokeTestRelease.py b/dev-tools/scripts/smokeTestRelease.py index f68a9b1f9f..af26ae195b 100644 --- a/dev-tools/scripts/smokeTestRelease.py +++ b/dev-tools/scripts/smokeTestRelease.py @@ -285,23 +285,22 @@ def checkAllJARs(topDir, project, gitRevision, version, tmpDir, baseURL): % (fullPath, luceneDistFilenames[jarFilename])) -def checkSigs(project, urlString, version, tmpDir, isSigned): +def checkSigs(project, urlString, version, tmpDir, isSigned, local_keys): print(' test basics...') ents = getDirEntries(urlString) artifact = None - keysURL = None changesURL = None mavenURL = None expectedSigs = [] if isSigned: expectedSigs.append('asc') expectedSigs.extend(['sha1', 'sha512']) - + artifacts = [] for text, subURL in ents: if text == 'KEYS': - keysURL = subURL + raise RuntimeError('%s: release dir should not contain a KEYS file - only toplevel /dist/lucene/KEYS is used' % project) elif text == 'maven/': mavenURL = subURL elif text.startswith('changes'): @@ -346,14 +345,16 @@ def checkSigs(project, urlString, version, tmpDir, isSigned): if expected != actual: raise RuntimeError('%s: wrong artifacts: expected %s but got %s' % (project, expected, actual)) - if keysURL is None: - raise RuntimeError('%s is missing KEYS' % project) - print(' get KEYS') - download('%s.KEYS' % project, keysURL, tmpDir) - - keysFile = '%s/%s.KEYS' % (tmpDir, project) - + if local_keys is not None: + print(" Using local KEYS file %s" % local_keys) + keysFile = local_keys + else: + keysFileURL = "https://archive.apache.org/dist/lucene/KEYS" + print(" Downloading online KEYS file %s" % keysFileURL) + download('KEYS', keysFileURL, tmpDir) + keysFile = '%s/KEYS' % (tmpDir) + # Set up clean gpg world; import keys file: gpgHomeDir = '%s/%s.gpg' % (tmpDir, project) if os.path.exists(gpgHomeDir): @@ -1291,6 +1292,8 @@ def parse_config(): help='Temporary directory to test inside, defaults to /tmp/smoke_lucene_$version_$revision') parser.add_argument('--not-signed', dest='is_signed', action='store_false', default=True, help='Indicates the release is not signed') + parser.add_argument('--local-keys', metavar='PATH', + help='Uses local KEYS file instead of fetching from https://archive.apache.org/dist/lucene/KEYS') parser.add_argument('--revision', help='GIT revision number that release was built with, defaults to that in URL') parser.add_argument('--version', metavar='X.Y.Z(-ALPHA|-BETA)?', @@ -1318,6 +1321,9 @@ def parse_config(): c.revision = revision_match.group(1) print('Revision: %s' % c.revision) + if c.local_keys is not None and not os.path.exists(c.local_keys): + parser.error('Local KEYS file "%s" not found' % c.local_keys) + c.java = make_java_config(parser, c.test_java9) if c.tmp_dir: @@ -1462,9 +1468,9 @@ def main(): raise RuntimeError('smokeTestRelease.py for %s.X is incompatible with a %s release.' % (scriptVersion, c.version)) print('NOTE: output encoding is %s' % sys.stdout.encoding) - smokeTest(c.java, c.url, c.revision, c.version, c.tmp_dir, c.is_signed, ' '.join(c.test_args)) + smokeTest(c.java, c.url, c.revision, c.version, c.tmp_dir, c.is_signed, c.local_keys, ' '.join(c.test_args)) -def smokeTest(java, baseURL, gitRevision, version, tmpDir, isSigned, testArgs): +def smokeTest(java, baseURL, gitRevision, version, tmpDir, isSigned, local_keys, testArgs): startTime = datetime.datetime.now() @@ -1500,14 +1506,14 @@ def smokeTest(java, baseURL, gitRevision, version, tmpDir, isSigned, testArgs): print() print('Test Lucene...') - checkSigs('lucene', lucenePath, version, tmpDir, isSigned) + checkSigs('lucene', lucenePath, version, tmpDir, isSigned, local_keys) for artifact in ('lucene-%s.tgz' % version, 'lucene-%s.zip' % version): unpackAndVerify(java, 'lucene', tmpDir, artifact, gitRevision, version, testArgs, baseURL) unpackAndVerify(java, 'lucene', tmpDir, 'lucene-%s-src.tgz' % version, gitRevision, version, testArgs, baseURL) print() print('Test Solr...') - checkSigs('solr', solrPath, version, tmpDir, isSigned) + checkSigs('solr', solrPath, version, tmpDir, isSigned, local_keys) for artifact in ('solr-%s.tgz' % version, 'solr-%s.zip' % version): unpackAndVerify(java, 'solr', tmpDir, artifact, gitRevision, version, testArgs, baseURL) solrSrcUnpackPath = unpackAndVerify(java, 'solr', tmpDir, 'solr-%s-src.tgz' % version, diff --git a/lucene/build.xml b/lucene/build.xml index b6131af65a..3c1439c7e2 100644 --- a/lucene/build.xml +++ b/lucene/build.xml @@ -387,7 +387,7 @@ - + @@ -396,21 +396,8 @@ - - - - - - - - - - - - diff --git a/lucene/common-build.xml b/lucene/common-build.xml index 68e17da2ea..9fbddc3d25 100644 --- a/lucene/common-build.xml +++ b/lucene/common-build.xml @@ -2380,32 +2380,6 @@ ${ant.project.name}.test.dependencies=${test.classpath.list} - - - - - - - - - - - Uploading artifacts to ${scp.user}@home.apache.org:${remote.staging.dir} - - - - - - - - - - - - - @@ -467,7 +463,7 @@ - + @@ -476,12 +472,6 @@ - - - - -

asfimport commented 6 years ago

Steven Rowe (@sarowe) (migrated from JIRA)

Also some changes to build-files. I believe this is dead code related to staging RCs to home.apache.org which is no longer in use but superseded by buildAndPushRelease.py? Please correct me if I'm wrong:

Those targets are described as part of the "manual" Building the Release Artifacts workflow (Option #2) on Lucene's ReleaseToDo wiki page: https://wiki.apache.org/lucene-java/ReleaseTodo#Building_the_Release_Artifacts . I don't know when the last time this was used - probably the last time @rmuir did a release, which AFAICT was 4.8.1 ?

asfimport commented 6 years ago

Jan Høydahl (@janhoy) (migrated from JIRA)

Is there a benefit of maintaining two different release procedures? They have already diverged and the ant version either needs to be kept up to date or to be removed?

asfimport commented 6 years ago

Jan Høydahl (@janhoy) (migrated from JIRA)

New version of LUCENE-5143-reopen-smoke.patch

Fixes in buildAndPushRelease.py:
- Fixed bug in GPG key check against KEYS, new regex pattern
- Check that GPG key is in KEYS file before asking for GPG pasphrase
- Abort script if 'git log origin/<branch>' command fails
Tested smokeTestRelease.py including support for --local-keys
Fixed typo in dev-tools/doap/solr.rdf - release 7.4.1 -> 7.4.0

I'll commit this tomorrow. I'm removing the copy-to-stage target and friends from ant build scripts since they are old and out of date - unless someone really want them there for some specific reason (and want to maintain the code).

LUCENE-5143-reopen-smoke.patch

```diff diff --git a/dev-tools/doap/solr.rdf b/dev-tools/doap/solr.rdf index 108eee3ea9..c739fb7d5f 100644 --- a/dev-tools/doap/solr.rdf +++ b/dev-tools/doap/solr.rdf @@ -71,7 +71,7 @@ solr-7.4.0 2018-06-27 - 7.4.1 + 7.4.0 diff --git a/dev-tools/scripts/buildAndPushRelease.py b/dev-tools/scripts/buildAndPushRelease.py index 3694f1a8b0..f9d615d562 100644 --- a/dev-tools/scripts/buildAndPushRelease.py +++ b/dev-tools/scripts/buildAndPushRelease.py @@ -78,9 +78,12 @@ def getGitRev(): raise RuntimeError('git clone is dirty:\n\n%s' % status) branch = os.popen('git rev-parse --abbrev-ref HEAD').read().strip() command = 'git log origin/%s..' % branch - unpushedCommits = os.popen(command).read().strip() - if len(unpushedCommits) > 0: - raise RuntimeError('There are unpushed commits - "%s" output is:\n\n%s' % (command, unpushedCommits)) + p = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) + stdout, stderr = p.communicate() + if len(stdout.strip()) > 0: + raise RuntimeError('There are unpushed commits - "%s" output is:\n\n%s' % (command, stdout.decode('utf-8'))) + if len(stderr.strip()) > 0: + raise RuntimeError('Command "%s" failed:\n\n%s' % (command, stderr.decode('utf-8'))) print(' git clone is clean') return os.popen('git rev-parse HEAD').read().strip() @@ -271,14 +274,6 @@ def parse_config(): config.version = read_version(config.root) print('Building version: %s' % config.version) - if config.sign: - sys.stdout.flush() - import getpass - config.key_id = config.sign - config.key_password = getpass.getpass('Enter GPG keystore password: ') - else: - config.gpg_password = None - return config def check_cmdline_tools(): # Fail fast if there are cmdline tool problems @@ -313,15 +308,15 @@ def check_key_in_keys(gpgKeyID, local_keys): if len(gpgKeyID) > 40: gpgKeyID = gpgKeyID.replace(" ", "") if len(gpgKeyID) == 8: - re_to_match = r"^pub\s+\d+[DR]/%s " % gpgKeyID + gpgKeyID8Char = "%s %s" % (gpgKeyID[0:4], gpgKeyID[4:8]) + re_to_match = r"^pub .*\n\s+\w{4} \w{4} \w{4} \w{4} \w{4} \w{4} \w{4} \w{4} %s" % gpgKeyID8Char elif len(gpgKeyID) == 40: gpgKeyID40Char = "%s %s %s %s %s %s %s %s %s %s" % \ (gpgKeyID[0:4], gpgKeyID[4:8], gpgKeyID[8:12], gpgKeyID[12:16], gpgKeyID[16:20], gpgKeyID[20:24], gpgKeyID[24:28], gpgKeyID[28:32], gpgKeyID[32:36], gpgKeyID[36:]) - print("Generated id string %s" % gpgKeyID40Char) - re_to_match = r"^\s+Key fingerprint = %s$" % gpgKeyID40Char + re_to_match = r"^pub .*\n\s+%s" % gpgKeyID40Char else: - print('Invalid gpg key id format. Must be 8 byte short ID or 40 byte fingerprint, with or without 0x prefix.') + print('Invalid gpg key id format. Must be 8 byte short ID or 40 byte fingerprint, with or without 0x prefix, no spaces.') exit(2) if re.search(re_to_match, keysFileText, re.MULTILINE): print(' Found key %s in KEYS file at %s' % (gpgKeyID, keysFileLocation)) @@ -337,7 +332,14 @@ def main(): c = parse_config() - check_key_in_keys(c.key_id, c.local_keys) + if c.sign: + sys.stdout.flush() + c.key_id = c.sign + check_key_in_keys(c.key_id, c.local_keys) + import getpass + c.key_password = getpass.getpass('Enter GPG keystore password: ') + else: + c.gpg_password = None if c.prepare: rev = prepare(c.root, c.version, c.key_id, c.key_password) diff --git a/dev-tools/scripts/smokeTestRelease.py b/dev-tools/scripts/smokeTestRelease.py index f68a9b1f9f..af26ae195b 100644 --- a/dev-tools/scripts/smokeTestRelease.py +++ b/dev-tools/scripts/smokeTestRelease.py @@ -285,23 +285,22 @@ def checkAllJARs(topDir, project, gitRevision, version, tmpDir, baseURL): % (fullPath, luceneDistFilenames[jarFilename])) -def checkSigs(project, urlString, version, tmpDir, isSigned): +def checkSigs(project, urlString, version, tmpDir, isSigned, local_keys): print(' test basics...') ents = getDirEntries(urlString) artifact = None - keysURL = None changesURL = None mavenURL = None expectedSigs = [] if isSigned: expectedSigs.append('asc') expectedSigs.extend(['sha1', 'sha512']) - + artifacts = [] for text, subURL in ents: if text == 'KEYS': - keysURL = subURL + raise RuntimeError('%s: release dir should not contain a KEYS file - only toplevel /dist/lucene/KEYS is used' % project) elif text == 'maven/': mavenURL = subURL elif text.startswith('changes'): @@ -346,14 +345,16 @@ def checkSigs(project, urlString, version, tmpDir, isSigned): if expected != actual: raise RuntimeError('%s: wrong artifacts: expected %s but got %s' % (project, expected, actual)) - if keysURL is None: - raise RuntimeError('%s is missing KEYS' % project) - print(' get KEYS') - download('%s.KEYS' % project, keysURL, tmpDir) - - keysFile = '%s/%s.KEYS' % (tmpDir, project) - + if local_keys is not None: + print(" Using local KEYS file %s" % local_keys) + keysFile = local_keys + else: + keysFileURL = "https://archive.apache.org/dist/lucene/KEYS" + print(" Downloading online KEYS file %s" % keysFileURL) + download('KEYS', keysFileURL, tmpDir) + keysFile = '%s/KEYS' % (tmpDir) + # Set up clean gpg world; import keys file: gpgHomeDir = '%s/%s.gpg' % (tmpDir, project) if os.path.exists(gpgHomeDir): @@ -1291,6 +1292,8 @@ def parse_config(): help='Temporary directory to test inside, defaults to /tmp/smoke_lucene_$version_$revision') parser.add_argument('--not-signed', dest='is_signed', action='store_false', default=True, help='Indicates the release is not signed') + parser.add_argument('--local-keys', metavar='PATH', + help='Uses local KEYS file instead of fetching from https://archive.apache.org/dist/lucene/KEYS') parser.add_argument('--revision', help='GIT revision number that release was built with, defaults to that in URL') parser.add_argument('--version', metavar='X.Y.Z(-ALPHA|-BETA)?', @@ -1318,6 +1321,9 @@ def parse_config(): c.revision = revision_match.group(1) print('Revision: %s' % c.revision) + if c.local_keys is not None and not os.path.exists(c.local_keys): + parser.error('Local KEYS file "%s" not found' % c.local_keys) + c.java = make_java_config(parser, c.test_java9) if c.tmp_dir: @@ -1462,9 +1468,9 @@ def main(): raise RuntimeError('smokeTestRelease.py for %s.X is incompatible with a %s release.' % (scriptVersion, c.version)) print('NOTE: output encoding is %s' % sys.stdout.encoding) - smokeTest(c.java, c.url, c.revision, c.version, c.tmp_dir, c.is_signed, ' '.join(c.test_args)) + smokeTest(c.java, c.url, c.revision, c.version, c.tmp_dir, c.is_signed, c.local_keys, ' '.join(c.test_args)) -def smokeTest(java, baseURL, gitRevision, version, tmpDir, isSigned, testArgs): +def smokeTest(java, baseURL, gitRevision, version, tmpDir, isSigned, local_keys, testArgs): startTime = datetime.datetime.now() @@ -1500,14 +1506,14 @@ def smokeTest(java, baseURL, gitRevision, version, tmpDir, isSigned, testArgs): print() print('Test Lucene...') - checkSigs('lucene', lucenePath, version, tmpDir, isSigned) + checkSigs('lucene', lucenePath, version, tmpDir, isSigned, local_keys) for artifact in ('lucene-%s.tgz' % version, 'lucene-%s.zip' % version): unpackAndVerify(java, 'lucene', tmpDir, artifact, gitRevision, version, testArgs, baseURL) unpackAndVerify(java, 'lucene', tmpDir, 'lucene-%s-src.tgz' % version, gitRevision, version, testArgs, baseURL) print() print('Test Solr...') - checkSigs('solr', solrPath, version, tmpDir, isSigned) + checkSigs('solr', solrPath, version, tmpDir, isSigned, local_keys) for artifact in ('solr-%s.tgz' % version, 'solr-%s.zip' % version): unpackAndVerify(java, 'solr', tmpDir, artifact, gitRevision, version, testArgs, baseURL) solrSrcUnpackPath = unpackAndVerify(java, 'solr', tmpDir, 'solr-%s-src.tgz' % version, diff --git a/lucene/build.xml b/lucene/build.xml index b6131af65a..3c1439c7e2 100644 --- a/lucene/build.xml +++ b/lucene/build.xml @@ -387,7 +387,7 @@ - + @@ -396,21 +396,8 @@ - - - - - - - - - - - - diff --git a/lucene/common-build.xml b/lucene/common-build.xml index 68e17da2ea..9fbddc3d25 100644 --- a/lucene/common-build.xml +++ b/lucene/common-build.xml @@ -2380,32 +2380,6 @@ ${ant.project.name}.test.dependencies=${test.classpath.list} - - - - - - - - - - - Uploading artifacts to ${scp.user}@home.apache.org:${remote.staging.dir} - - - - - - - - - - - - - @@ -467,7 +463,7 @@ - + @@ -476,12 +472,6 @@ - - - - -

asfimport commented 6 years ago

ASF subversion and git services (migrated from JIRA)

Commit 5b96f89d2b038bff2ed3351887a87108f7cc6ea3 in lucene-solr's branch refs/heads/master from @janhoy https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=5b96f89

LUCENE-5143: Fix smoketester, fix RM PGP key check, fix solr DOAP file, add CHANGES entry Remove unused/stale 'copy-to-stage' and '-dist-keys' targets from ant build

asfimport commented 6 years ago

ASF subversion and git services (migrated from JIRA)

Commit 63a95ea9e95de5584a3debc7d860a33495380f42 in lucene-solr's branch refs/heads/branch_7x from @janhoy https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=63a95ea

LUCENE-5143: Fix smoketester, fix RM PGP key check, fix solr DOAP file, add CHANGES entry Remove unused/stale 'copy-to-stage' and '-dist-keys' targets from ant build

(cherry picked from commit 5b96f89)

asfimport commented 6 years ago

ASF subversion and git services (migrated from JIRA)

Commit 308504f828a770e0508e5b2eb8027f347f995b27 in lucene-solr's branch refs/heads/branch_7_5 from @janhoy https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=308504f

LUCENE-5143: Fix smoketester, fix RM PGP key check, fix solr DOAP file, add CHANGES entry Remove unused/stale 'copy-to-stage' and '-dist-keys' targets from ant build

(cherry picked from commit 5b96f89)

asfimport commented 6 years ago

Jan Høydahl (@janhoy) (migrated from JIRA)

Resolving this issue again after committing yesterday's proposed patch

I chose to remove the unused build targets both due to the fact that they have not been used for years, are stale, and do not comply with release policy of staging RCs to official dist area (see http://www.apache.org/legal/release-policy.html#host-rc)

I have successfully tested a full cycle of buildAndPushRelease.py and smokeTestRelease.py. @jimczi this should hopefully be good to go for friday's RC.

asfimport commented 6 years ago

ASF subversion and git services (migrated from JIRA)

Commit 5b96f89d2b038bff2ed3351887a87108f7cc6ea3 in lucene-solr's branch refs/heads/jira/http2 from @janhoy https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=5b96f89

LUCENE-5143: Fix smoketester, fix RM PGP key check, fix solr DOAP file, add CHANGES entry Remove unused/stale 'copy-to-stage' and '-dist-keys' targets from ant build

apache / lucene

rm or formalize dealing with "general" KEYS files in our dist dir [LUCENE-5143] #6207