Closed knz closed 3 years ago
Thanks for doing this @knz!
optimizer
setting is now deprecated and the only valid setting is "on".enable_zigzag_join
session setting. Note: more recently, we also added a corresponding sql.defaults.zigzag_join.enabled
cluster setting (in #42182).EXPLAIN (DISTSQL)
and EXPLAIN (ANALYZE)
now shows the SQL query in the diagram page.I have a slight dislike the framing of this and the implication that use of ‘none’ can just be assumed to be mistaken.
There seems to be an assumption here that one commit = one user-facing-change, but one big monster commit is not how we usually want to go about building big new features. Many cases above include many commits working towards implementing one new user-visible feature — there is no reason for those to each have individual release notes as they are not changing an existing user-visible feature. In other cases, they are fixing bugs, but in similarly unreleased functionality and thus again, are not user-visible changes.
The release notes in the PRs are meant to be accurate between alpha releases, so it is reasonable to document small changes that are part of a bigger new feature (though I agree it's not terribly useful to provide a lot of detail). How these get massaged into the release notes for a major release is not trivial - the improvements related to a bigger change do need to be coalesced into one item.
There seems to be an assumption here that one commit = one user-facing-change, but one big monster commit is not how we usually want to go about building big new features. Many cases above include many commits working towards implementing one new user-visible feature — there is no reason for those to each have individual release notes as they are not changing an existing user-visible feature.
David, I stand behind your view here. I have been careful to de-duplicate entries and collapse multiple PRs that are part of a larger whole under a single entry (for example, I removed all the "opt: implement X or Y" entries because they all fall under "remove the heuristic planner", which was documented eventually). I did the same with various IMPORT FROM PRs.
Unfortunately it is hard for me to be 100% accurate so I apologize if I forgot to collapse some groups.
Thank you for driving this, @knz. Since our Eng team has grown so much, I had a nagging worry that many user-facing changes weren't getting flagged for announcement and documentation. At the end of this exercise, we'll have a clear sense of that.
(bug fix) #38345 A bug where recalling past history entries spanning multiple lines would cause a corrupted display in cockroach demo
and cockroach sql
has been corrected.
(cli change) #39685 The command workload run tpcc
now supports --wait
with a fractional value to configure how much to wait between queries relative to the delays mandated by the TPC-C specification.
(bug fix) #37800 Fix a bug where a shema change would fail, and not be retried, if a node was disconnected while the schema change is running. This bug had been introduced in a 19.2 alpha release.
(sql change) #38762 CockroachDB will now produce more details when printing out some internal error in the log files, to ease troubleshooting.
(cli change) #37805 cockroach start
will now produce more details if a node fails to start, to ease troubleshooting.
(cli change) #39492 cockroach start-single-node
and cockroach demo
with a single configured node will now use zone configs to disable replication.
(bug fix) #39378 cockroach zip
will now use properly the application name "cockroach zip" when establishing SQL connections instead of "cockroach sql", to clarify recognition of its activities in statement statistics.
(build changes) #39315 It is now possible to build CockroachDB with Node 12.
(bug fix) #37705 Fixed a bug where CockroachDB was unable to verify the maximum number of open file descriptors on macOS.
* #40880 (telemetry change) Telemetry is collected when ALTER ALL PARTITIONS
is used.
SHOW CREATE TABLE
would omit zone configuration information if the zone had an empty constraints list.crdb_internal.zones
table.cockroach demo
is used with the flags "--nodes", "--demo-locality", "--with-load" and "--geo-partitioned-replicas".SHOW RANGES
.SHOW PARTITIONS
, SHOW RANGES
, SHOW LOCALITY
and SHOW CREATE TABLE
.movr
workload.Done, but I think we should have a conversation about what changes are worthy to be included in release notes. For one, I enjoy reading release notes for software, but not if they're endless. I'm afraid that activism like this, coupled with more and more monkeys banging on this, will lead to a million bullets of very varying degrees of importance.
Thanks for picking this off! Most of these are of very marginal use to the user, but the changefeed one is a great find. The rest of mine were correctly omitted.
workload init
is now 15% faster when used with --data-loader=import
. This also lays a foundation for future performance work.init
s faster when used with --data-loader=import
.workload init
when used with --data-loader=import
.fall back to using 19.1 vectorize setting on syntax error. Release note (bug fix): The querybench workload has been modified to use the correct syntax for vectorized execution in 19.1, attempting to run this workload with vectorized execution would previously return an error.
release-19.2: colrpc: downgrade log messages from Error to Warning. Release note (bug fix): Some log messages reported as Errors have been downgraded to Warnings.
wrap hashRouter work with CatchVectorizedRuntimeError. Release note (bug fix): An expected panic in the vectorized engine is now reported as an error without crashing the server.
wait for internal goroutine completion in UnorderedSynchronizer. Release note (bug fix): A race condition in the vectorized engine has been fixed.
set output columnTypes when creating noop operator. Release note (bug fix): A bug planning noop operators in the vectorzied execution engine has been fixed.
reset internal unsafeBatch in orderedAggregator. Release note (bug fix): A bug with vectorized aggregations due to incorrect state transitions has been fixed.
protect against unset syncFlowConsumer. Release note (bug fix): A vectorized engine plan-time state assertion was added to avoid a panic.
fix metadata drain cycles. Release note (bug fix): A bug that led to queries hanging in the vectorized engine due to incorrect plans has been fixed.
check for unset output columnTypes. Release note (bug fix): A vectorized engine plan-time state assertion was added.
include StartScan stall time in EXPLAIN ANALYZE. Release note (sql change): EXPLAIN ANALYZE now reports the time taken to look up rows in a lookup join.
bump index joiner batch size from 100 to 10k. Release note (performance improvement): Index join performance has been improved.
col{data,serde}: various fixes related to TestArrowBatchConverterRandom (2 commits). Release note (bug fix): A bug in data serialization in the vectorized engine caused by incorrectly determining the presence of nulls in rows has been fixed.
On Mon, Nov 11, 2019 at 10:48 AM irfan sharif notifications@github.com wrote:
41764: Release note (bug fix): Previously a disk stall could allow
a node to continue heartbeating its liveness record and prevent other nodes from taking over its leases, despite being completely unresponsive.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/cockroachdb/docs/issues/5819?email_source=notifications&email_token=ACQSGZ7AXGRGIKBUZ2XGSMTQTF5FTA5CNFSM4JLMNMWKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDXHLPY#issuecomment-552498623, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACQSGZ4EXRRJXSGT26KTAWLQTF5FTANCNFSM4JLMNMWA .
-- Alfonso
Spas Bojanov:
make new index backfill status tracking bwd compatible. Release note (bug fix): this was introduced by #39773 which will cause backfill to crash when running on cluster with upgraded nodes.
improve the status reporting of index backfill jobs. Release note (bug fix): fixes #36601.
clean running jobs with no mutations left. Release note (bug fix): fixes
fix mysqqoutfile import bug. Release note (bug fix): When importing from csv if the number of fields is less than the columns in the schema the server will panic.
On Tue, Nov 12, 2019 at 12:22 AM Alfonso Subiotto Marqués < notifications@github.com> wrote:
39662 https://github.com/cockroachdb/cockroach/pull/39662 workload:
fall back to using 19.1 vectorize setting on syntax error. Release note (bug fix): The querybench workload has been modified to use the correct syntax for vectorized execution in 19.1, attempting to run this workload with vectorized execution would previously return an error.
41467 https://github.com/cockroachdb/cockroach/pull/41467
release-19.2: colrpc: downgrade log messages from Error to Warning. Release note (bug fix): Some log messages reported as Errors have been downgraded to Warnings.
37088 https://github.com/cockroachdb/cockroach/pull/37088 exec:
wrap hashRouter work with CatchVectorizedRuntimeError. Release note (bug fix): An expected panic in the vectorized engine is now reported as an error without crashing the server.
39093 https://github.com/cockroachdb/cockroach/pull/39093 exec:
wait for internal goroutine completion in UnorderedSynchronizer. Release note (bug fix): A race condition in the vectorized engine has been fixed.
38006 https://github.com/cockroachdb/cockroach/pull/38006 exec:
set output columnTypes when creating noop operator. Release note (bug fix): A bug planning noop operators in the vectorzied execution engine has been fixed.
40668 https://github.com/cockroachdb/cockroach/pull/40668 exec:
reset internal unsafeBatch in orderedAggregator. Release note (bug fix): A bug with vectorized aggregations due to incorrect state transitions has been fixed.
38557 https://github.com/cockroachdb/cockroach/pull/38557 exec:
protect against unset syncFlowConsumer. Release note (bug fix): A vectorized engine plan-time state assertion was added to avoid a panic.
39248 https://github.com/cockroachdb/cockroach/pull/39248 exec:
fix metadata drain cycles. Release note (bug fix): A bug that led to queries hanging in the vectorized engine due to incorrect plans has been fixed.
37571 https://github.com/cockroachdb/cockroach/pull/37571 exec:
check for unset output columnTypes. Release note (bug fix): A vectorized engine plan-time state assertion was added.
38329 https://github.com/cockroachdb/cockroach/pull/38329 distsqlrun:
include StartScan stall time in EXPLAIN ANALYZE. Release note (sql change): EXPLAIN ANALYZE now reports the time taken to look up rows in a lookup join.
38622 https://github.com/cockroachdb/cockroach/pull/38622 distsqlrun:
bump index joiner batch size from 100 to 10k. Release note (performance improvement): Index join performance has been improved.
37745 https://github.com/cockroachdb/cockroach/pull/37745
col{data,serde}: various fixes related to TestArrowBatchConverterRandom (2 commits). Release note (bug fix): A bug in data serialization in the vectorized engine caused by incorrectly determining the presence of nulls in rows has been fixed.
On Mon, Nov 11, 2019 at 10:48 AM irfan sharif notifications@github.com wrote:
41764: Release note (bug fix): Previously a disk stall could allow
a node to continue heartbeating its liveness record and prevent other nodes from taking over its leases, despite being completely unresponsive.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub < https://github.com/cockroachdb/docs/issues/5819?email_source=notifications&email_token=ACQSGZ7AXGRGIKBUZ2XGSMTQTF5FTA5CNFSM4JLMNMWKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDXHLPY#issuecomment-552498623 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/ACQSGZ4EXRRJXSGT26KTAWLQTF5FTANCNFSM4JLMNMWA
.
-- Alfonso
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/cockroachdb/docs/issues/5819?email_source=notifications&email_token=AANL635GCJRAF2GECKWGYBDQTGBF7A5CNFSM4JLMNMWKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDXKWQQ#issuecomment-552512322, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANL637KSPBTOLESRUX35EDQTGBF7ANCNFSM4JLMNMWA .
Done, but I think we should have a conversation about what changes are worthy to be included in release notes. For one, I enjoy reading release notes for software, but not if they're endless. I'm afraid that activism like this, coupled with more and more monkeys banging on this, will lead to a million bullets of very varying degrees of importance.
I generally think release notes should be as comprehensive as possible. They're not necessarily read, but it's important that they be searchable. If the notes are too large (as they are for a full release), this should be addressed with a separate summary document instead of dropping details. (I like python's what's new/changelog split)
cockroach debug unsafe-remove-dead-replicas
can nowrecover ranges with more than one remaining live replica
Windows version
status is not written before the node starts
file not found
errormessage is interpreted incorrectly under Windows
even number of replicas
On Mon, Nov 11, 2019 at 9:36 AM Ben Darnell notifications@github.com wrote:
Done, but I think we should have a conversation about what changes are worthy to be included in release notes. For one, I enjoy reading release notes for software, but not if they're endless. I'm afraid that activism like this, coupled with more and more monkeys banging on this, will lead to a million bullets of very varying degrees of importance.
I generally think release notes should be as comprehensive as possible. They're not necessarily read, but it's important that they be searchable. If the notes are too large (as they are for a full release), this should be addressed with a separate summary document instead of dropping details. (I like python's what's new https://docs.python.org/3/whatsnew/3.8.html/changelog https://docs.python.org/3/whatsnew/changelog.html#changelog split)
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/cockroachdb/docs/issues/5819?email_source=notifications&email_token=ABVZ3FKE7L2UF2YSKRPO3B3QTGJYLA5CNFSM4JLMNMWKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDXRPYA#issuecomment-552540128, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABVZ3FNXQSZKAJOFI554DHTQTGJYLANCNFSM4JLMNMWA .
ADD CONSTRAINT
for foreign keys to be backward-compatible with 19.1.N/A: #39383 (no user-facing changes)
FWIW, with the exception of #36032 and #35742, all of these were bug fixes for bugs introduced in earlier alpha/beta releases, and therefore don't represent differences from 19.1. It's not clear to me what the plan is for incorporating these release notes into the 19.2.0 release notes, but I worry that adding release notes that were bug fixes from earlier alphas/betas indiscriminately to all the other release notes will be misleading.
I omitted PRs which didn't represent a user-visible change from 19.1 or weren't really of interest to users, like adding a HINT to an error message.
Not adding notes for #40757, since that was a fix for a PR/feature that was added in this release, and that feature already has release notes.
Also, I noticed that the telemetry change
category is being used a lot here, but that category is not documented in Confluence nor in the commit message template. Should we add it?
workload fixtures make
uses IMPORT to load data before backing it up, replacing complex and brittle manual construction of a raw BACKUP.rocksdb.ingest_backpressure.delay_l0_file
setting that was added but never used.querybench
workload now has an argument that specifies whether to use the vectorized engine.tpcds
workload is added.bytes_read
and rows_read
fields have been added to node_statement_statistics
virtual table.EXPLAIN
ing a query (with DISTSQL
or VEC
option) that contains a subquery the result of which is needed to create the physical plan returns a more useful error instead of an internal error as it did previously.vectorize
cluster setting no longer has experimental_always
option.GROUPS BETWEEN offset FOLLOWING AND UNBOUNDED FOLLOWING
are now handled correctly.GROUPS
mode of window framing now requires an ORDER BY
clause and will error out if the latter is not provided.has_column_privilege
builtin now works correctly in cases when columns are dropped.PARTITION BY
and ORDER BY
clauses of another window function, CRDB would crash while trying to execute it. Now such queries will be error out.array_agg
and concat_agg
being the most prone to it functions).AND
and OR
operators are now handled correctly by the vectorized execution engine when operating on NULLs.count
aggregate function as an operand.OID
type.SUM
aggregate is executed via the vectorized execution engine on integers other than INT8
.experimental_vectorize
cluster setting and session variables have been renamed to vectorize
. A new option auto
is introduced for both of these and is made the default option; with auto
, all "streaming" queries (i.e. those that do not need to buffer all temporary data before they can proceed - sorts, joins, etc) will now be executed via the vectorized engine. Options on
and always
have been renamed to experimental_on
and experimental_always
, respectively.@knz, @bdarnell, should we retroactively add these notes to their relevant release note pages? If so, I'll need authors to identify the releases. I suspect that's too much work, but I'm open to your guidance. More importantly, once all authors have completed their reviews here, I'll open docs issues, where relevant.
If ben and you decide that retroactive addition is the way to go, then we can automate the classification. That is not too much work.
However I am not sure this is the way to go? Or perhaps, not the only place where this should be reported: all the users that have already read the past release notes will not come across this new content. Maybe we should instead publish an "erratum" page specific to 19.2, even communicate it on the forum and perhaps other channels, and then link to that page from the other release note pages for reference.
I meant "addendum" not "erratum".
Whatever we do here should be fairly quiet. https://www.cockroachlabs.com/docs/releases/v19.2.0.html is already a summary of the release, and if we were going to have a more detailed "addendum" it would primarily include the union of all the alpha/beta/rc release notes before it would include these things barely got mentioned. It doesn't make sense to make a new page for these. Fitting these into their corresponding alpha/etc release notes is the only place we have to put these notes today and I'm not sure it's worth the effort.
Going forward, perhaps we should consider going to a model where minor releases are lumped together on one page, so we'd have three pages per branch: one for all the pre-releases of a branch (which would also serve as the detailed changelog for the release and would be a reasonable place to drop in unsorted "forgotten" changes like these), one for the release summary itself, and one for the post-release patches.
It would be nice to have a link to this issue/s from somewhere in docs.
I faced:
https://github.com/cockroachdb/cockroach/pull/39202
Looked into release notes and didn't found anything related.
Took me a while to land here.
I'm closing this issue in favor of discrete issues, which I will create soon.
Pursuant to https://github.com/cockroachdb/cockroach/pull/42345 I have scanned the entire 19.2 commit history (automatically) to pick up PRs where the author blindly used 'Release note: None' while there were user-facing changes. These are the changes that likely did not show up in release notes, and the doc team likely could not pick up the change to be documented.
If some of your changes are listed here, please review the PR(s) and suggest missing release notes + doc changes as a comment below.
Also don't forget the category!
Then click the checkbox on the left to indicate you've handled the change.
Note: I did a manual filtering on the automatic scan to remove false positives (no release note actually needed) but I may have left some. Mark the checkbox after you've confirmed that the PR indeed contains no user-facing change.
Changes without release note annotation
PRs merged by contributors
Hall of fame: all user-facing changes are properly documented
Honorable mentions: just a few release note omissions
Aayush Shah: @aayushshah15
Andy Kimball: @andy-kimball
Arul Ajmani: @arulajmani
Bilal Akhtar: @itsbilal
Bram Gruneir: @BramGruneir
George Papadrosou: @tbg
Irfan Sharif: @irfansharif
Pete Vilter: @vilterp
Peter Mattis: @petermattis
Ridwan Sharif: @jordanlewis
UPDATE ... FROM
statements (3 commits)Roland Crosby: @rolandcrosby
psql
Taufiq Rahman: @andreimatei
Vivek Menezes: @lucy-zhang
Missing release notes (rest)
Aditya Maru: @jordanlewis
Alfonso Subiotto Marqués: @asubiotto
Andrei Matei: @andreimatei
Andrew Kryczka: @ajkr
Andrew Werner: @ajwerner
Arseni Lapunov: @jordanlewis
Daniel Harrison: @danhhz
fixtures import
tooBen Darnell, Darin Peshev: @bdarnell @darinpp
Darin Peshev: @darinpp
David Taylor: @dt
Jeffrey Xiao: @jordanlewis
cockroach demo
Jordan Lewis, Lucy Zhang: @jordanlewis @lucy-zhang
Jordan Lewis, Nathan VanBenschoten: @jordanlewis @nvanbenschoten
Jordan Lewis: @jordanlewis
Justin Jaffray: @justinj
Lucy Zhang: @lucy-zhang
Nathan VanBenschoten: @nvanbenschoten
Nikhil Benesch, Raphael 'kena' Poss: @knz
Nikhil Benesch: @knz @bobvawter
Paul Bardea: @pbardea
Radu Berinde: @RaduBerinde
Rafi Shamim: @rafiss
Raphael 'kena' Poss: @knz
Rebecca Taft: @rytaft
Rohan Yadav: @rohany
Sean Loiselle: @nvanbenschoten
Solon Gordon: @solongordon
Spas Bojanov: @spaskob
Tobias Schottdorf: @tbg
Yahor Yuzefovich: @yuzefovich