Release notes for 304

[electrum]

Dain Sundstrom

• [x] all checked
• 02-21 12:42:06 012af8c113 Disallow bucket filter for bucket evolution
• 02-22 08:29:43 0c1d2fd4a9 Short circuit block creation for null data streams in orc
• 02-22 08:29:43 efd2394045 Add benchmark for reading a null data stream

  David Phillips

• [ ] all checked
• 02-14 13:16:41 1036831bd0 Add test to verify extra credential delivery to ConnectorIdentity
• 02-14 17:00:26 82ff60d741 Include ORC encoding in writer validation
• 02-14 17:04:16 f086b525d6 Use and record writer time zone in ORC files
• 02-15 10:41:18 e54f938e52 Remove exists check when creating a file in S3
• 02-17 22:01:41 7467cf48a8 Make thrift metastore client retry configurable
• 02-22 14:38:43 d7a4142a79 Fix int value cast to float and then passed to Math.round
• 02-22 14:39:27 6bafbb16dd Minor fixes to TLS documentation
• 02-23 12:15:38 1f70a1b2af Map PostgreSQL JSON to Presto JSON
• 02-23 23:29:49 5b65d20476 Prefetch Parquet footer to reduce the number of read calls
• 02-23 23:36:37 50431de86b Move ClassLoaderSafeConnectorPageSink to correct package
• 02-24 17:46:35 d098f530d9 Allow PrestoS3FileSystem to support skipping Glacier files
• 02-25 17:56:37 02a2e379f8 Merge Hive scripts for creating test tables
• 02-25 17:56:37 949eb893b2 Remove unused retention property from Hive test tables
• 02-25 17:56:37 af79531619 Unify Parquet types test table
• 02-25 17:56:37 db53b971aa Ensure that Hive test tables exist for ORC and Parquet
• 02-25 17:56:37 eea240dab5 Use command mode of su for running Hive command
• 02-25 21:40:56 78cde41c3c Add Hive procedure to sync table partitions
• 02-25 23:50:59 73f8c5a4ca Remove deprecated datasources config property
• 02-26 02:18:30 13e7d803d5 Always run full RCFile tests
• 02-26 02:29:15 169d8285a7 Avoid repeated JsonCodec creation in RemoteNodeState

  Grzegorz Kokosiński

• [ ] all checked
• 02-20 14:32:15 48c2e368c4 Move constants to PrestoS3FileSystem
• 02-20 14:32:15 6f21bd2dc4 Add missing @Override
• 02-20 14:32:15 f499b7c53d Inject set of ConfigurationInitializers to HdfsConfigurationInitializer
• 02-25 04:45:35 ec72ab9fbf Limit access to ThriftMetastoreUtil methods
• 02-26 00:53:09 73dea9db4c Updated RPM installation & removal behavior
• 02-26 04:45:22 00fe3dab94 Use case sensitve user name for Hive sql-standard access control
• 02-26 04:45:22 cfc06ce29c Remove unused HivePrincipal class
• 02-26 04:45:22 fa5d7110aa Remove unused code

  Karol Sobczak

• [x] all checked
• 02-14 06:23:52 7ab4c60d05 Use predictable collections in classes that participate in planning
• 02-15 12:49:33 e7fa566079 Replace anonymous classes with lambda in TestDriver
• 02-20 02:08:00 44fb155f14 Shutdown executors after tests and benchmarks
• 02-20 07:07:06 9c92fce9a5 Replace lambdas with method references
• 02-21 02:52:19 17e3e09674 Port window operator to work processors
• 02-21 02:52:19 24868b3481 Add Spill To Disk for ORDER BY
• 02-21 02:52:19 2b821e1297 Use OrderingCompiler in WindowOperator spilling
• 02-21 02:52:19 472fb5ad36 Use OrderingCompiler in OrderBy spilling
• 02-21 02:52:19 48c8d348b7 Add feature toggle for ORDER BY spill
• 02-21 02:52:19 7200a16fc1 Introduce revocable aggregated memory context in OperatorContext
• 02-21 02:52:19 7e8f59a45a Add documentation for spill to disk for window and order by
• 02-21 02:52:19 94b4d0a42f Extract order by queries tests to separate class
• 02-21 02:52:19 b164253c7f Separate input PageSource work processor
• 02-21 02:52:19 c9d5041c1a Add feature toggle for WindowOperator spill
• 02-21 02:52:19 d35e5fb834 Convert revocable memory to user memory on OrderBy finish
• 02-21 02:52:19 d398d1c374 Implement spill to disk For WindowOperator
• 02-21 02:52:19 d76a830ae0 Convert revocable memory to user memory on fully buffered group
• 02-21 02:52:19 e7d8dd5e70 Use WorkProcessor in OrderByOperator
• 02-21 02:52:19 f01067ba9d Extract DummySpillerFactory from TestHashAggregationOperator
• 02-21 02:52:19 fcc18ff1c5 Extract window queries tests to separate class
• 02-21 04:32:08 41cfe7ef32 Display physical input rows/data size in detailed Query view
• 02-21 04:32:08 6a0e3a0e7a Compute physical and internal network positions explicitly
• 02-21 04:32:08 8954c17b46 Display internal network stats in Query detail page
• 02-21 04:32:08 9fbdc0600d Remove unused variable
• 02-21 04:32:08 b5701e6545 Remove physical/internalNetwork input stat fields from BasicQueryStats
• 02-21 04:32:08 c46d298ac6 Take into account physical/internal network stats for all stages
• 02-21 04:32:08 d933d15f2d Slightly increase info-title width to fit longer stats descriptions
• 02-21 07:25:32 252aae9d51 Restore d3 vendor library
• 02-22 04:35:13 b2ec5d8726 Add note about revocable memory and reserved memory pool
• 02-22 04:38:08 ce877abc3b Correct roles documentation for catalog support
• 02-25 02:05:35 8df714f2cd Remove unused enum
• 02-25 02:54:03 e0e9a1d6eb Add missing @VisibleForTesting annotation
• 02-25 06:36:17 9f3a48433d Remove unused isTableOwner method

  Martin Traverso

• [x] all checked
• 02-14 11:02:51 2edc5f6e37 Provide a valid connector session to ConnectorNodePartitioningProvider
• 02-14 11:37:04 d398751661 Fix a small typo at DriverFactory.createDriver() error message
• 02-17 08:07:34 907fcd1c71 Fix failure when resolving fully qualified column from view
• 02-20 10:07:52 1f1d4ad92b Add a comment about ColumnStatistics' unknown estimates semantics
• 02-20 13:48:03 100746266c Fix test for empty sql path
• 02-20 13:48:03 d57db47a55 Disallow empty identifiers
• 02-22 10:36:07 2b4ef73dcf Remove unnecessary parentheses around individual values
• 02-22 10:36:07 61e5bb38d2 Ensure argument order insensitivity for outer join derivation
• 02-22 10:36:07 968081a96f Fix formatting
• 02-22 10:36:07 bdc36188fb Fix property derivation for full join with non-trivial coalesce
• 02-22 10:36:07 cf675c304b Add missing toString to AggregationStepMatcher
• 02-22 10:36:07 cf7e2eda04 Change method visibility

  Piotr Findeisen

• [x] all checked
• 02-14 05:00:45 7c1f7d4876 Remove import of Builder
• 02-14 13:17:55 58cf77ff18 Fail build if one of the invocations of Kudu tests fails
• 02-15 01:12:37 96375f85c1 Report offending table name when metadata invalid
• 02-15 12:30:52 d0baef0e9e Remove incorrect check in QueryBuilder
• 02-18 08:12:42 19b086f341 Fix a small typo at DriverFactory c-tor error message
• 02-19 05:27:14 41a141df43 Remove unused parameter
• 02-19 11:25:46 e9c217ae4a Disable Airlift admin servlet
• 02-20 01:58:25 3c46d367ce Pass session to JdbcClient#toWriteMapping
• 02-20 12:21:10 6c4a28dab8 Disable Airlift admin servlet in Proxy
• 02-21 04:38:26 381bcbb03a Fix exception message in TupleDomain#toLinkedMap
• 02-22 03:13:30 130bbcbe0d Minor refactorings for elasticsearch plugin
• 02-22 03:13:30 2ff7179829 Remove unnecessary catch clause
• 02-22 03:13:30 4d3258a30d Remove installation of MultibindingsScanner
• 02-22 03:13:30 9652483cb2 Remove unnecessary null checks
• 02-22 03:13:30 ca91cd0607 Mark some fields as final
• 02-25 00:21:50 a0ebeec678 Skip unsupported types when pushing down in Cassandra
• 02-26 02:55:10 5d053b1f7f Rename variables in CassandraType
• 02-26 02:55:10 6a46ec1551 Support Cassandra smallint, tinyint and date types
• 02-26 08:35:23 dd5ec48ab5 Document Cassandra smallint, tinyint and date types

[electrum]

Hive Connector Changes
----------------------

* Record the configured Hive time zone when writing ORC files.
* Use the write time zone recorded in ORC files when reading timestamps.
  The configured Hive time zone, which was previously always used, is now
  used as a default when the writer did not record the time zone.

[martint]

* Fix failure when column from view is referenced using its fully qualified form. (:issue:`250`)

250

[electrum]

Hive Connector Changes
----------------------

* Allow configuring the retry policy for the Thrift metastore client.

240

[sopel39]

General Changes
---------------

* Add experimental support for Spill to Disk for OrderBy.
* Add experimental support for Spill to Disk for WindowOperator.

[sopel39]

General Changes
---------------
* Correctly report internal network/physical position count for operators.
* Sum internal network/physical data size and position count from all stages in query statistics.
* Report internal network/physical data size and position count in detailed query UI view.
* Fix broken timeline UI view.
* Make planning more deterministic.

[dain]

Hive Connector Changes
----------------------

* Fix wrong results when filtering on the hidden ``$bucket`` column for tables containing partitions with different bucket counts. Instead, queries will now fail in this case. (:issue:`286`)

286

[martint]

* Fix wrong results for queries involving ``FULL OUTER JOIN`` and :func:`coalesce` expressions over the join keys. (:issue:`288`)

288

[electrum]

Hive Connector Changes
----------------------

* Potentially reduce file system read operations when reading Parquet file footers.

296

[electrum]

PostgreSQL Connector Changes
----------------------------

* Map PostgreSQL ``json`` and ``jsonb`` types to Presto ``json`` type.

81

[electrum]

Hive Connector Changes
----------------------

* Allow ignoring Glacier objects in S3 rather than failing the query. This is
  disabled by default, as it may skip data that is expected to exist, but it can
  be enabled using the ``hive.s3.skip-glacier-objects`` configuration property.

305

[findepi]

Cassandra Connector Changes
---
* Support queries over tables containing partitioning columns for which predicate pushdown is currently not supported. (:issue:`252`)

https://github.com/prestosql/presto/pull/252

[electrum]

Hive Connector Changes
----------------------

* Add procedure ``system.sync_partition_metadata()`` to synchronize the partitions
  in the metastore with the partitions that are physically on the file system.

223

[electrum]

General Changes
---------------

* Remove deprecated ``datasources`` configuration property.

306

[kokosing]

RPM
-----
* Do not remove modified configuration files during uninstallation

267

[findepi]

Cassandra Connector Changes
--
* Support ``smallint``, ``tinyint`` and  ``date`` Cassandra types. (:issue:`141`)

https://github.com/prestosql/presto/pull/141

[kokosing]

Hive Connector Changes
------------------------
* Fix regression in Hive ``sql-standard`` connector access control that user with mixed case name was not capable to access their privileges.

289

[dain]

Hive Connector Changes
------------------------

* Improve performance of ORC reader for all null columns. (:issue:`229`) 

229

[martint]

* Improve error message when a query contains zero-length delimited identifiers. (:issue:`249`)

249