trinodb / trino

Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)
https://trino.io
Apache License 2.0
9.88k stars 2.86k forks source link

Release notes for 341 #4755

Closed martint closed 3 years ago

martint commented 3 years ago

Dain Sundstrom

dain commented 3 years ago
## Prometheus Changes

* Change `timestamp` column to `TIMESTAMP WITH TIME ZONE` type.  ({issue}`4756`)

4756

martint commented 3 years ago
* Add support for variable-precision `TIME` type.  ({issue}`4381`)

https://github.com/prestosql/presto/pull/4381

losipiuk commented 3 years ago
## Oracle Changes

* Add `oracle.connection-pool.inactive-timeout` config property to specify how long 
  pooled connection can be inactive before it is closed. ({issue}`4779`)

4779

ebyhr commented 3 years ago
## Oracle Changes

* Add support for database globalization. ({issue}`4775`)

4775

losipiuk commented 3 years ago
General
* Fix serialization of NULL values in structural types for old Presto clients ({issue}`4778`)

4778, #4780, #4781

martint commented 3 years ago
* Fix failure when aggregation query contains duplicate expressions. ({issue}`4872`)

https://github.com/prestosql/presto/issues/4782

dain commented 3 years ago
General
* Only request HTTPS client certificate when certificate authentication is enabled. ({issue}`4804`)

4804

dain commented 3 years ago
* Fix compiler failure when querying long timestamp type. ({issue}`4824`)

4824

martint commented 3 years ago
Pinot
* Add support for `REAL` and `INTEGER` types. ({issue}`4725`)

https://github.com/prestosql/presto/pull/4725

dain commented 3 years ago
Securty
* Add :doc:`/security/salesforce` password authentication. ({issue}`4372`)

4372

dain commented 3 years ago
## General Changes

* Change timestamp operations to match the SQL specification. The value of a `TIMESTAMP` type is not affected by the session time zone.  ({issue}`4799`)
* Fix parsing failure of timestamps due to daylight saving changes. ({issue}`4799`)
* Remove the `deprecated.legacy-timestamp` configuration property and the `legacy_timestamp` session property. ({issue}`4799`)

## Hive Connector Changes

* Fix invalid timestamp values for nested data in Text, Avro, Sequence File, JSON and CSV formats. ({issue}`4799`)
* Add Parquet and RCBinary configuration properties `hive.parquet.time-zone` and `hive.rcfile.time-zone` to adjust binary timestamp values to a specific time zone.  For Hive 3.1+, this should be set to UTC. The default value is the JVM default time zone, for backwards compatibility with earlier versions of Hive. ({issue}`4799`)
* Add ORC configuration property `hive.orc.time-zone` to set the default time zone for legacy ORC files that did not declare a time zone. ({issue}`4799`)
* Replace the `hive.time-zone` configuration property with the format specific properties. ({issue}`4799`)

## Cassandra Connector Changes

* Map Cassandra `TIMESTAMP` type to Presto `TIMESTAMP(3) WITH TIME ZONE` type. ({issue}`4799`)

## JMX Connector Changes

* Change the type of the `timestamp` column for tables in the `history` schema to `TIMESTAMP(3) WITH TIME ZONE` type. ({issue}`4799`)

## Prometheus Connector Changes

* Change the type of the `timestamp` column to `TIMESTAMP(3) WITH TIME ZONE` type. ({issue}`4799`)

## Local File Connector Changes

* Change the type of the `timestamp` column in the `http_request_log` table to `TIMESTAMP(3) WITH TIME ZONE` type. ({issue}`4799`)

## Raptor Connector Changes

* Change timestamp internal storage to match Presto new semantics.  Tables containing timestamp data written by a client not running in UTC will see timestamps adjusted to UTC. ({issue}`4799`)
* Remove the `storage.shard-day-boundary-time-zone` configuration property, which was used to work around legacy timestamp semantics in Presto. ({issue}`4799`)

## {Kafka , Redis, Kinesis} Connector Changes

* Preserve time zone when parsing `TIMESTAMP WITH TIME ZONE` values. ({issue}`4799`)

## SPI Changes

* The `TIMESTAMP` type is encoded as a number of fractional seconds from `1970-01-01 00:00:00`, where every day is defined to contain exactly 86400 seconds. This value is no longer adjusted to the session time zone. ({issue}`4799`)
* Remove `isLegacyTimestamp()` from `ConnectorSession`. ({issue}`4799`)

4799

sopel39 commented 3 years ago
Client
* Expose physical input bytes read statistics. ({issue}`4600`}

https://github.com/prestosql/presto/pull/4600

sopel39 commented 3 years ago
General
* Improve query scalability when new nodes are added to cluster. ({issue}`4294`)

https://github.com/prestosql/presto/pull/4294

Praveen2112 commented 3 years ago
Kudu
* Fix delete when applied on table having primary key of decimal type.

4683

Praveen2112 commented 3 years ago
Phoenix
* Fix overwriting of former value when insert is applied without specifying that column. ({issue}`4562`)

4670

Praveen2112 commented 3 years ago
Hive
* Skip stripes and row group based on Timestamp statistics for ORC files. 

Iceberg
* Skip stripes and row group based on Timestamp statistics for ORC files.

1147

kokosing commented 3 years ago
Hive
* Use temporary staging directory for write operation to sorted bucketed table. ({issue}`3434`)

3434

findepi commented 3 years ago
PostgreSQL 
* Improve performance of aggregation queries with predicates (the `WHERE` clause) by computing both filtering and
  aggregations within PostgreSQL database where possible. ({issue}`4111`)

https://github.com/prestosql/presto/issues/4111 https://github.com/prestosql/presto/pull/4565

martint commented 3 years ago
* Fix failure when calling `EXTRACT` with `TIMEZONE_HOUR` and `TIMEZONE_MINUTE` for `TIMESTAMP WITH TIME ZONE` type. ({issue}`4867`)

4867

kokosing commented 3 years ago
Security
 * Allow to configure refresh period for system level access control with environment variables. ({issue}`4854`)

4854

findepi commented 3 years ago
General
* Add Iceberg connector
sopel39 commented 3 years ago
General
* Report dynamic filters stats. ({issue}`4440`)

https://github.com/prestosql/presto/pull/4440

kokosing commented 3 years ago
Hive
* Invalidate cached information on `GRANT` and `REVOKE`. ({issue}`4768`)

4768

sopel39 commented 3 years ago
General
* Reduce latency of how fast lazy dynamic filters are collected. ({issue}`4924`) 

https://github.com/prestosql/presto/pull/4924

sopel39 commented 3 years ago
UI
* Fix displaying physical input read time in detailed query view. ({issue}`4962`)

https://github.com/prestosql/presto/pull/4962

ebyhr commented 3 years ago
## Hive Changes

* Improve performance of partition fetching from Glue. ({issue}`4938`)

4938

sopel39 commented 3 years ago
General
* Fix query deadlock for connectors that wait for dynamic filters. ({issue}`4946`)

https://github.com/prestosql/presto/pull/4946

martint commented 3 years ago
* Add {func}`human_readable_seconds`. ({issue}`4344`)

4344

findepi commented 3 years ago
## JDBC Driver Changes

* Implement `ResultSet. getStatement()`. ({issue}`4957`)

4957

findepi commented 3 years ago
MySQL Connector Changes

* Improve performance of aggregation queries by computing aggregations within MySQL database.
  Currently, the following aggregate functions are eligible for pushdown:
  `count`,  `min`, `max`, `sum` and `avg`. ({issue}`4138`)

https://github.com/prestosql/presto/issues/4138 https://github.com/prestosql/presto/pull/4428

kokosing commented 3 years ago
Oracle
* Resilience to momentary JDBC connection authentication issues with Oracle. (:issue:`4947`)

4947

findepi commented 3 years ago
PostgreSQL
* Fix handling of PostgreSQL arrays when `unsupported-type-handling` is set to `CONVERT_TO_VARCHAR`. ({issue}`4981`)

https://github.com/prestosql/presto/issues/4981 https://github.com/prestosql/presto/pull/4983

findepi commented 3 years ago
Hive
* Fix query failure when reading a ORC ACID table with a filter, after the table underwent a minor compaction. ({issue}`4622`)

https://github.com/prestosql/presto/pull/4622

sopel39 commented 3 years ago
SPI
* Enable connectors to wait for dynamic filters before producing data on worker nodes. ({issue}`3414`)

https://github.com/prestosql/presto/pull/3414

kokosing commented 3 years ago
Security
* Allow to lowercase/uppercase matched user name in user mapping. ({issue}`4736`)

4736

Praveen2112 commented 3 years ago
Hive
* Skip DeepArchive objects when `skip-glacier-objects` is set. ({issue}`5002`)

5010

kokosing commented 3 years ago
General
 * Remove deprecated `reorder_joins` session property. ({issue}`5027`)

5027

losipiuk commented 3 years ago
Hive
 * Fix issue where queries could fail if Rubix caching was used and cached data was evicted during query execution. ({issue}`3580`)

3580 #4551

losipiuk commented 3 years ago
Oracle
 * Add possibility to force mapping of certain types to ``varchar``. This can be enabled
   by setting ``jdbc-types-mapped-to-varchar`` to comma-separated list of type names. ({issue}`4955`)

4955

losipiuk commented 3 years ago
Oracle
 * Properly pushdown predicates for queries where number of conjuncts exceeded 1000. Previously query would fail in such case. ({issue}`4918`)

4918

losipiuk commented 3 years ago
General
 * Fix issue where ``query_max_scan_physical_bytes`` session property 
   was ignored if ``query.max-scan-physical-bytes`` config property was not defined. ({issue}`5009`)

5009

losipiuk commented 3 years ago
BigQuery
 * Add support for HOURLY partitioning of tables. ({issue}`4968`)
 * Redact value of `bigquery.credentials-key` from logs. ({issue}`4968`)

4968

kokosing commented 3 years ago
Hive
 - Allow to optionally use cluster role in s3 mapping. ({issue}`4931`)

4931

findepi commented 3 years ago
General
- Fix incorrect sampling ratio when `TABLESAMPLE` is used with percentage fraction. ({issue}`5074`)
- Correct rejecting queries using `TABLESAMPLE` with ratio being not a number. ({issue}`5074`)

5074

electrum commented 3 years ago
# JMX Connector Changes

* Change `timestamp` column type in history tables to `TIMESTAMP WITH TIME ZONE`. ({issue}`4753`)

4753

electrum commented 3 years ago
# Local File Connector Changes

* Change `timestamp` column type to `TIMESTAMP WITH TIME ZONE`. ({issue}`4752`)

4752

electrum commented 3 years ago
# Hive Connector Changes

* Remove support for bucketing on timestamp. ({issue}`4759`)

4759

electrum commented 3 years ago
# General Changes

* Add `reverse()` function for `VARBINARY`. ({issue}`4741`)

4741

electrum commented 3 years ago
# General Changes

* Fail with an explicit error rather than `OutOfMemoryError` for certain operations. ({issue}`4890`)

4890

electrum commented 3 years ago
# General Changes

* Improve error message when JSON parsing fails. ({issue}`4616`)

4616