russcam
commented
4 years ago Closing this issue as all checklist items have been checked, and will be going into a 7.6.0 release shortly.
Closed russcam closed 4 years ago
russcam
commented
4 years ago Closing this issue as all checklist items have been checked, and will be going into a 7.6.0 release shortly.
Preliminary release notes for 7.6.0
(Generated from asciidoc release notes on 2020-02-03)
Breaking changes
Mapping
Breaking Java changes
Security
Deprecations
Analysis
Authorization
Distributed
[x] Deprecate synced flush #50835 (issue: #50776)
[x] Deprecate indices without soft-deletes #50502
Features/Indices APIs
Infra/Core
MULTIPLE AREA LABELS
[x] Ensure we emit a warning when using the deprecated template field. #50831 (issue: #49460)
[x] Add a deprecation warning regarding allocation awareness in search request #48351 (issue: #43453)
Reindex
Search
[x] Update the signature of vector script functions. #48604
[x] Deprecate the sparse_vector field type. #48315
New features
Aggregations
[x] New Histogram field mapper that supports percentiles aggregations. #48580 (issue: #48578) PR #4367
[x] Implement stats aggregation for string terms #47468 PR #4370
Analysis
Features/ILM+SLM
[x] ILM action to wait for SLM policy execution #50454 (issue: #45067) PR #4375
[x] Add ILM histore store index #50287 (issue: #49180)
Features/Ingest
MULTIPLE AREA LABELS
Machine Learning
[x] Implement
precisionandrecallmetrics for classification evaluation #49671 (issue: #48759)[x] [ML] Explain data frame analytics API #49455
[x] [ML] ML Model Inference Ingest Processor #49052 draft PR https://github.com/elastic/elasticsearch-net/pull/4412
[x] Implement accuracy metric for multiclass classification #47772 (issue: #48759)
Mapping
Search
[x] Add fuzzy intervals source #49762 (issue: #49595) PR #4379
[x] Add a listener to track the progress of a search request locally #49471 (issue: #49091)
Enhancements
Aggregations
[x] Add reusable HistogramValue object #49799 (issue: #49683)
[x] Optimize composite aggregation based on index sorting #48399 (issue: #48130)
Allocation
[x] Auto-expand indices according to allocation filtering rules #48974
[x] Do not cancel ongoing recovery for noop copy on broken node #48265 (issue: #47974)
[x] Quieter logging from the DiskThresholdMonitor #48115 (issue: #48038)
[x] Faster access to INITIALIZING/RELOCATING shards #47817 (issues: #46941, #48579)
Analysis
[x] Check for deprecations when analyzers are built #50908 (issue: #42349)
[x] Make Multiplexer inherit filter chains analysis mode #50662 (issue: #50554)
[x] Allow custom characters in token_chars of ngram tokenizers #49250 (issue: #25894) PR #4384
Authentication
Authorization
[x] Increase Size and lower TTL on DLS BitSet Cache #50535 (issues: #43669, #49260)
[x] Add monitor_snapshot cluster privilege #50489 (issue: #50210)
[x] Remove reserved roles for code search #50068 (issue: #49842)
[x] [Code] Remove code_admin/code_user roles #48164
[x] Resolve the role query and the number of docs lazily #48036
CCR
[x] Improve error message when pausing index #48915
[x] Use MultiFileTransfer in CCR remote recovery #44514 (issue: #44468)
CRUD
[x] print id detail when id is too long. #49433
[x] Add preflight check to dynamic mapping updates #48817 (issue: #35564)
Cluster Coordination
[x] Move metadata storage to Lucene #50907 (issue: #48701)
[x] Remove custom metadata tool #50813 (issue: #48701)
Distributed
Engine
[x] Do not force refresh when write indexing buffer #50769
[x] Deleted docs disregarded for if_seq_no check #50526
[x] Allow realtime get to read from translog #48843
[x] Do not warm up searcher in engine constructor #48605 (issue: #47186)
[x] Add a new merge policy that interleaves old and new segments on force merge #48533 (issue: #37043)
[x] Refresh should not acquire readLock #48414 (issue: #47186)
Features/ILM+SLM
[x] Refresh cached phase policy definition if possible on new poli… #50820 (issue: #48431)
[x] Make the UpdateRolloverLifecycleDateStep retryable #50702 (issue: #48183)
[x] Make InitializePolicyContextStep retryable #50685 (issue: #48183)
[x] ILM retryable async action steps #50522 (issues: #44135, #48183)
[x] Make the TransportRolloverAction execute in one cluster state update #50388
[x] ILM open/close steps are noop if idx is open/close #48614
[x] ILM Make the
check-rollover-readystep retryable #48256 (issue: #44135) PR #4391Features/Ingest
[x] Foreach processor - fork recursive call #50514
[x] Sync grok patterns with logstash patterns #50381
[x] Replace required pipeline with final pipeline #49470 (issue: #49247) PR https://github.com/elastic/elasticsearch-net/pull/4401
[x] Add templating support to enrich processor #49093
[x] Introduce on_failure_pipeline ingest metadata inside on_failure block #49076 (issue: #44920)
[x] Add templating support to pipeline processor. #49030 (issue: #39955)
[x] Add option to split processor for preserving trailing empty fields #48664 (issue: #48498) PR https://github.com/elastic/elasticsearch-net/pull/4402
[x] Change grok watch dog to be Matcher based instead of thread based. #48346 (issues: #43673, #47374)
[x] update ingest-user-agent regexes.yml #47807
Features/Java High Level REST Client
[x] Add remote info to the HLRC #49657 (issue: #47678)
[x] Add delete alias to the HLRC #48819 (issue: #47678)
Features/Monitoring
[x] Significantly Lower Monitoring HttpExport Memory Footprint #48854
[x] Validate proxy base path at parse time #47912 (issue: #47711)
[x] Validate index name time format setting at parse time #47911 (issue: #47711)
[x] Validate monitoring header overrides at parse time #47848 (issue: #47711)
[x] Validate monitoring username at parse time #47821 (issue: #47711)
[x] Validate monitoring password at parse time #47740 (issue: #47711)
Features/Watcher
[x] Log attachment generation failures #50080
[x] Don’t dump a stacktrace for invalid patterns when executing elasticse… #49744 (issue: #49642)
Geo
[x] "CONTAINS" support for BKD-backed geo_shape and shape fields #50141 (issue: #41204)
[x] Adds support for geo-bounds filtering in geogrid aggregations #50002 PR https://github.com/elastic/elasticsearch-net/pull/4403
[x] Introduce faster approximate sinh/atan math functions #49009 (issue: #41166)
[x] Add IndexOrDocValuesQuery to GeoPolygonQueryBuilder #48449
Infra/Core
[x] Add "did you mean" to ObjectParser #50938
[x] Consistent case in CLI option descriptions #49635
[x] Improve resiliency to formatting JSON in server #48553 (issue: #48450)
[x] Don’t close stderr under
--quiet#47208 (issue: #46900)Infra/Packaging
[x] Respect ES_PATH_CONF on package install #50158
[x] Restrict support for CMS to pre-JDK 14 #49123 (issue: #46973)
[x] Remove parsed JVM settings from general settings in Windows service daemon manager #49061 (issue: #48796)
[x] Package the JDK into jdk.app on macOS #48765
[x] Add UBI-based Docker images #48710 (issue: #48429)
Infra/Plugins
[x] Report progress of multiple plugin installs #51001 (issue: #50924)
[x] Allow installing multiple plugins as a transaction #50924 (issue: #50443)
Infra/Scripting
[x] Scripting: ScriptFactory not required by compile #50344 (issue: #49466)
[x] Scripting: Cache script results if deterministic #50106 (issue: #49466)
[x] Scripting: Groundwork for caching script results #49895 (issue: #49466)
[x] Scripting: add available languages & contexts API #49652 (issue: #49463) EXPERIMENTAL API
[x] Scripting: fill in get contexts REST API #48319 (issue: #47411) EXPERIMENTAL API
[x] Scripting: get context names REST API #48026 (issue: #47411) EXPERIMENTAL API
Infra/Settings
[x] Add parameter to make sure that log of updating IndexSetting be more detailed #49969 (issue: #49818)
[x] Enable dependent settings values to be validated #49942
[x] Do not reference values for filtered settings #48066
License
[x] Add max_resource_units to enterprise license #50735
[x] Add setting to restrict license types #49418 (issue: #48508)
[x] Support "enterprise" license types #49223 (issue: #48510) PR https://github.com/elastic/elasticsearch-net/pull/4406
MULTIPLE AREA LABELS
Machine Learning
[x] [ML] Add audit warning for 1000 categories found early in job #51146 (issue: #50749)
[x] [ML] Add num_top_feature_importance_values param to regression and classi… #50914
[x] [ML] Implement force deleting a data frame analytics job #50553 (issue: #48124) will be picked up by generator
[x] [ML] Delete unused data frame analytics state #50243
[x] Make each analysis report desired field mappings to be copied #50219 (issue: #50119)
[x] [ML] retry bulk indexing of state docs #50149 (issue: #50143)
[x] [ML] Persist/restore state for DFA classification #50040
[x] [ML] Introduce randomize_seed setting for regression and classification #49990
[x] Pass
prediction_field_typeto C++ analytics process #49861 (issue: #49796)[x] [ML] Add optional source filtering during data frame reindexing #49690 (issue: #49531)
[x] [ML] Add default categorization analyzer definition to ML info #49545 PR https://github.com/elastic/elasticsearch-net/pull/4408
[x] [ML] Add graceful retry for anomaly detector result indexing failures #49508 (issue: #45711)
[x] Lower minimum model memory limit value from 1MB to 1kB. #49227 (issue: #49168)
[x] Throw an exception when memory usage estimation endpoint encounters empty data frame. #49143 (issue: #49140)
[x] Change format of MulticlassConfusionMatrix result to be more self-explanatory #48174 (issue: #46735)
[x] Make num_top_classes parameter’s default value equal to 2 #48119 (issue: #46735)
[x] [ML] Improve model_memory_limit UX for data frame analytics jobs [ISSUE] #44699
Mapping
Network
[x] Add certutil http command #49827
[x] Do not load SSLService in plugin contructor #49667 (issue: #44536)
[x] Netty4: switch to composite cumulator #49478
[x] Add the simple strategy to cluster settings #49414 (issue: #49067)
[x] Deprecate misconfigured SSL server config #49280 (issue: #45892)
[x] Improved diagnostics for TLS trust failures #48911
Percolator
Ranking
search_typein Rank Evaluation API #48542 (issue: #48503)Recovery
[x] Use peer recovery retention leases for indices without soft-deletes #50351 (issues: #45136, #46959)
[x] Recovery buffer size 16B smaller #50100
Reindex
SQL
[x] SQL: Add TRUNC alias for TRUNCATE #49571 (issue: #41195)
[x] SQL: binary communication implementation for drivers and the CLI #48261 (issue: #47785)
Search
[x] Add Validation for maxQueryTerms to be greater than 0 for MoreLikeThisQuery #49966 (issue: #49927)
[x] Optimize numeric sort on match_all queries #49717 (issue: #48804)
[x] Pre-sort shards based on the max/min value of the primary sort field #49092 (issue: #49091)
[x] Optimize sort on long field #48804
[x] Search optimisation - add canMatch early aborts for queries on "_index" field #48681 (issue: #48473)
[x] #48475 Pure disjunctions should rewrite to a MatchNoneQueryBuilder #48557
[x] Disable caching when queries are profiled #48195 (issue: #33298)
[x] BlendedTermQuery’s equals method should consider boosts #48193 (issue: #48184)
[x] Increase the number of vector dims to 2048 #46895
Security
[x] Make .async-search-* a restricted namespace #50294
[x] Security should not reload files that haven’t changed #50207 (issue: #50063)
Snapshot/Restore
[x] Use Cluster State to Track Repository Generation #49729 (issue: #49060)
[x] Track Repository Gen. in BlobStoreRepository #48944 (issues: #38941, #47520, #47834, #49048)
[x] Restore from Individual Shard Snapshot Files in Parallel #48110 (issue: #42791)
[x] Track Shard-Snapshot Index Generation at Repository Root #46250 (issues: #38941, #45736)
Store
Transform
[x] [Transform] Improve force stop robustness in case of an error #51072
[x] [Transform] add actual timeout in message #50140
[x] [Transform] automatic deletion of old checkpoints #49496
[x] [Transform] improve error handling of script errors #48887 (issue: #48467)
[x] [ML][Transforms] add wait_for_checkpoint flag to stop #47935 (issue: #45293)
Bug fixes
Aggregations
[x] Fix format problem in composite of unmapped #50869 (issue: #50600)
[x] SingleBucket aggs need to reduce their bucket’s pipelines first #50103 (issue: #50054)
[x] Avoid precision loss in DocValueFormat.RAW#parseLong #49063 (issue: #38692)
[x] Fix ignoring missing values in min/max aggregations #48970 (issue: #48905)
Allocation
[x] Collect shard sizes for closed indices #50645 (issue: #33888)
[x] Auto-expand replicated closed indices #48973
[x] Ignore dangling indices created in newer versions #48652 (issue: #34264)
[x] Handle negative free disk space in deciders #48392 (issue: #48380)
Analysis
[x] Throw Error on deprecated nGram and edgeNGram custom filters #50376 (issue: #50360)
[x] _analyze api does not correctly use normalizers when specified #48866 (issue: #48650)
Audit
Authentication
[x] Don’t fallback to anonymous for tokens/apikeys #51042 (issue: #50171)
[x] Populate User metadata with OpenIDConnect collections #50521 (issue: #50250)
[x] Always return 401 for not valid tokens #49736 (issue: #38866)
[x] Fix iterate-from-1 bug in smart realm order #49473
[x] Remove unnecessary details logged for OIDC #48746
[x] Add owner flag parameter to the rest spec #48500 (issue: #48499) will be picked up by generator.
Authorization
[x] Fix memory leak in DLS bitset cache #50635 (issue: #49261)
[x] Validate field permissions when creating a role #50212 (issues: #46275, #48108)
[x] Validate field permissions when creating a role #48108 (issue: #46275)
CCR
CRUD
[x] Block too many concurrent mapping updates #51038 (issue: #50670)
[x] Ensure meta and document field maps are never null in GetResult #50112 (issue: #48215)
[x] Replicate write actions before fsyncing them #49746
[x] Do not mutate request on scripted upsert #49578 (issue: #48670)
[x] Fix Transport Stopped Exception #48930 (issue: #42612)
[x] Return consistent source in updates #48707
[x] Close query cache on index service creation failure #48230 (issue: #48186)
Cluster Coordination
[x] Import replicated closed dangling indices #50649
[x] Ignore metadata of deleted indices at start #48918
[x] Make elasticsearch-node tools custom metadata-aware #48390
Discovery-Plugins
[x] Make EC2 Discovery Cache Empty Seed Hosts List #50607 (issue: #50550)
[x] Make EC2 Discovery Plugin Retry Requests #50550 (issue: #50462)
Distributed
Engine
[x] Account trimAboveSeqNo in committed translog generation #50205 (issue: #49970)
[x] Greedily advance safe commit on new global checkpoint #48559 (issue: #48532)
[x] Do not ignore exception when trim unreferenced readers #48470
Features/Features
Features/ILM+SLM
[x] Fix SLM check for restore in progress #50868
[x] Handle failure to retrieve ILM policy step better #49193 (issue: #49128)
[x] Don’t halt policy execution on policy trigger exception #49128
[x] Re-read policy phase JSON when using ILM’s move-to-step API #48827
[x] Don’t schedule SLM jobs when services have been stopped #48658 (issue: #47749)
[x] Ensure SLM stats does not block an in-place upgrade from 7.4 #48367
[x] Ensure SLM stats does not block an in-place upgrade from 7.4 #48361
[x] Add SLM support to xpack usage and info APIs #48096 (issue: #43663)
[x] Change policy_id to list type in slm.get_lifecycle #47766 (issue: #47765) will be picked up by generator
Features/Ingest
[x] Fix ignore_missing in CsvProcessor #51600
[x] Don’t overwrite target field with SetSecurityUserProcessor #51454 (issue: #51428)
[x] Fix ingest simulate response document order if processor executes async #50244
[x] Allow list of IPs in geoip ingest processor #49573 (issue: #46193) PR https://github.com/elastic/elasticsearch-net/pull/4410
[x] Do not wrap ingest processor exception with IAE #48816 (issue: #48810)
[x] Introduce dedicated ingest processor exception #48810 (issue: #48803)
Features/Java High Level REST Client
[x] Support es7 node http publish_address format #49279 (issue: #48950)
[x] Add slices to delete and update by query in HLRC #48420
[x] fix incorrect comparison #48208
[x] Fix HLRC parsing of CancelTasks response #47017
Features/Java Low Level REST Client
Features/Monitoring
Geo
[x] Guard against null geoBoundingBox #50506 (issue: #50505)
[x] Geo: Switch generated GeoJson type names to camel case (#50285) #50400 (issue: #49568)
[x] Geo: Switch generated WKT to upper case #50285 (issue: #49568)
[x] Fix typo when assigning null_value in GeoPointFieldMapper #49645
[x] Fix handling of circles in legacy geo_shape queries #49410 (issue: #49296)
[x] GEO: intersects search for geo_shape return wrong result [ISSUE] #49017
[x] Geo: improve handling of out of bounds points in linestrings #47939 (issue: #43916)
Highlighting
Infra/Core
[x] Ignore virtual ethernet devices that disappear #51581 (issue: #49914)
[x] Guess root cause support unwrap #50525 (issue: #50417)
[x] Allow parsing timezone without fully provided time #50178 (issue: #49351)
[x] [Java.time] Retain prefixed date pattern in formatter #48703 (issue: #48698)
[x] Don’t drop user’s MaxDirectMemorySize flag on jdk8/windows #48657 (issues: #44174, #48365)
[x] Warn when MaxDirectMemorySize may be incorrect (Windows/JDK8 only issue) #48365 (issue: #47384)
[x] [Java.time] Calculate week of a year with ISO rules #48209 (issues: #41670, #42588, #43275, #43652)
Infra/Logging
Infra/Packaging
Infra/REST API
[x] Return 400 when handling invalid JSON #49552 (issue: #49428)
[x] Slash missed in indices.put_mapping url #49468
MULTIPLE AREA LABELS
[x] Fix meta version of task index mapping #50363 (issue: #48393)
[x] Prevent deadlock by using separate schedulers #48697 (issues: #41451, #47599)
[x] APM system_user #47668 (issues: #2708, #40876)
Machine Learning
[x] [ML] Validate classification dependent_variable cardinality is at lea… #51232
[x] Do not copy mapping from dependent variable to prediction field in regression analysis #51227
[x] Handle nested and aliased fields correctly when copying mapping. #50918 (issue: #50787)
[x] [ML] Fix off-by-one error in ml_classic tokenizer end offset #50655
[x] [ML] Improve uniqueness of result document IDs #50644 (issue: #50613)
[x] [7.x] Synchronize processInStream.close() call #50581 (issue: #49680)
[x] Fix accuracy metric #50310 (issue: #48759)
[x] Fix race condition when stopping a data frame analytics jobs immediately after starting it #50276 (issues: #49680, #50177)
[x] Use query in cardinality check #49939
[x] Make only a part of
stop()method a critical section. #49756 (issue: #49680)[x] Fix expired job results deletion audit message #49560 (issue: #49549)
[x] [ML] Apply source query on data frame analytics memory estimation #49517 (issue: #49454)
[x] Stop timing stats failure propagation #49495
[x] [ML] Fix r_squared eval when variance is 0 #49439
[x] Blacklist a number of prediction field names. #49371 (issue: #48808)
[x] Make AnalyticsProcessManager class more robust #49282 (issue: #49095)
[x] [ML] Fixes for stop datafeed edge cases #49191 (issues: #43670, #48931)
[x] [ML] Avoid NPE when node load is calculated on job assignment #49186 (issue: #49150)
[x] Do not throw exceptions resulting from persisting datafeed timing stats. #49044 (issue: #49032)
[x] [ML] Deduplicate multi-fields for data frame analytics #48799 (issues: #48756, #48770)
[x] [ML] Prevent fetching multi-field from source #48770 (issue: #48756)
Mapping
[x] Ensure that field collapsing works with field aliases. #50722 (issues: #32648, #50121)
[x] Improve DateFieldMapper
ignore_malformedhandling #50090 (issues: #46675, #50081)[x] Annotated text type should extend TextFieldType #49555 (issue: #49289)
[x] Ensure parameters are updated when merging flattened mappings. #48971 (issue: #48907)
Network
Percolator
[x] Correctly handle MSM for nested disjunctions #50669 (issue: #50305)
[x] Fix query analyzer logic for mixed conjunctions of terms and ranges #49803 (issue: #49684)
Recovery
[x] Check allocation id when failing shard on recovery #50656 (issue: #50508)
[x] Migrate peer recovery from translog to retention lease #49448 (issue: #45136)
[x] Ignore Lucene index in peer recovery if translog corrupted #49114
Reindex
SQL
[x] SQL: Optimisation fixes for conjunction merges #50703 (issue: #49637)
[x] SQL: Fix issue with CAST and NULL checking. #50371 (issue: #50191)
[x] SQL: fix NPE for JdbcResultSet.getDate(param, Calendar) calls #50184 (issue: #50174)
[x] SQL: COUNT DISTINCT returns 0 instead of NULL for no matching docs #50037 (issue: #50013)
[x] Fix LOCATE function optional parameter handling #49666 (issue: #49557)
[x] Fix NULL handling for FLOOR and CEIL functions #49644 (issue: #49556)
[x] Handle NULL arithmetic operations with INTERVALs #49633 (issue: #49297)
[x] Fix issue with GROUP BY YEAR() #49559 (issue: #49386)
[x] Fix issue with CASE/IIF pre-calculating results #49553 (issue: #49388)
[x] Fix issue with folding of CASE/IIF #49449 (issue: #49387)
[x] Fix issues with WEEK/ISO_WEEK/DATEDIFF #49405 (issues: #48209, #49376)
[x] SQL: Fix issue with mins & hours for DATEDIFF #49252
[x] SQL: Failing Group By queries due to different ExpressionIds #43072 (issues: #33361, #34543, #36074, #37044, #40001, #40240, #41159, #42041, #46316)
Search
[x] Fix upgrade of custom similarity #50851 (issue: #50763)
[x] Fix NPE bug inner_hits #50709 (issue: #50539)
[x] Collect results in a local list when notifying partial results #49828 (issue: #49778)
[x] Fixes a bug in interval filter serialization #49793 (issue: #49519)
[x] Correctly handle duplicates in unordered interval matching #49775
[x] Correct rewritting of script_score query #48425 (issue: #48081)
[x] Do not throw errors on unknown types in SearchAfterBuilder #48147 (issue: #48074)
Security
Snapshot/Restore
[x] Fix Overly Aggressive Request DeDuplication #51270 (issue: #51253)
[x] Guard Repository#getRepositoryData for exception throw #50970
[x] Fix Index Deletion During Partial Snapshot Create #50234 (issues: #50200, #50202)
[x] Fix Index Deletion during Snapshot Finalization #50202 (issues: #45689, #50200)
[x] Fix RepoCleanup not Removed on Master-Failover #49217
[x] Make FsBlobContainer Listing Resilient to Concurrent Modifications #49142 (issue: #37581)
[x] Fix SnapshotShardStatus Reporting for Failed Shard #48556 (issue: #48526)
[x] Cleanup Concurrent RepositoryData Loading #48329 (issue: #48122)
Transform
[x] [Transform] fail to start/put on missing pipeline #50701 (issue: #50135)
[x] Fix possible audit logging disappearance after rolling upgrade #49731 (issue: #49730)
[x] [Transform] do not fail checkpoint creation due to global checkpoint mismatch #48423 (issue: #48379)
Upgrades
Engine
Infra/Packaging
NOT CLASSIFIED
Aggregations
Authentication