search

FusionAuth / fusionauth-issues

FusionAuth issue submission project
https://fusionauth.io
90 stars 12 forks source link

Import API can cause the elastic index `refresh_interval` to be set to `-1s` unintentionally #2679

Closed jobannon closed 3 months ago

jobannon commented 3 months ago

Concurrent Import API can cause the elastic index refresh_interval to be set to -1s unintentionally

Description

If two imports are completed at the roughly same time, a race condition exists where the refresh_internal might be set to -1 after the import is complete. Once an import completes, the refresh interval should be 1s and our default setting for the refresh interval is 1s. When the refresh_interval is -1, indexed data (well since the last time that ES has updated its cache) is not available in searches.

Observed versions

1.48.3

Affects versions

Steps to reproduce

Steps to reproduce the behavior:

  1. Run an import of users ("batch one")
  2. Run a second import shortly after the first ("batch two")
  3. Make sure that the second batch starts before the first batch completes, and that the second batch finishes after the first batch completes.
  4. Observe that the refresh interval is not returned to 1s after the import process completes.

Expected behavior

For performance reasons, we increase/turn off the refresh_interval during an import. However, we should always return this interval to a default like 1s.

Screenshots

If applicable, add screenshots to help explain your problem. Delete this section if it is not applicable.

Platform

FA Dev setup

Community guidelines

All issues filed in this repository must abide by the FusionAuth community guidelines.

Additional context

The refresh interval can be checked using the following command

curl -X GET "http://localhost:9021/fusionauth_user/_settings"

The refresh interval can be updated with the following command (must have access to ES service)

curl --request PUT \
  --url http://localhost:9021/fusionauth_user/_settings \
  --header 'Content-Type: application/json' \
  --data '{
        "index": {
                "refresh_interval": "1s"
        }
}

'

Documentation

Release Notes

Fixes a race condition in the bulk user import API /api/user/import that could set the Elasticsearch refresh_interval to -1 and prevent newly created users from being searchable. This condition was triggered by concurrent bulk requests with specific timing.

robotdan commented 3 months ago

Internal: