elastic / kibana

Your window into the Elastic Stack
https://www.elastic.co/products/kibana
Other
19.76k stars 8.16k forks source link

[Snapshot Restore] Not handling snapshot error #98782

Open alisonelizabeth opened 3 years ago

alisonelizabeth commented 3 years ago

Kibana version: 7.x

Describe the bug: The "Snapshots" tab in the Snapshot and Restore UI indicates there are no snapshots when in fact there was an error.

Steps to reproduce:

  1. Create a repository in Snapshot and Restore
  2. Copy the unzipped snapshot in https://github.com/elastic/kibana/issues/83194#issuecomment-732462855 into the repository created in step 1.
  3. Navigate to the "Snapshots" tab. Note the UI shows that no snapshots are found. However, if you look in the ES logs you will see the error:
[2021-04-29T11:57:37,192][WARN ][o.e.a.a.c.s.g.TransportGetSnapshotsAction] [Alisons-MacBook-Pro.local] failed to get snapshot [test-nzbkmuenr7c7mswcuesbvq/hiIv4DJoQN6PJz1dwbxS2w]
org.elasticsearch.snapshots.SnapshotMissingException: [test:test-nzbkmuenr7c7mswcuesbvq/hiIv4DJoQN6PJz1dwbxS2w] is missing
    at org.elasticsearch.repositories.blobstore.BlobStoreRepository.getSnapshotInfo(BlobStoreRepository.java:1205) ~[elasticsearch-7.14.0-SNAPSHOT.jar:7.14.0-SNAPSHOT]
    at org.elasticsearch.action.admin.cluster.snapshots.get.TransportGetSnapshotsAction.snapshots(TransportGetSnapshotsAction.java:200) [elasticsearch-7.14.0-SNAPSHOT.jar:7.14.0-SNAPSHOT]
    at org.elasticsearch.action.admin.cluster.snapshots.get.TransportGetSnapshotsAction.lambda$loadSnapshotInfos$1(TransportGetSnapshotsAction.java:159) [elasticsearch-7.14.0-SNAPSHOT.jar:7.14.0-SNAPSHOT]
    at org.elasticsearch.action.ActionRunnable.lambda$supply$0(ActionRunnable.java:47) [elasticsearch-7.14.0-SNAPSHOT.jar:7.14.0-SNAPSHOT]
    at org.elasticsearch.action.ActionRunnable$2.doRun(ActionRunnable.java:62) [elasticsearch-7.14.0-SNAPSHOT.jar:7.14.0-SNAPSHOT]
    at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:732) [elasticsearch-7.14.0-SNAPSHOT.jar:7.14.0-SNAPSHOT]
    at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) [elasticsearch-7.14.0-SNAPSHOT.jar:7.14.0-SNAPSHOT]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?]
    at java.lang.Thread.run(Thread.java:832) [?:?]
Caused by: java.nio.file.NoSuchFileException: /tmp/es_backups/snap-hiIv4DJoQN6PJz1dwbxS2w.dat
    at sun.nio.fs.UnixException.translateToIOException(UnixException.java:92) ~[?:?]
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106) ~[?:?]
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) ~[?:?]
    at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:218) ~[?:?]
    at java.nio.file.Files.newByteChannel(Files.java:375) ~[?:?]
    at java.nio.file.Files.newByteChannel(Files.java:426) ~[?:?]
    at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:420) ~[?:?]
    at java.nio.file.Files.newInputStream(Files.java:160) ~[?:?]
    at org.elasticsearch.common.blobstore.fs.FsBlobContainer.readBlob(FsBlobContainer.java:174) ~[elasticsearch-7.14.0-SNAPSHOT.jar:7.14.0-SNAPSHOT]
    at org.elasticsearch.repositories.blobstore.ChecksumBlobStoreFormat.read(ChecksumBlobStoreFormat.java:100) ~[elasticsearch-7.14.0-SNAPSHOT.jar:7.14.0-SNAPSHOT]
    at org.elasticsearch.repositories.blobstore.BlobStoreRepository.getSnapshotInfo(BlobStoreRepository.java:1203) ~[elasticsearch-7.14.0-SNAPSHOT.jar:7.14.0-SNAPSHOT]
    ... 9 more

If you run the request in Console, you'll see:

GET /_snapshot/<REPO_NAME>/_all
{
  "error" : {
    "root_cause" : [
      {
        "type" : "snapshot_missing_exception",
        "reason" : "[test:test-nzbkmuenr7c7mswcuesbvq/hiIv4DJoQN6PJz1dwbxS2w] is missing"
      }
    ],
    "type" : "snapshot_missing_exception",
    "reason" : "[test:test-nzbkmuenr7c7mswcuesbvq/hiIv4DJoQN6PJz1dwbxS2w] is missing",
    "caused_by" : {
      "type" : "no_such_file_exception",
      "reason" : "/tmp/es_backups/snap-hiIv4DJoQN6PJz1dwbxS2w.dat"
    }
  },
  "status" : 404
}

Expected behavior: It appears there's an issue with the snapshot itself. However, I think I'd expect the UI to surface this error rather than to just show that no snapshots are found.

Screen Shot 2021-04-29 at 12 06 58 PM
elasticmachine commented 3 years ago

Pinging @elastic/es-ui (Team:Elasticsearch UI)

elasticmachine commented 2 weeks ago

Pinging @elastic/kibana-management (Team:Kibana Management)