插件添加oss后误报： [ErrorCode]: NoSuchKey。

[oldthreefeng]

目前我这边运行了一个6.8.8版本的es集群，

想安装这个插件。 发现如下报错。 请问如何解决呢？

Exception in thread "main" java.lang.IllegalArgumentException: Plugin [elasticsearch-repository-oss] was built for Elasticsearch version 6.8.6 but version 6.8.8 is running
#5 24.07        at org.elasticsearch.plugins.PluginsService.verifyCompatibility(PluginsService.java:346)
#5 24.07        at org.elasticsearch.plugins.InstallPluginCommand.loadPluginInfo(InstallPluginCommand.java:728)
#5 24.07        at org.elasticsearch.plugins.InstallPluginCommand.installPlugin(InstallPluginCommand.java:803)
#5 24.07        at org.elasticsearch.plugins.InstallPluginCommand.install(InstallPluginCommand.java:786)
#5 24.07        at org.elasticsearch.plugins.InstallPluginCommand.execute(InstallPluginCommand.java:232)
#5 24.07        at org.elasticsearch.plugins.InstallPluginCommand.execute(InstallPluginCommand.java:217)
#5 24.07        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
#5 24.07        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124)
#5 24.07        at org.elasticsearch.cli.MultiCommand.execute(MultiCommand.java:77)
#5 24.07        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124)
#5 24.07        at org.elasticsearch.cli.Command.main(Command.java:90)

[oldthreefeng]

非5.5.3需要修改plguins/plugin-descriptor.properties中的elasticsearch.version和version,改为自己es集群的版本

非6.8.6需要修改plguins/plugin-descriptor.properties中的elasticsearch.version和version,改为自己es集群的版本

可以这么理解么?

[oldthreefeng]

改了之后。

curl -H "Content-Type: application/json" -XPUT localhost:9200/_snapshot/es_backup_test \
-d '{
    "type":"oss",
    "settings":{
        "endpoint":"https://oss-accelerate.aliyuncs.com",
        "access_key_id":"xxxxxx",
        "secret_access_key":"xxxxxxxxxx",
        "bucket":"****",
        "compress":true,
        "base_path": "snapshot/"
    }
}'
{"acknowledged":true}

查询es日志后

[2021-06-11T06:01:34,226][INFO ][c.a.oss                  ] [9VBOvSU] [Server]Unable to execute HTTP request: The specified key does not exist.
[ErrorCode]: NoSuchKey
[RequestId]: 60C2FC3F35879138A325371D
[HostId]: *****.oss-accelerate.aliyuncs.com
[ResponseError]:
<?xml version="1.0" encoding="UTF-8"?>
<Error>
  <Code>NoSuchKey</Code>
  <Message>The specified key does not exist.</Message>
  <RequestId>60C2FC3F35879138A325371D</RequestId>
  <HostId>es-backup-kupeiai.oss-accelerate.aliyuncs.com</HostId>
  <Key>snapshot/tests-olOzPM5ISmWebgDZcWcvEw/data-9VBOvSU7T661254eNpbpuA.dat</Key>
</Error>

[oldthreefeng]

[ErrorCode]: NoSuchKey

这个错误请忽略。 经过测试， 是可以备份的。 不过在第一步添加 oss的会报这个错误。 最好开发者能去掉这个， 避免误导

[qq752982939]

[ErrorCode]: NoSuchKey

错误这个请忽略。经过测试，是可以备份的。浏览本次添加oss的报告错误。

请问您在7.X版本的集群上使用过此插件吗? 我修改了拉取了最新的https://github.com/aliyun/elasticsearch-repository-oss/releases/download/v7.7.1/elasticsearch-repository-oss-7.7.1.zip 按照您的说法将plguins/plugin-descriptor.properties中的elasticsearch.version和version修改为7.8.0 但是我安装插件启动后出现 [2021-08-25T00:22:16,546][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [node-064] [controller/5885] [Main.cc@110] controller (64 bit): Version 7.8.0 (Build 58ff6912e20047) Copyright (c) 2020 Elasticsearch BV fatal error in thread [main], exiting java.lang.NoClassDefFoundError: org/elasticsearch/cluster/metadata/RepositoryMetaData at org.elasticsearch.plugin.repository.oss.OssRepositoryPlugin.getRepositories(OssRepositoryPlugin.java:54) at org.elasticsearch.repositories.RepositoriesModule.(RepositoriesModule.java:49) at org.elasticsearch.node.Node.(Node.java:522) at org.elasticsearch.node.Node.(Node.java:266) at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:227) at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:227) at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:393) at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:161) at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:127) at org.elasticsearch.cli.Command.main(Command.java:90) at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126) at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) Caused by: java.lang.ClassNotFoundException: org.elasticsearch.cluster.metadata.RepositoryMetaData at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:435) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589) at java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:855) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522) ... 14 more [2021-08-25T00:22:18,300][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [node-064] fatal error in thread [main], exiting java.lang.NoClassDefFoundError: org/elasticsearch/cluster/metadata/RepositoryMetaData at org.elasticsearch.plugin.repository.oss.OssRepositoryPlugin.getRepositories(OssRepositoryPlugin.java:54) ~[?:?] at org.elasticsearch.repositories.RepositoriesModule.(RepositoriesModule.java:49) ~[elasticsearch-7.8.0.jar:7.8.0] at org.elasticsearch.node.Node.(Node.java:522) ~[elasticsearch-7.8.0.jar:7.8.0] at org.elasticsearch.node.Node.(Node.java:266) ~[elasticsearch-7.8.0.jar:7.8.0] at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:227) ~[elasticsearch-7.8.0.jar:7.8.0] at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:227) ~[elasticsearch-7.8.0.jar:7.8.0] at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:393) ~[elasticsearch-7.8.0.jar:7.8.0] at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) ~[elasticsearch-7.8.0.jar:7.8.0] at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:161) ~[elasticsearch-7.8.0.jar:7.8.0] at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.8.0.jar:7.8.0] at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:127) ~[elasticsearch-cli-7.8.0.jar:7.8.0] at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.8.0.jar:7.8.0] at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126) ~[elasticsearch-7.8.0.jar:7.8.0] at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.8.0.jar:7.8.0] Caused by: java.lang.ClassNotFoundException: org.elasticsearch.cluster.metadata.RepositoryMetaData at java.net.URLClassLoader.findClass(URLClassLoader.java:435) ~[?:?] at java.lang.ClassLoader.loadClass(ClassLoader.java:589) ~[?:?] at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:855) ~[?:?] at java.lang.ClassLoader.loadClass(ClassLoader.java:522) ~[?:?] ... 14 more

经排查 elasticsearch7.8.0中的RepositoryMetaData已经更改为RepositoryMetadata 但是似乎aliyun并没有提供可供修改的代码 请问您有出现过类似的情况吗

[oldthreefeng]

目前我们有这些版本.. 都兼容好了.@qq752982939

$ k get es -A
NAMESPACE            NAME      HEALTH   NODES   VERSION   PHASE   AGE
deps-prod            elastic   green    3       7.4.0     Ready   77d
deps-tx              elastic   green    3       7.4.0     Ready   61d
elastic-system       elastic   green    3       7.4.2     Ready   47d
prod                 elastic   green    3       7.4.2     Ready   422d
rasp-prod            elastic   green    3       6.8.8     Ready   454d
skywalking-prod-v8   elastic   green    3       7.4.2     Ready   228d
sls-prod             elastic   green    3       6.8.8     Ready   427d

[qq752982939]

目前我们有这些版本.. 都兼容好了.@qq752982939

$ k get es -A
NAMESPACE            NAME      HEALTH   NODES   VERSION   PHASE   AGE
deps-prod            elastic   green    3       7.4.0     Ready   77d
deps-tx              elastic   green    3       7.4.0     Ready   61d
elastic-system       elastic   green    3       7.4.2     Ready   47d
prod                 elastic   green    3       7.4.2     Ready   422d
rasp-prod            elastic   green    3       6.8.8     Ready   454d
skywalking-prod-v8   elastic   green    3       7.4.2     Ready   228d
sls-prod             elastic   green    3       6.8.8     Ready   427d

感谢回复 如果您方便的话 可否分享一下能适配7.4版本ES的elasticsearch-repository-oss? 我使用这位朋友分享的https://github.com/anjia0532/elasticsearch-repository-oss/releases/tag/v7.3.2并进行了一些修改 我将ES的版本修改到7.8以后发现由于7.8相对于7.3.2 BlobContainer这个接口修改了部分内容 我在重新实现了部分接口后尝试创建snapshot备份的时出现了如下的异常,并且我尝试对index进行备份 发现并没有创建snapshot成功 `[2021-08-25T00:48:23,191][WARN ][r.suppressed ] [node-064] path: /_snapshot/alioss-snapshot, params: {pretty=true, repository=alioss-snapshot} org.elasticsearch.common.blobstore.BlobStoreException: Failed to check if blob [master.dat] exists at org.elasticsearch.aliyun.oss.blobstore.OssBlobContainer.blobExists(OssBlobContainer.java:47) ~[?:?] at org.elasticsearch.aliyun.oss.blobstore.OssBlobContainer.writeBlob(OssBlobContainer.java:94) ~[?:?] at org.elasticsearch.aliyun.oss.blobstore.OssBlobContainer.writeBlobAtomic(OssBlobContainer.java:110) ~[?:?] at org.elasticsearch.repositories.blobstore.BlobStoreRepository.startVerification(BlobStoreRepository.java:1140) ~[elasticsearch-7.8.0.jar:7.8.0] at org.elasticsearch.repositories.RepositoriesService$3.doRun(RepositoriesService.java:248) ~[elasticsearch-7.8.0.jar:7.8.0] at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:695) [elasticsearch-7.8.0.jar:7.8.0] at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.8.0.jar:7.8.0] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?] at java.lang.Thread.run(Thread.java:832) [?:?] Caused by: com.aliyun.oss.OSSException: Failed to parse the response result.

[ResponseError]: <?xml version="1.0" encoding="UTF-8"?>

NoSuchKey The specified key does not exist. 612522DB886C823535034BE1 xxxxxxx-shanghai-finance-1-internal.aliyuncs.com alioss/tests-5UagXGj8TmaYw_aK0dGBtA/master.dat

at com.aliyun.oss.common.utils.ExceptionFactory.createOSSException(ExceptionFactory.java:110) ~[?:?] at com.aliyun.oss.common.utils.ExceptionFactory.createInvalidResponseException(ExceptionFactory.java:92) ~[?:?] at com.aliyun.oss.common.utils.ExceptionFactory.createInvalidResponseException(ExceptionFactory.java:81) ~[?:?] at com.aliyun.oss.internal.OSSErrorResponseHandler.handle(OSSErrorResponseHandler.java:72) ~[?:?] at com.aliyun.oss.common.comm.ServiceClient.handleResponse(ServiceClient.java:257) ~[?:?] at com.aliyun.oss.common.comm.ServiceClient.sendRequestImpl(ServiceClient.java:140) ~[?:?] at com.aliyun.oss.common.comm.ServiceClient.sendRequest(ServiceClient.java:70) ~[?:?] at com.aliyun.oss.internal.OSSOperation.send(OSSOperation.java:83) ~[?:?] at com.aliyun.oss.internal.OSSOperation.doOperation(OSSOperation.java:145) ~[?:?] at com.aliyun.oss.internal.OSSOperation.doOperation(OSSOperation.java:102) ~[?:?] at com.aliyun.oss.internal.OSSObjectOperation.getSimplifiedObjectMeta(OSSObjectOperation.java:450) ~[?:?] at com.aliyun.oss.internal.OSSObjectOperation.doesObjectExist(OSSObjectOperation.java:948) ~[?:?] at com.aliyun.oss.OSSClient.doesObjectExist(OSSClient.java:675) ~[?:?] at com.aliyun.oss.OSSClient.doesObjectExist(OSSClient.java:655) ~[?:?] at org.elasticsearch.aliyun.oss.service.OssStorageClient.doesObjectExist(OssStorageClient.java:85) ~[?:?] at org.elasticsearch.aliyun.oss.service.OssServiceImpl.doesObjectExist(OssServiceImpl.java:32) ~[?:?] at org.elasticsearch.aliyun.oss.blobstore.OssBlobStore.lambda$blobExists$3(OssBlobStore.java:151) ~[?:?] at java.security.AccessController.doPrivileged(AccessController.java:554) ~[?:?] at org.elasticsearch.utils.PermissionHelper.doPrivileged(PermissionHelper.java:26) ~[?:?] at org.elasticsearch.aliyun.oss.blobstore.OssBlobStore.doPrivilegedAndRefreshClient(OssBlobStore.java:206) ~[?:?] at org.elasticsearch.aliyun.oss.blobstore.OssBlobStore.blobExists(OssBlobStore.java:151) ~[?:?] at org.elasticsearch.aliyun.oss.blobstore.OssBlobContainer.blobExists(OssBlobContainer.java:44) ~[?:?] ... 9 more`