gchq / stroom

Stroom is a highly scalable data storage, processing and analysis platform.
https://gchq.github.io/stroom-docs/
Apache License 2.0
424 stars 57 forks source link

LMDB errors 7.3 #4133

Open stroomdev10 opened 4 months ago

stroomdev10 commented 4 months ago
INFO   [2024-02-28T08:45:24.687Z] [Data Processor# #43] stroom.pipeline.refdata.store.offheapstore.DelegatingRefDataOffHeapStore -  No legacy reference data store found in /xxxx/xxxxxxx/stroom/reference_data. No migration required.
DEBUG  [2024-02-28T08:45:24.697Z] [Data Processor# #43] stroom.pipeline.errorhandler.ErrorReceiverProxy -  Unable to provision, see the following errors:

1) [Guice/ErrorInjectingConstructor]: NoClassDefFoundError: Could not initialize class Library
  at DelegatingRefDataOffHeapStore.<init>(DelegatingRefDataOffHeapStore.java:111)
  at DelegatingRefDataOffHeapStore.class(DelegatingRefDataOffHeapStore.java:69)
  at RefDataStoreFactory.<init>(RefDataStoreFactory.java:35)
      \_ for 1st parameter
  at RefDataStoreFactory.class(RefDataStoreFactory.java:30)
  at RefDataStoreHolder.<init>(RefDataStoreHolder.java:54)
      \_ for 1st parameter
  at RefDataStoreHolder.class(RefDataStoreHolder.java:42)
  at ReferenceData.<init>(ReferenceData.java:94)
      \_ for 7th parameter
  at Lookup.<init>(Lookup.java:48)
      \_ for 1st parameter
  while locating Lookup

Learn more:
  https://github.com/google/guice/wiki/ERROR_INJECTING_CONSTRUCTOR

1 error

======================
Full classname legend:
======================
DelegatingRefDataOffHeapStore: "stroom.pipeline.refdata.store.offheapstore.DelegatingRefDataOffHeapStore"
Library:                       "org.lmdbjava.Library"
Lookup:                        "stroom.pipeline.xsltfunctions.Lookup"
RefDataStoreFactory:           "stroom.pipeline.refdata.store.RefDataStoreFactory"
RefDataStoreHolder:            "stroom.pipeline.refdata.RefDataStoreHolder"
ReferenceData:                 "stroom.pipeline.refdata.ReferenceData"
========================
End of classname legend:
========================
stroomdev10 commented 4 months ago
INFO   [2024-02-28T08:34:49.048Z] [Stroom Job - ScheduledExecutor] stroom.pipeline.refdata.store.offheapstore.DelegatingRefDataOffHeapStore -  No legacy reference data store found in /xxxxxxx/xxxxx/stroom/reference_data. No migration required.
ERROR  [2024-02-28T08:34:49.204Z] [Stroom Job - ScheduledExecutor] stroom.job.impl.ScheduledTaskExecutor -  Unable to provision, see the following errors:

1) [Guice/ErrorInjectingConstructor]: UnsatisfiedLinkError: could not load FFI provider Provider
  at DelegatingRefDataOffHeapStore.<init>(DelegatingRefDataOffHeapStore.java:111)
  at DelegatingRefDataOffHeapStore.class(DelegatingRefDataOffHeapStore.java:69)
  at RefDataStoreFactory.<init>(RefDataStoreFactory.java:35)
      \_ for 1st parameter
  at RefDataStoreFactory.class(RefDataStoreFactory.java:30)
  at ReferenceDataServiceImpl.<init>(ReferenceDataServiceImpl.java:144)
      \_ for 1st parameter
  while locating ReferenceDataServiceImpl
  while locating Searchable annotated with @Element(setName=,uniqueId=442, type=MULTIBINDER, keyType=)
  at SearchableProviderImpl.<init>(SearchableProviderImpl.java:22)
      \_ for 1st parameter
  at SearchableProviderImpl.class(SearchableProviderImpl.java:22)
  while locating SearchableProviderImpl
  at SearchableSearchProvider.<init>(SearchableSearchProvider.java:71)
      \_ for 5th parameter
  while locating SearchableSearchProvider
  while locating SearchProvider annotated with @Element(setName=,uniqueId=619, type=MULTIBINDER, keyType=)
  at StoreFactoryRegistry.<init>(StoreFactoryRegistry.java:19)
      \_ for 1st parameter
  at StoreFactoryRegistry.class(StoreFactoryRegistry.java:16)
  at ResultStoreManager.<init>(ResultStoreManager.java:76)
      \_ for 4th parameter
  at ResultStoreManager.class(ResultStoreManager.java:62)
  at SearchModule$EvictExpiredElements.<init>(SearchModule.java:89)
      \_ for 1st parameter
  while locating SearchModule$EvictExpiredElements
  while locating Runnable annotated with @Element(setName=,uniqueId=571, type=MAPBINDER, keyType=ScheduledJob)

Learn more:
  https://github.com/google/guice/wiki/ERROR_INJECTING_CONSTRUCTOR

1 error

======================
Full classname legend:
======================
DelegatingRefDataOffHeapStore:     "stroom.pipeline.refdata.store.offheapstore.DelegatingRefDataOffHeapStore"
Element:                           "com.google.inject.internal.Element"
Provider:                          "jnr.ffi.provider.jffi.Provider"
RefDataStoreFactory:               "stroom.pipeline.refdata.store.RefDataStoreFactory"
ReferenceDataServiceImpl:          "stroom.pipeline.refdata.ReferenceDataServiceImpl"
ResultStoreManager:                "stroom.query.common.v2.ResultStoreManager"
ScheduledJob:                      "stroom.job.api.ScheduledJob"
SearchModule$EvictExpiredElements: "stroom.search.impl.SearchModule$EvictExpiredElements"
SearchProvider:                    "stroom.query.common.v2.SearchProvider"
Searchable:                        "stroom.searchable.api.Searchable"
SearchableProviderImpl:            "stroom.searchable.impl.SearchableProviderImpl"
SearchableSearchProvider:          "stroom.searchable.impl.SearchableSearchProvider"
StoreFactoryRegistry:              "stroom.query.common.v2.StoreFactoryRegistry"
========================
End of classname legend:
========================
at055612 commented 4 months ago

relates to https://github.com/gchq/stroom/issues/3485

at055612 commented 4 months ago

This would seem to be an issue with running lmdb 0.8.3 on centos 7. 0.8.3 runs fine on other OSs, e.g. arch, alpine. Rolling back to 0.8.2 fixes the problem on centos 7.

Have raised this issue (and PR) against lmdb to uplift its jnr-ffi lib https://github.com/lmdbjava/lmdbjava/issues/234

We need to wait for that PR to be merged and the following issues to be fixed/released before we can uplift lmdb https://github.com/lmdbjava/lmdbjava/issues/230 https://github.com/lmdbjava/lmdbjava/issues/228

cdprete commented 4 months ago

This would seem to be an issue with running lmdb 0.8.3 on centos 7. 0.8.3 runs fine on other OSs, e.g. arch, alpine. Rolling back to 0.8.2 fixes the problem on centos 7.

Have raised this issue (and PR) against lmdb to uplift its jnr-ffi lib lmdbjava/lmdbjava#234

We need to wait for that PR to be merged and the following issues to be fixed/released before we can uplift lmdb lmdbjava/lmdbjava#230 lmdbjava/lmdbjava#228

I've opened https://github.com/lmdbjava/lmdbjava/pull/237 to fix https://github.com/lmdbjava/lmdbjava/issues/228 and https://github.com/lmdbjava/lmdbjava/issues/230 🤞