Closed JorisLemay closed 2 months ago
Thank you for taking the time to write a thorough bug report! It looks as though there are a few issues happening all at once.
Yes unfortunately GenAI is only available in the enterprise edition of Neo4j. It's available in aura and aura free, which is probably what the training is using for the neo4j backend. The error message could definitely be better though.
I know you didn't raise that as a problem but it looks bad. Plus anyone googling the error messages might find this issue.
I asked the APOC developers and they said all the Failed to load *
errors are because APOC doesn't ship with some optional dependencies to keep the size down. The relevant documentation (including how to fix) is here:
https://neo4j.com/docs/apoc/current/installation/#additional_dependencies
The neosemantics plugin can be quite slow in getting updates and releases. It looks as though the most recent one was 5.17.0, so the error here is correct because there is not a compatible 5.18 version. https://github.com/neo4j-labs/neosemantics
The graphql
and graph-algorithms
plugins seem to be quite out of date. This is a bug in the docker image. Either these aren't supported in 5.x or the image is checking the wrong place for compatibility information.
TLDR:
Thank you, Jennyowen, for the update. Learning that the GenAI plugin, essential for integrating AI services like OpenAI to generate vector embeddings, is exclusive to Neo4j's Enterprise Edition is profoundly disappointing for several reasons:
1. Functionality Misrepresented: The error messages imply a technical issue rather than a clear licensing restriction, misleading users, especially when educational materials suggest open-source access. This misalignment between the Aura Free service used in training and the inaccessibility in similar self-hosted setups creates barriers, particularly troubling for projects in sectors like healthcare.
2. Community Impact: Limiting GenAI to the Enterprise Edition curtails the Neo4j AI community's growth. New developers rely on such essential tools being accessible and affordable for their proof-of-concept projects. This restriction stifles innovation and could alienate a large part of the community looking for low-cost solutions.
3. Missed Opportunity: The paywall potentially pushes developers towards other technologies, overlooking Neo4j for AI-based projects. This could limit the adoption and innovation in the field, a short-sighted move given the potential for open access to drive widespread engagement and development in AI and graph technology.
Moreover, this situation underscores the need for transparency in educational resources like those offered by Deeplearning.AI. Accurate representation of tool availability and licensing would help set realistic expectations, guiding learners more effectively through their educational journey in AI and graph databases. This approach is vital for fostering an informed, prepared, and inclusive community.
As stated on https://neo4j.com/licensing , "Neo4j Loves Open Source. Neo4j is committed to open source." I, and I believe many others, hope this AI API call plugin can reflect this commitment.
Regards, Joris
Documentation Plugin: https://neo4j.com/docs/cypher-manual/current/genai-integrations/
fig. showing the used training with genai.vector.encode plugin and the database is {'name': 'Neo4j Kernel', 'versions': ['5.17.0'], 'edition': 'community'}
Source: https://learn.deeplearning.ai/courses/knowledge-graphs-rag/lesson/4/preparing-text-for-rag
I spent a lot of time in the course named Knowledge Graphs for RAG by Neo4J, and I meet the same issue. Now you tell me GenAI is only available in the enterprise edition of Neo4j, I have no choice but to give up on using and learning this tool.
We've addressed the issue of having unsupported plugins in the list. Also you will be able to get GenAI plugin on the neo4j community edition for versions after 5.23.0 :)
Issue
When running a local docker install for a training, the proposed function genai.vector.encode can not be executed.
Training: Knowledge Graphs for RAG by Neo4J by Andreas Kollegger https://learn.deeplearning.ai/courses/knowledge-graphs-rag/lesson/2/knowledge-graph-fundamentals
The plugin genai is missing in the plugin directory. The plugin apoc.jar is available.
Steps to reproduce.
1. create docker Graph DB in Ubuntu WSL
2. after a successful install
dbms.security.procedures.unrestricted=algo.,graphql.,semantics.,genai.,apoc.*
dbms.unmanaged_extension_classes=org.neo4j.graphql=/graphql
server.memory.pagecache.size=512M
server.default_listen_address=0.0.0.0 server.directories.plugins=/plugins server.directories.logs=/logs
root@LAPTOP20210403:/home/ehealtho/data/test/db/neo4j/logs# more debug.log | grep plug [classpath] /plugins/apoc.jar server.directories.plugins=/plugins server.directories.plugins=/plugins 2024-03-18 12:41:15.991+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to load
com.opencsv.bean.BeanFieldJoinStringIndex
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: org/apache/commons/collections4/MultiValuedMap 2024-03-18 12:41:15.997+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadcom.jayway.jsonpath.spi.mapper.JakartaMappingProvider
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: jakarta/json/stream/JsonParser 2024-03-18 12:41:16.045+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadcom.opencsv.bean.AbstractMappingStrategy
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: org/apache/commons/collections4/MultiValuedMap 2024-03-18 12:41:16.054+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadcom.opencsv.bean.concurrent.BeanExecutor
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: org/apache/commons/collections4/ListValuedMap 2024-03-18 12:41:16.077+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadcom.opencsv.bean.HeaderIndex
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: org/apache/commons/collections4/MultiValuedMap 2024-03-18 12:41:16.079+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadcom.opencsv.bean.FuzzyMappingStrategy
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: org/apache/commons/collections4/MultiValuedMap 2024-03-18 12:41:16.090+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadnet.minidev.asm.ASMUtil
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: org/objectweb/asm/MethodVisitor 2024-03-18 12:41:16.091+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadapoc.util.hdfs.HDFSUtils$1
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: org/apache/hadoop/fs/FSDataInputStream 2024-03-18 12:41:16.092+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadcom.opencsv.bean.BeanFieldJoin
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: org/apache/commons/collections4/MultiValuedMap 2024-03-18 12:41:16.094+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadMETA-INF.versions.9.module-info
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: META-INF/versions/9/module-info is not a class because access_flag ACC_MODULE is set 2024-03-18 12:41:16.112+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadcom.jayway.jsonpath.spi.json.JakartaJsonProvider$JsonObjectProxy$1
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: jakarta/json/JsonObject 2024-03-18 12:41:16.116+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadcom.opencsv.bean.customconverter.ConvertGermanToBoolean
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: org/apache/commons/beanutils/ConversionException 2024-03-18 12:41:16.125+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadcom.opencsv.bean.StatefulBeanToCsv
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: org/apache/commons/collections4/MultiValuedMap 2024-03-18 12:41:16.128+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadcom.opencsv.bean.HeaderNameBaseMappingStrategy
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: org/apache/commons/collections4/MultiValuedMap 2024-03-18 12:41:16.130+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadcom.jayway.jsonpath.spi.mapper.JakartaMappingProvider$JsonStructureScope
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: jakarta/json/JsonValue 2024-03-18 12:41:16.132+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadcom.opencsv.bean.concurrent.IntolerantThreadPoolExecutor
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: org/apache/commons/collections4/ListValuedMap 2024-03-18 12:41:16.154+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadcom.opencsv.bean.customconverter.ConverterLanguageToBoolean
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: org/apache/commons/beanutils/ConversionException 2024-03-18 12:41:16.192+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadcom.opencsv.bean.customconverter.ConvertFrenchToBoolean
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: org/apache/commons/beanutils/ConversionException 2024-03-18 12:41:16.193+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadcom.opencsv.bean.ColumnPositionMappingStrategy
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: org/apache/commons/collections4/MultiValuedMap 2024-03-18 12:41:16.199+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadcom.jayway.jsonpath.spi.mapper.JakartaMappingProvider$JsonObjectScope
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: jakarta/json/JsonValue 2024-03-18 12:41:16.205+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadapoc.util.google.cloud.GCStorageURLConnection
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: com/google/cloud/storage/Storage 2024-03-18 12:41:16.211+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadcom.jayway.jsonpath.spi.json.JakartaJsonProvider
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: jakarta/json/JsonObject 2024-03-18 12:41:16.219+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadcom.opencsv.bean.CsvToBeanBuilder
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: org/apache/commons/collections4/ListValuedMap 2024-03-18 12:41:16.233+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadcom.jayway.jsonpath.spi.json.JakartaJsonProvider$JsonArrayProxy
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: jakarta/json/JsonArray 2024-03-18 12:41:16.247+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadcom.opencsv.bean.FieldMapByPosition$1
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: org/apache/commons/collections4/iterators/LazyIteratorChain 2024-03-18 12:41:16.249+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadcom.opencsv.bean.HeaderColumnNameMappingStrategy
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: org/apache/commons/collections4/MultiValuedMap 2024-03-18 12:41:16.252+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadapoc.util.hdfs.HDFSUtils
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: org/apache/hadoop/fs/FSDataOutputStream 2024-03-18 12:41:16.265+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadcom.jayway.jsonpath.spi.json.JakartaJsonProvider$JsonArrayProxy$1
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: jakarta/json/JsonValue 2024-03-18 12:41:16.265+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadcom.jayway.jsonpath.spi.json.JakartaJsonProvider$JsonArrayProxy$2
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: jakarta/json/JsonValue 2024-03-18 12:41:16.277+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadcom.opencsv.bean.StatefulBeanToCsvBuilder
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: org/apache/commons/collections4/ListValuedMap 2024-03-18 12:41:16.281+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadcom.opencsv.bean.HeaderColumnNameTranslateMappingStrategy
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: org/apache/commons/collections4/MultiValuedMap 2024-03-18 12:41:16.293+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadapoc.util.s3.S3URLConnection
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: com/amazonaws/ClientConfiguration 2024-03-18 12:41:16.295+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadcom.opencsv.bean.BeanFieldJoinIntegerIndex
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: org/apache/commons/collections4/MultiValuedMap 2024-03-18 12:41:16.296+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadcom.jayway.jsonpath.spi.json.JsonOrgJsonProvider
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: org/json/JSONException 2024-03-18 12:41:16.298+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadcom.opencsv.bean.concurrent.LineExecutor
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: org/apache/commons/collections4/ListValuedMap 2024-03-18 12:41:16.313+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadnet.minidev.asm.BeansAccessBuilder
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: org/objectweb/asm/MethodVisitor 2024-03-18 12:41:16.321+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadcom.jayway.jsonpath.spi.json.JakartaJsonProvider$JsonObjectProxy
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: jakarta/json/JsonObject 2024-03-18 12:41:16.324+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadcom.opencsv.bean.MappingStrategy
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: org/apache/commons/collections4/MultiValuedMap 2024-03-18 12:41:16.360+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadapoc.util.s3.S3OutputStream$S3UploadManager
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: com/amazonaws/services/s3/AmazonS3 2024-03-18 12:41:16.369+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadcom.jayway.jsonpath.spi.mapper.JakartaMappingProvider$JsonArrayScope
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: jakarta/json/JsonValue 2024-03-18 12:41:16.371+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadcom.opencsv.bean.ConverterPrimitiveTypes
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: org/apache/commons/beanutils/ConversionException 2024-03-18 12:41:16.373+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadapoc.util.s3.S3Aws
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: com/amazonaws/auth/AWSCredentials 2024-03-18 12:41:16.375+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadcom.jayway.jsonpath.spi.mapper.JakartaMappingProvider$JsonStructureToParserAdapter
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: jakarta/json/stream/JsonParser 2024-03-18 12:41:16.383+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadcom.opencsv.bean.concurrent.AccumulateCsvResults
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: org/apache/commons/collections4/ListValuedMap 2024-03-18 12:41:16.385+0000 WARN [o.n.k.a.p.GlobalProcedures] Failed to loadapoc.util.s3.S3Aws$1
from plugin jar/plugins/apoc.jar
: java.lang.NoClassDefFoundError: com/amazonaws/services/s3/model/S3ObjectInputStream(venv) root@LAPTOP20210403:/home/ehealtho/data/test/db/neo4j/plugins# ls -la total 2828 drwxr-xr-x 2 7474 7474 4096 Mar 18 13:40 . drwxr-xr-x 7 root root 4096 Mar 18 13:40 .. -rwxr--r-- 1 7474 7474 2885593 Mar 7 14:56 apoc.jar
4. Actual behaviour
5. Neo4j image tag being used
neo4j:5.18-community
6. The output of the
docker version
command7. Operating system: (for example Windows 95/Ubuntu 16.04)
Ubuntu 22.04.3 LTS on Windows 11