apache / age

Graph database optimized for fast analysis and real-time data processing. It is provided as an extension to PostgreSQL.
https://age.apache.org
Apache License 2.0
3.03k stars 406 forks source link

entry_id error while loading csv #644

Closed DeoLeung closed 1 month ago

DeoLeung commented 1 year ago

Describe the bug

ERROR:  entry_id must be 1 .. 281474976710655
SELECT * FROM create_graph('neurond');
SELECT create_vlabel('neurond','entity');
SELECT * FROM load_labels_from_file('neurond',
                             'entity,
                             '/var/lib/postgresql/data/entity.csv');

loading a file with ~240k line and id ranging from 10k to 330k, after processing for 50s then it fails

How are you accessing AGE (Command line, driver, etc.)? raw sql

Environment (please complete the following information):

DeoLeung commented 1 year ago

similar for edges, from a file of around 100k

ERROR:  label_id must be 1 .. 65535
SELECT create_elabel('neurond','belongs');
SELECT * FROM load_edges_from_file('neurond',
                             'belongs',
                             '/var/lib/postgresql/data/relation.csv');
jrgemignani commented 1 year ago

This could be due to a patch for the PG11 version needing to be applied here.

fatimafatimaprogrammer commented 1 year ago

According on the error messages you supplied, there appears to be a problem with the label id and entry id values when attempting to import data from CSV files into the 'neurond' database.

The error messages suggest that the label id must be between 1 and 65535 and the entry id must be between 1 and 281474976710655. It's possible that the values in your CSV files are outside of the permitted ranges, creating the errors.

One possible option noted in the comments is to apply a PostgreSQL 11 patch. It's vital to note that keeping your database software up to date with the newest patches and upgrades is typically a good idea, since these frequently include critical bug fixes and security improvements.

Munmud commented 1 year ago

I found the same error. While debugging I found that this is happening because its getting some garbage value at the end of csv file.

eyab commented 1 year ago

@DeoLeung Could you please update us on this issue?

ticomse commented 10 months ago

My situation:

I used the CREATE command to create around 340,000 vertices. Apache Age automatically assigned ids with values like '20829148276967924'. (I believe that this is a 64-bit number.)

When I try to load vertices from a file, I get this error: psycopg2.errors.InvalidParameterValue: entry_id must be 1 .. 281474976710655

I'm pretty sure that 281474976710655 is MAXINT for a 48-bit integer.

It seems like psycopg2 has an input validation check that causes it to discard ids that are too large.

Any thoughts about how to fix or workaround this issue?

Thanks

LuckyLub commented 9 months ago

I ran into the same issue.

psycopg2.errors.InvalidParameterValue: label_id must be 1 .. 65535

Though I don't understand why there would be a range for the label ids, I have checked and the start_id and end_id do not exceed the indicated range of 1-65535.

The file resulting in the error: edges_afwijking_afhandeling.csv PG: 14 AGE: 1.4 python: 3.12 psycopg2==2.9.9

github-actions[bot] commented 4 months ago

This issue is stale because it has been open 45 days with no activity. Remove "Abondoned" label or comment or this will be closed in 7 days.

jrgemignani commented 4 months ago

This is referring to the label id which is imbedded in the id. But, unless you are creating more than 65535 distinct labels, it shouldn't complain.

jrgemignani commented 4 months ago

We have a new version of Apache AGE and Python driver. Is this still an issue?

github-actions[bot] commented 2 months ago

This issue is stale because it has been open 60 days with no activity. Remove "Abondoned" label or comment or this will be closed in 14 days.

github-actions[bot] commented 1 month ago

This issue was closed because it has been stalled for further 14 days with no activity.