postgrespro / lsm3

LSM tree implementation based on standard B-Tree
Other
29 stars 1 forks source link

WARNING: you don't own a lock of type RowExclusiveLock #4

Closed 9072997 closed 3 years ago

9072997 commented 3 years ago

I have added several lsm3 indexes to a table and am now bulk-loading data in to it using COPY t FROM STDIN via psql. I am getting "WARNING: you don't own a lock of type RowExclusiveLock" 2-6 times per 100000 rows. The rows do get inserted, so this might be ignore-able?

I just did the same bulk-load using btree indexes without the lsm3 extension loaded and I do not get the warnings.

I am on PostgreSQL 13.3 on Debian. If there is any additional information I can provide, let me know.

knizhnik commented 3 years ago

I will be glad to receive from you any example reproducing this problem. I used the following script but there are no any warnings:

create table foo(pk bigint primary key,  val bigint);
insert into foo values (generate_series(1,1000000), 0);
copy foo to '/tmp/foo.csv';
create table foo3(pk bigint,  val bigint);
create index on foo3 using lsm3(pk);
copy foo3 from '/tmp/foo.csv';

May be you can modify this script or propose yours...

9072997 commented 3 years ago

Here is an example based on my schema with fake data and with non-indexd columns removed:

Download and uncompress this file, or you can generate example data using this program in golang:

fake-ping-generator.go ```golang package main import ( "encoding/csv" "fmt" "math/rand" "os" "time" "github.com/google/uuid" "syreclabs.com/go/faker" ) var fakeEmails [20000]string var fakeSerials [20000]string // returns value odds% of the time, else empty string func maybe(odds int, value string) string { n := rand.Int() % 100 if n < odds { return value } else { return "" } } func main() { for i := 0; i < 20000; i++ { fakeEmails[i] = faker.Internet().UserName() + "@example.com" fakeSerials[i] = faker.Lorem().Characters(7) } w := csv.NewWriter(os.Stdout) var i int64 for { w.Write([]string{ "9999", time.Now().Add(time.Second * time.Duration(i/100)).Format("2006-01-02T15:04:05.000Z"), faker.Internet().IpV4Address(), maybe(10, uuid.New().String()), time.Now().Add(time.Second * time.Duration(i/100+(rand.Int63()%60-30))).Format("2006-01-02T15:04:05.000Z"), time.Now().Add(time.Second * time.Duration(i/100-rand.Int63()%604800)).Format("2006-01-02T15:04:05.000Z"), fakeSerials[rand.Int()%20000], faker.Internet().IpV4Address(), maybe(50, faker.Internet().IpV6Address()), fmt.Sprint(faker.Address().Latitude()), fmt.Sprint(faker.Address().Longitude()), fmt.Sprint(rand.Int() % 1000), fmt.Sprint(rand.Int()%300, "Seconds"), fakeEmails[rand.Int()%20000], }) i++ } } ```

Set up the database like this:

simplified-schema.sql ```sql CREATE EXTENSION lsm3; CREATE COLLATION case_insensitive (provider = icu, locale = 'en-US-u-ks-level2'); CREATE TABLE pings ( endpoint integer NOT NULL, server_time timestamp with time zone NOT NULL DEFAULT CURRENT_TIMESTAMP, request_ip character varying(250) COLLATE case_insensitive NOT NULL, device_id uuid, client_time timestamp with time zone, session_start timestamp with time zone, serial_number character varying(250) COLLATE case_insensitive, local_ipv4 character varying(250) COLLATE case_insensitive, local_ipv6 character varying(250) COLLATE case_insensitive, latitude real, longitude real, accuracy real, location_age interval, email character varying(250) COLLATE case_insensitive ); CREATE INDEX pings_accuracy_idx ON pings USING lsm3 (endpoint ASC NULLS LAST, accuracy ASC NULLS LAST); CREATE INDEX pings_client_time_idx ON pings USING brin (client_time); CREATE INDEX pings_device_id_idx ON pings USING lsm3 (endpoint ASC NULLS LAST, device_id ASC NULLS LAST, server_time DESC NULLS FIRST); CREATE INDEX pings_email_idx ON pings USING lsm3 (endpoint ASC NULLS LAST, email COLLATE case_insensitive ASC NULLS LAST, server_time DESC NULLS FIRST); CREATE INDEX pings_latitude_idx ON pings USING lsm3 (endpoint ASC NULLS LAST, latitude ASC NULLS LAST); CREATE INDEX pings_local_ipv4_idx ON pings USING lsm3 (endpoint ASC NULLS LAST, local_ipv4 ASC NULLS LAST, server_time DESC NULLS FIRST); CREATE INDEX pings_local_ipv6_idx ON pings USING lsm3 (endpoint ASC NULLS LAST, local_ipv6 ASC NULLS LAST, server_time DESC NULLS FIRST); CREATE INDEX pings_location_age_idx ON pings USING lsm3 (endpoint ASC NULLS LAST, location_age ASC NULLS LAST); CREATE INDEX pings_longitude_idx ON pings USING lsm3 (endpoint ASC NULLS LAST, longitude ASC NULLS LAST); CREATE INDEX pings_request_ip_idx ON pings USING lsm3 (endpoint ASC NULLS LAST, request_ip ASC NULLS LAST, server_time DESC NULLS FIRST); CREATE INDEX pings_serial_number_idx ON pings USING lsm3 (endpoint ASC NULLS LAST, serial_number COLLATE case_insensitive ASC NULLS LAST, server_time DESC NULLS FIRST); CREATE INDEX pings_server_time_idx ON pings USING brin (server_time); CREATE INDEX pings_session_start_idx ON pings USING lsm3 (endpoint ASC NULLS LAST, session_start DESC NULLS FIRST, server_time DESC NULLS FIRST); ```

Do either something like COPY pings FROM '/tmp/test-data.csv' DELIMITER ',' CSV; in psql or pgadmin 4. You should see something like "WARNING: you don't own a lock of type RowExclusiveLock" on stderr or in the messages tab of pgadmin.

knizhnik commented 3 years ago

Please try new version

9072997 commented 3 years ago

This resolved the issue. Thank you.