jaiminpan / pg_jieba

Postgresql full-text search extension for chinese
BSD 3-Clause "New" or "Revised" License
330 stars 64 forks source link

Feature Request: Support PG16 #62

Open cobolbaby opened 5 months ago

cobolbaby commented 5 months ago
$ docker run --rm -ti --name pgtest --user root --entrypoint=/bin/bash postgres:16.2
root@49f0e9960dd3:/home/postgres# apt-get update \
    && apt-get install -y postgresql-server-dev-${PG_MAJOR} make gcc
...
root@49f0e9960dd3:/home/postgres# apt-get install -y cmake g++ git
...
root@49f0e9960dd3:/home/postgres# cd /tmp/ && git clone --depth 1 https://github.com/jaiminpan/pg_jieba
Cloning into 'pg_jieba'...
remote: Enumerating objects: 15, done.
remote: Counting objects: 100% (15/15), done.
remote: Compressing objects: 100% (15/15), done.
remote: Total 15 (delta 0), reused 5 (delta 0), pack-reused 0
Receiving objects: 100% (15/15), 13.16 KiB | 66.00 KiB/s, done.
root@49f0e9960dd3:/tmp# cd pg_jieba && git submodule update --init --recursive
Submodule 'libjieba' (https://github.com/yanyiwu/cppjieba) registered for path 'libjieba'
Cloning into '/tmp/pg_jieba/libjieba'...
Submodule path 'libjieba': checked out '391121d5db0f31dd5ce9795d4d34812f20eeb25c'
Submodule 'deps/limonp' (https://github.com/yanyiwu/limonp.git) registered for path 'libjieba/deps/limonp'
Cloning into '/tmp/pg_jieba/libjieba/deps/limonp'...
Submodule path 'libjieba/deps/limonp': checked out 'a269e34dc4948d5a9209e21a7887b52daa0d3e78'
root@49f0e9960dd3:/tmp/pg_jieba# mkdir build && cd build && cmake ..
-- The C compiler identification is GNU 10.2.1
-- The CXX compiler identification is GNU 10.2.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Setting pg_jieba build type - 
CMake Error at /usr/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:165 (message):
  Could NOT find PostgreSQL (missing: PostgreSQL_TYPE_INCLUDE_DIR) (found
  version "16.2")
Call Stack (most recent call first):
  /usr/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:458 (_FPHSA_FAILURE_MESSAGE)
  /usr/share/cmake-3.18/Modules/FindPostgreSQL.cmake:248 (find_package_handle_standard_args)
  CMakeLists.txt:11 (find_package)

-- Configuring incomplete, errors occurred!
See also "/tmp/pg_jieba/build/CMakeFiles/CMakeOutput.log".
cobolbaby commented 5 months ago

Fixed by cmake -DPostgreSQL_TYPE_INCLUDE_DIR=/usr/include/postgresql/${PG_MAJOR}/server ..

cobolbaby commented 5 months ago

While the compilation and installation phases went smoothly, the pg_upgrade operation encounters an issue, stating that it cannot find the specified plugin.


dev@kubernetes-4 16:28:12 CST ~$ PGDATA_DEFAULT_TABLESPACE=/data/hdd/pg
dev@kubernetes-4 16:28:12 CST ~$ PGDATA_SECONDARY_TABLESPACE=/data/ssd/pg
dev@kubernetes-4 16:28:12 CST ~$ docker run --rm \
> -e POSTGRES_INITDB_ARGS="--data-checksums" \
> -v ${PGDATA_DEFAULT_TABLESPACE}:/var/lib/postgresql \
> -v ${PGDATA_SECONDARY_TABLESPACE}:/data/postgresql \
> registry.inventec/infra/pg_upgrade:12-to-16.2 \
> --check
Performing Consistency Checks
-----------------------------
Checking cluster versions                                     ok

Checking database user is the install user                    ok
Checking database connection settings                         ok
Checking for prepared transactions                            ok
Checking for system-defined composite types in user tables    ok
Checking for reg* data types in user tables                   ok
Checking for contrib/isn with bigint-passing mismatch         ok
Checking for incompatible "aclitem" data type in user tables  ok
Checking for user-defined encoding conversions                ok
Checking for user-defined postfix operators                   ok
Checking for incompatible polymorphic functions               ok
Checking for presence of required libraries                   ok
Checking database user is the install user                    ok
Checking for prepared transactions                            ok
Checking for new cluster tablespace directories               ok

*Clusters are compatible*
dev@kubernetes-4 16:51:34 CST ~$ docker run --rm -e POSTGRES_INITDB_ARGS="--data-checksums" -v ${PGDATA_DEFAULT_TABLESPACE}:/var/lib/postgresql -v ${PGDATA_SECONDARY_TABLESPACE}:/data/postgresql registry.inventec/infra/pg_upgrade:12-to-16.2 --link

Performing Consistency Checks
-----------------------------
Checking cluster versions                                     ok
Checking database user is the install user                    ok
Checking database connection settings                         ok
Checking for prepared transactions                            ok
Checking for system-defined composite types in user tables    ok
Checking for reg* data types in user tables                   ok
Checking for contrib/isn with bigint-passing mismatch         ok
Checking for incompatible "aclitem" data type in user tables  ok
Checking for user-defined encoding conversions                ok
Checking for user-defined postfix operators                   ok
Checking for incompatible polymorphic functions               ok
Creating dump of global objects                               ok
Creating dump of database schemas                             
*failure*

Consult the last few lines of "/var/lib/postgresql/16/data/pg_upgrade_output.d/20240218T085024.257/log/pg_upgrade_dump_16476.log" for
the probable cause of the failure.
Failure, exiting
dev@kubernetes-4 16:51:34 CST ~$ 
dev@kubernetes-4 16:51:34 CST ~$ sudo cat ${PGDATA_DEFAULT_TABLESPACE}/16/data/pg_upgrade_output.d/20240218T085024.257/log/pg_upgrade_dump_16476.log
[sudo] password for dev: 
command: "/usr/lib/postgresql/16/bin/pg_dump" --host /var/lib/postgresql --port 50432 --username postgres --schema-only --quote-all-identifiers --binary-upgrade --format=custom  --file="/var/lib/postgresql/16/data/pg_upgrade_output.d/20240218T085024.257/dump/pg_upgrade_dump_16476.custom" 'dbname=F3_BDC_test' >> "/var/lib/postgresql/16/data/pg_upgrade_output.d/20240218T085024.257/log/pg_upgrade_dump_16476.log" 2>&1
pg_dump: error: query failed: ERROR:  could not access file "$libdir/pg_jieba": No such file or directory
pg_dump: detail: Query was: SELECT
  ( SELECT alias FROM pg_catalog.ts_token_type('8749831'::pg_catalog.oid) AS t
    WHERE t.tokid = m.maptokentype ) AS tokenname,
  m.mapdict::pg_catalog.regdictionary AS dictname
FROM pg_catalog.pg_ts_config_map AS m
WHERE m.mapcfg = '8749835'
ORDER BY m.mapcfg, m.maptokentype, m.mapseqno

dev@kubernetes-2 19:07:46 CST ~$ docker run --rm -ti --name pgtest --entrypoint=/bin/bash registry.inventec/infra/pg_upgrade:12-to-16.2
postgres@df54efbb4da7:/var/lib/postgresql$ find / -type f -name "*jieba*"
find: ‘/var/cache/ldconfig’: Permission denied
find: ‘/var/cache/apt/archives/partial’: Permission denied
find: ‘/etc/ssl/private’: Permission denied
find: ‘/proc/tty/driver’: Permission denied
find: ‘/root’: Permission denied
/usr/share/postgresql/16/extension/pg_jieba.control
/usr/share/postgresql/16/extension/pg_jieba--1.1.1.sql
/usr/share/postgresql/16/tsearch_data/jieba.idf
/usr/share/postgresql/16/tsearch_data/jieba_hmm.model
/usr/share/postgresql/16/tsearch_data/jieba_base.dict
/usr/share/postgresql/16/tsearch_data/jieba_user.dict
/usr/share/postgresql/16/tsearch_data/jieba.stop
/usr/lib/postgresql/16/lib/pg_jieba.so
find: ‘/raft/datadir’: Permission denied
postgres@df54efbb4da7:/var/lib/postgresql$ 
cobolbaby commented 5 months ago

@jaiminpan Could you please help fix this bug?

jaiminpan commented 5 months ago

Thanks for the suggestion, it would be appreciated if you could submit a patch.