search

dimitri / pgloader

Migrate to PostgreSQL in a single command!
http://pgloader.io
Other
5.45k stars 548 forks source link

Trying to migrate Sybase ASE database to PostgresQl #1478

Open gvkkarthick opened 1 year ago

gvkkarthick commented 1 year ago

1) Took a dump of Sybase database using "dump to path" 2) Scp to the postgres server 3) Trying to convert the dump compatible to postgres using " pgloader" pgloader --verbose --debug /u01/pgsql/data/sybase_dump/test.dmp /u01/pgsql/data/postgres_dump/

am getting bellow error...

pgloader version 3.6.2 compiled with SBCL 2.0.1-4.el8 sb-impl::default-external-format :UTF-8 tmpdir: #P"/tmp/pgloader/" KABOOM! FATAL error: :UTF-8 stream decoding error on #<SB-SYS:FD-STREAM for "file /u01/pgsql/data/sybase_dump/migratetable.dmp" {1004FAB913}>: the octet sequence #(149) cannot be decoded. Date/time: 2023-04-12-11:57! An unhandled error condition has been signalled: :UTF-8 stream decoding error on #<SB-SYS:FD-STREAM for "file /u01/pgsql/data/sybase_dump/migratetable.dmp" {1004FAB913}>: the octet sequence #(149) cannot be decoded.

Backtrace for: #<SB-THREAD:THREAD "main thread" RUNNING {1000568083}> 0: (TRIVIAL-BACKTRACE:PRINT-BACKTRACE-TO-STREAM #<SB-IMPL::CHARACTER-STRING-OSTREAM {1004FD01A3}>) 1: (TRIVIAL-BACKTRACE:PRINT-BACKTRACE #<SB-INT:STREAM-DECODING-ERROR {1004FD0003}> :OUTPUT NIL :IF-EXISTS :APPEND :VERBOSE NIL) 2: ((FLET "H0" :IN PGLOADER::MAIN) #<SB-INT:STREAM-DECODING-ERROR {1004FD0003}>) 3: (SB-KERNEL::%SIGNAL #<SB-INT:STREAM-DECODING-ERROR {1004FD0003}>) 4: (ERROR #<SB-INT:STREAM-DECODING-ERROR {1004FD0003}>) 5: (SB-KERNEL:WITH-SIMPLE-CONDITION-RESTARTS ERROR NIL SB-INT:STREAM-DECODING-ERROR :EXTERNAL-FORMAT :UTF-8 :STREAM #<SB-SYS:FD-STREAM for "file /u01/pgsql/data/sybase_dump/migratetable.dmp" {1004FAB913}> :OCTETS #(149)) 6: (SB-IMPL::STREAM-DECODING-ERROR-AND-HANDLE #<SB-SYS:FD-STREAM for "file /u01/pgsql/data/sybase_dump/migratetable.dmp" {1004FAB913}> 1) 7: (SB-IMPL::FD-STREAM-READ-N-CHARACTERS/UTF-8 #<SB-SYS:FD-STREAM for "file /u01/pgsql/data/sybase_dump/migratetable.dmp" {1004FAB913}> #<(SIMPLE-ARRAY CHARACTER (512)) EOF1banking23101126D5000100 23101 23101 ... {1004FABAEF}> 4 508 NIL) 8: (SB-INT:FAST-READ-CHAR-REFILL #<SB-SYS:FD-STREAM for "file /u01/pgsql/data/sybase_dump/migratetable.dmp" {1004FAB913}> NIL) 9: (SB-IMPL::ANSI-STREAM-READ-STRING-FROM-FRC-BUFFER #<(SIMPLE-ARRAY CHARACTER (4096)) HDR2F 0204803 banking 00000000008000006402 0000008... {1004FC000F}> #<SB-SYS:FD-STREAM for "file /u01/pgsql/data/sybase_dump/migratetable.dmp" {1004FAB913}> 0 NIL) 10: (SB-IMPL:ANSI-STREAM-READ-SEQUENCE #<(SIMPLE-ARRAY CHARACTER (4096)) HDR2F 0204803 banking 00000000008000006402 0000008... {1004FC000F}> #<SB-SYS:FD-STREAM for "file /u01/pgsql/data/sybase_dump/migratetable.dmp" {1004FAB913}> 0 NIL) 11: (READ-SEQUENCE #<(SIMPLE-ARRAY CHARACTER (4096)) HDR2F 0204803 banking 00000000008000006402 0000008... {1004FC000F}> #<SB-SYS:FD-STREAM for "file /u01/pgsql/data/sybase_dump/migratetable.dmp" {1004FAB913}> :START 0 :END NIL) 12: (ALEXANDRIA.1.0.0:READ-STREAM-CONTENT-INTO-STRING #<SB-SYS:FD-STREAM for "file /u01/pgsql/data/sybase_dump/migratetable.dmp" {1004FAB913}> :BUFFER-SIZE 4096) 13: (ALEXANDRIA.1.0.0:READ-FILE-INTO-STRING #P"/u01/pgsql/data/sybase_dump/migratetable.dmp" :BUFFER-SIZE 4096 :EXTERNAL-FORMAT NIL) 14: (PGLOADER.PARSER:PARSE-COMMANDS-FROM-FILE #P"/u01/pgsql/data/sybase_dump/migratetable.dmp") 15: (PGLOADER:RUN-COMMANDS #P"/u01/pgsql/data/sybase_dump/migratetable.dmp" :START-LOGGER NIL :FLUSH-SUMMARY T :SUMMARY NIL :LOG-FILENAME NIL :LOG-MIN-MESSAGES NIL :CLIENT-MIN-MESSAGES NIL) 16: (PGLOADER::PROCESS-COMMAND-FILE ("/u01/pgsql/data/sybase_dump/migratetable.dmp" "/u01/pgsql/data/postgres_dump/") :FLUSH-SUMMARY T) 17: (PGLOADER::PROCESS-SOURCE-AND-TARGET "/u01/pgsql/data/sybase_dump/migratetable.dmp" "/u01/pgsql/data/postgres_dump/" NIL NIL NIL NIL NIL NIL NIL NIL) 18: (PGLOADER.MONITOR::CALL-WITH-MONITOR #<CLOSURE (LAMBDA NIL :IN PGLOADER::MAIN) {1004D4C02B}>) 19: (PGLOADER::MAIN ("pgloader" "--verbose" "--debug" "/u01/pgsql/data/sybase_dump/migratetable.dmp" "/u01/pgsql/data/postgres_dump/")) 20: ((LAMBDA NIL :IN "/builddir/build/BUILD/pgloader-bundle-3.6.2/dumper-2SKVI5f7.lisp")) 21: ((FLET SB-UNIX::BODY :IN SAVE-LISP-AND-DIE)) 22: ((FLET "WITHOUT-INTERRUPTS-BODY-14" :IN SAVE-LISP-AND-DIE)) 23: ((LABELS SB-IMPL::RESTART-LISP :IN SAVE-LISP-AND-DIE))

2023-04-12T15:57:10.007000+06:00 NOTICE Starting pgloader, log system is ready. 2023-04-12T15:57:10.020000+06:00 INFO Starting monitor 2023-04-12T15:57:10.022000+06:00 LOG pgloader version "3.6.2" 2023-04-12T15:57:10.022000+06:00 INFO Stopping monitor

What I am doing here?

:UTF-8 stream decoding error on

<SB-SYS:FD-STREAM for "file /u01/pgsql/data/sybase_dump/migratetable.dmp" {1004FAB913}>:

the octet sequence #(149) cannot be decoded.

gvkkarthick commented 1 year ago

read the pgloader command as pgloader --verbose --debug /u01/pgsql/data/sybase_dump/migratetable.dmp /u01/pgsql/data/postgres_dump/

blzzua commented 1 year ago

Hello. the dump-file made by "dump to path" can be loaded only into Sybase ASE pre-created database using "load database .. from path" command. This file have snapshot of database binary pages of sybase database, and can not be accessed without sybase ase dataserver.

Migration sybase ase dump file to Sybase IQ is not possible with this tool. Firstly because it use postgresql as destination, secondly limited formats as source.

I recommend use tools which generate DDL-script (create table). (Sybase power designer as instance) . convert it into Sybase IQ types (Sybase power designer can convert "physical model" from one RDBMS type to other and generate DDL scripts). migrate data using BCP tool for export and LOAD sybase iq command for load. Or directly from Sybase ASE.