search

dimitri / pgloader

Migrate to PostgreSQL in a single command!
http://pgloader.io
Other
5.45k stars 548 forks source link

Issue with pgloader migration from MySQL MariaDB to Supabase #1544

Closed mashwishi closed 11 months ago

mashwishi commented 1 year ago

I am encountering difficulties while attempting to migrate my MySQL MariaDB database to Supabase (PostgreSQL). My local database is set up using XAMPP, and Supabase is the target database hosted online. I've followed the migration instructions provided by Supabase in their documentation: Supabase MySQL Migration Guide.

I have encountered issues with the example provided in the documentation, specifically with lines 4 and 5:

load database
  from mysql://user:password@host/source_db
  into postgres://postgres:password@db.xxxx.supabase.co:6543/postgres
alter schema 'public' owner to 'postgres';
set wal_buffers = '64MB', max_wal_senders = 0, statement_timeout = 0, work_mem to '2GB';

In particular, I am using pgloader for the migration and have configured it as follows:

load database
  from mysql://user:password@host/source_db
  into postgres://postgres:password@db.xxxx.supabase.co:6543/postgres;

These lines seem to be causing an error. The error message I am receiving is:

.....0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0) :START 0 :COUNT 0) :LOCK #<SB-THREAD:MUTEX "Anonymous lock" (free)> :NOTIFY-PUSH NIL :NOTIFY-POP NIL) :DISABLE-TRIGGERS NIL :ON-ERROR-STOP T :COLUMNS NIL)
7: ((FLET "BODY-FN0" :IN LPARALLEL.KERNEL::MAKE-CHANNELED-TASK))
8: ((LAMBDA NIL :IN LPARALLEL.KERNEL::MAKE-CHANNELED-TASK))
9: (LPARALLEL.KERNEL::EXEC-TASK/WORKER #<unavailable argument> #S(LPARALLEL.KERNEL::WORKER :HANDSHAKE/FROM-WORKER #S(LPARALLEL.CONS-QUEUE:CONS-QUEUE :IMPL #S(LPARALLEL.RAW-QUEUE:RAW-QUEUE :HEAD NIL :TAIL NIL) :LOCK #<SB-THREAD:MUTEX #1="Anonymous lock" (free)> :CVAR NIL) :HANDSHAKE/TO-WORKER #S(LPARALLEL.CONS-QUEUE:CONS-QUEUE :IMPL #S(LPARALLEL.RAW-QUEUE:RAW-QUEUE :HEAD NIL :TAIL NIL) :LOCK #<SB-THREAD:MUTEX #1# (free)> :CVAR #<SB-THREAD:WAITQUEUE Anonymous condition variable {700EB74F73}>) :EXIT-NOTIFICATION #S(LPARALLEL.CONS-QUEUE:CONS-QUEUE :IMPL #S(LPARALLEL.RAW-QUEUE:RAW-QUEUE :HEAD NIL :TAIL NIL) :LOCK #<SB-THREAD:MUTEX #1# (free)> :CVAR NIL) :THREAD #<SB-THREAD:THREAD "lparallel" RUNNING {700EB00103}> :RUNNING-CATEGORY :DEFAULT :INDEX 1 :TASKS #S(LPARALLEL.SPIN-QUEUE:SPIN-QUEUE :HEAD #2=(LPARALLEL.SPIN-QUEUE::DUMMY) :TAIL #2#)))
10: (LPARALLEL.KERNEL::WORKER-LOOP #<LPARALLEL.KERNEL:KERNEL :NAME "lparallel" :WORKER-COUNT 4 :USE-CALLER NIL :ALIVE T :SPIN-COUNT 2000 {700EAE0023}> #S(LPARALLEL.KERNEL::WORKER :HANDSHAKE/FROM-WORKER #S(LPARALLEL.CONS-QUEUE:CONS-QUEUE :IMPL #S(LPARALLEL.RAW-QUEUE:RAW-QUEUE :HEAD NIL :TAIL NIL) :LOCK #<SB-THREAD:MUTEX #1="Anonymous lock" (free)> :CVAR NIL) :HANDSHAKE/TO-WORKER #S(LPARALLEL.CONS-QUEUE:CONS-QUEUE :IMPL #S(LPARALLEL.RAW-QUEUE:RAW-QUEUE :HEAD NIL :TAIL NIL) :LOCK #<SB-THREAD:MUTEX #1# (free)> :CVAR #<SB-THREAD:WAITQUEUE Anonymous condition variable {700EB74F73}>) :EXIT-NOTIFICATION #S(LPARALLEL.CONS-QUEUE:CONS-QUEUE :IMPL #S(LPARALLEL.RAW-QUEUE:RAW-QUEUE :HEAD NIL :TAIL NIL) :LOCK #<SB-THREAD:MUTEX #1# (free)> :CVAR NIL) :THREAD #<SB-THREAD:THREAD "lparallel" RUNNING {700EB00103}> :RUNNING-CATEGORY :DEFAULT :INDEX 1 :TASKS #S(LPARALLEL.SPIN-QUEUE:SPIN-QUEUE :HEAD #2=(LPARALLEL.SPIN-QUEUE::DUMMY) :TAIL #2#)))
11: (LPARALLEL.KERNEL::%CALL-WITH-TASK-HANDLER #<unavailable argument>)
12: ((LAMBDA NIL :IN LPARALLEL.KERNEL::CALL-WITH-WORKER-CONTEXT))
13: (LPARALLEL.KERNEL::CALL-WITH-WORKER-CONTEXT #<FUNCTION (LAMBDA NIL :IN LPARALLEL.KERNEL::ENTER-WORKER-LOOP) {700EAE032B}> #<FUNCTION FUNCALL> #<LPARALLEL.KERNEL:KERNEL :NAME "lparallel" :WORKER-COUNT 4 :USE-CALLER NIL :ALIVE T :SPIN-COUNT 2000 {700EAE0023}> #S(LPARALLEL.KERNEL::WORKER :HANDSHAKE/FROM-WORKER #S(LPARALLEL.CONS-QUEUE:CONS-QUEUE :IMPL #S(LPARALLEL.RAW-QUEUE:RAW-QUEUE :HEAD NIL :TAIL NIL) :LOCK #<SB-THREAD:MUTEX #1="Anonymous lock" (free)> :CVAR NIL) :HANDSHAKE/TO-WORKER #S(LPARALLEL.CONS-QUEUE:CONS-QUEUE :IMPL #S(LPARALLEL.RAW-QUEUE:RAW-QUEUE :HEAD NIL :TAIL NIL) :LOCK #<SB-THREAD:MUTEX #1# (free)> :CVAR #<SB-THREAD:WAITQUEUE Anonymous condition variable {700EB74F73}>) :EXIT-NOTIFICATION #S(LPARALLEL.CONS-QUEUE:CONS-QUEUE :IMPL #S(LPARALLEL.RAW-QUEUE:RAW-QUEUE :HEAD NIL :TAIL NIL) :LOCK #<SB-THREAD:MUTEX #1# (free)> :CVAR NIL) :THREAD #<SB-THREAD:THREAD "lparallel" RUNNING {700EB00103}> :RUNNING-CATEGORY :DEFAULT :INDEX 1 :TASKS #S(LPARALLEL.SPIN-QUEUE:SPIN-QUEUE :HEAD #2=(LPARALLEL.SPIN-QUEUE::DUMMY) :TAIL #2#)))
14: ((LAMBDA NIL :IN LPARALLEL.KERNEL::MAKE-WORKER-THREAD))
15: ((LAMBDA NIL :IN BORDEAUX-THREADS::BINDING-DEFAULT-SPECIALS))
16: ((FLET SB-UNIX::BODY :IN SB-THREAD::RUN))
17: ((FLET "WITHOUT-INTERRUPTS-BODY-151" :IN SB-THREAD::RUN))
18: ((FLET SB-UNIX::BODY :IN SB-THREAD::RUN))
19: ((FLET "WITHOUT-INTERRUPTS-BODY-144" :IN SB-THREAD::RUN))
20: (SB-THREAD::RUN)

What I am doing here?

Failed to connect to pgsql at "db.xxxx.supabase.co" (port 5432) as user "postgres": Database error: Socket error in "connect": EINTR (Interrupted system call)

Version

Additional Information:

Request for Assistance: I would appreciate any guidance or assistance in resolving this issue. If there are alternative configurations or steps that need to be taken, please provide the necessary information. Additionally, if there are ways to ensure a complete and accurate migration without encountering the socket error, I would be grateful for the guidance.

Thank you for your time and assistance in resolving this matter.

ccfontes commented 11 months ago

Exact same message here, but sqlite3 -> postgres.

This issue happens on the mac, while in linux goes beyond this point. Later crashes in Linux because I don't have enough RAM there, being the reason I switch to the mac.

Versions:

mashwishi commented 11 months ago

@ccfontes i found the issue, i got ip blocked. all you need to do is fix your supabase ip block list remove your ip