search

maxlapshin / mysql2postgres

Mysqldump, writing in postgresql format
MIT License
709 stars 156 forks source link

packet is not EOF #73

Open demofly opened 9 years ago

demofly commented 9 years ago

I've got an error:

  /usr/local/rvm/gems/ruby-1.9.3-p551/gems/mysqltopostgres-0.2.20/bin/mysqltopostgres mysql2pg.yml
  Mysql2psql: Conversion failed: packet is not EOF
  packet is not EOF
  /usr/local/rvm/gems/ruby-2.2.0/gems/mysql-pr-2.9.11/lib/mysql-pr/protocol.rb:562:in `read_eof_packet'
  /usr/local/rvm/gems/ruby-2.2.0/gems/mysql-pr-2.9.11/lib/mysql-pr/protocol.rb:274:in `retr_fields'
  /usr/local/rvm/gems/ruby-2.2.0/gems/mysql-pr-2.9.11/lib/mysql-pr.rb:322:in `query'

the very short dump is appeared:

  -- MySQL 2 PostgreSQL dump

  SET client_encoding = 'UTF8';
  SET standard_conforming_strings = off;
  SET check_function_bodies = false;
  SET client_min_messages = warning;
paazmaya commented 9 years ago

Could you share a minimal sql file contents and the settings used, that trigger this behaviour?

demofly commented 9 years ago 
-- MySQL dump 10.15  Distrib 10.0.12-MariaDB, for Linux (x86_64)
--
-- Host: localhost    Database: hadoop_stat7_cache_part1
-- ------------------------------------------------------
-- Server version       10.0.12-MariaDB-log
/*!40103 SET @OLD_TIME_ZONE=@@TIME_ZONE */;
/*!40103 SET TIME_ZONE='+00:00' */;
/*!40014 SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0 */;
/*!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0 */;
/*!40101 SET @OLD_SQL_MODE=@@SQL_MODE, SQL_MODE='NO_AUTO_VALUE_ON_ZERO,POSTGRESQL' */;
/*!40111 SET @OLD_SQL_NOTES=@@SQL_NOTES, SQL_NOTES=0 */;

--
-- Table structure for table "cache_data_426e3a6b5e74033fc220e0b36c81650a"
--

DROP TABLE IF EXISTS "cache_data_426e3a6b5e74033fc220e0b36c81650a";
/*!40101 SET @saved_cs_client     = @@character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE "cache_data_426e3a6b5e74033fc220e0b36c81650a" (
  "period_id" int(11) NOT NULL DEFAULT '0',
  "period_spec" varchar(32) DEFAULT NULL,
  "keys_hash" varchar(32) NOT NULL DEFAULT '',
  "date_ymd" int(11) DEFAULT NULL,
  "d01" varchar(500) NOT NULL,
  "n01" int(11) NOT NULL,
  "n02" int(11) NOT NULL,
  "m001" double NOT NULL,
  "m002" double NOT NULL,
  "m003" double NOT NULL,
  "m004" double NOT NULL,
  "m005" double NOT NULL,
  "m006" double NOT NULL,
  "m007" double NOT NULL,
  "m008" double NOT NULL,
  "m009" double NOT NULL,
  "m010" double NOT NULL,
  "m011" double NOT NULL,
  "m012" double NOT NULL,
  "m013" double NOT NULL,
  "m014" double NOT NULL,
  "m015" double NOT NULL,
  "m016" double NOT NULL,
  "m017" double NOT NULL,
  "m018" double NOT NULL,
  "m019" double NOT NULL,
  "m020" double NOT NULL,
  "m021" double NOT NULL,
  "m022" double NOT NULL,
  "m023" double NOT NULL,
  "m024" double NOT NULL,
  "m025" double NOT NULL,
  "m026" double NOT NULL,
  "m027" double NOT NULL,
  "m028" double NOT NULL,
  "m029" double NOT NULL,
  "m030" double NOT NULL,
  PRIMARY KEY ("period_id","keys_hash","n02","n01","d01"(30)) "CLUSTERING"=YES,
  KEY "i1" ("period_id","n02") "CLUSTERING"=YES
);
/*!40101 SET character_set_client = @saved_cs_client */;
demofly commented 9 years ago

It is the only table in the database dump made by mysqldump

paazmaya commented 9 years ago

Thank you. What kind of settings are you using?

keichan34 commented 9 years ago

I've been able to reproduce this error on Ruby 2.x. Using Ruby 1.9.3 seems to work without problems.

ideaoforder commented 8 years ago

+1

n-rodriguez commented 8 years ago

The fix https://github.com/ajokela/mysql-pr/pull/1

procommerz commented 8 years ago

n-rodriguez's patch worked for me (OSX 10.11), thanks

bhfailor commented 8 years ago

I am getting the same error:

vagrant@jessie:/vagrant/mysql2postgres$ mysqltopostgres config_options.yml
Mysql2psql: Conversion failed: packet is not EOF
packet is not EOF
/home/vagrant/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/mysql-pr-2.9.11/lib/mysql-pr/protocol.rb:562:in `read_eof_packet'
/home/vagrant/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/mysql-pr-2.9.11/lib/mysql-pr/protocol.rb:274:in `retr_fields'
/home/vagrant/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/mysql-pr-2.9.11/lib/mysql-pr.rb:322:in `query'
vagrant@jessie:/vagrant/mysql2postgres$ which ruby
/home/vagrant/.rbenv/shims/ruby
vagrant@jessie:/vagrant/mysql2postgres$ ruby -v
ruby 2.3.0p0 (2015-12-25 revision 53290) [x86_64-linux]

Listing of the config file:

# vagrant@jessie:/vagrant/mysql2postgres$ cat config_options.yml
default: &default
  adapter:  postgresql
  host:     localhost
  encoding: unicode
  pool:     5
  username: rails_user
  password: sekret
  template: template0

development: &development
  <<: *default
  database: stp_dev

test: &test
  <<: *default
  database: stp_test

mysql2psql:
  mysql:
    host: localhost
    socket: /var/run/mysqld/mysqld.sock
    username: root
    database: stp

  destination:
    test:
      <<: *test
    development:
      <<: *development

  tables:
  - certification_types

  dump_file_directory: /tmp
  remove_dump_file: false
bhfailor commented 8 years ago

Wondering how procommerz applied the patch from @n-rodriguez to this issue.

Suggested work arounds?

onpaws commented 8 years ago

The patch worked for me. Since it was a one liner I manually applied it. Open packet.rb from gem mysql-pr and insert it as first line. Thanks @n-rodriguez At first I was concerned with Postgres saying no COPY in progress but the data does appear to have transferred over.

n-rodriguez commented 8 years ago

@bhfailor :

cd $GEM_HOME
cd gems/mysql-pr-<version>/
vi lib/mysql-pr/packet.rb

Add #encoding: ascii-8bit at the top of the file.

bhfailor commented 8 years ago

Thanks, @n-rodriguez and @onpaws - I got it working! And I agree that the no COPY in progress message is misleading.

jvkiran commented 8 years ago

I got same issues, but i can't find out the solution.

valuenet@valuenet-lap:~/Sites/branch/mysql2postgres$ mysqltopostgres default.database.yml /home/valuenet/.rvm/gems/ruby-2.2.3@rails425/gems/pg-0.18.4/lib/pg/connection.rb:230: warning: already initialized constant PGconn /home/valuenet/.rvm/gems/ruby-2.2.3@rails425/gems/postgres-pr-0.7.0/lib/postgres-pr/postgres-compat.rb:6: warning: previous definition of PGconn was here /home/valuenet/.rvm/gems/ruby-2.2.3@rails425/gems/pg-0.18.4/lib/pg/result.rb:26: warning: already initialized constant PGresult /home/valuenet/.rvm/gems/ruby-2.2.3@rails425/gems/postgres-pr-0.7.0/lib/postgres-pr/postgres-compat.rb:69: warning: previous definition of PGresult was here Mysql2psql: Conversion failed: packet is not EOF packet is not EOF /home/valuenet/.rvm/gems/ruby-2.2.3@rails425/gems/mysql-pr-2.9.11/lib/mysql-pr/protocol.rb:562:in read_eof_packet' /home/valuenet/.rvm/gems/ruby-2.2.3@rails425/gems/mysql-pr-2.9.11/lib/mysql-pr/protocol.rb:274:inretr_fields' /home/valuenet/.rvm/gems/ruby-2.2.3@rails425/gems/mysql-pr-2.9.11/lib/mysql-pr.rb:322:in `query' valuenet@valuenet-lap:~/Sites/branch/mysql2postgres$

bhfailor commented 8 years ago

@jvkiran - I just made sure that the suggested line is inserted at the beginning of the suggested file:

$ bundle show mysql-pr
/home/vagrant/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/mysql-pr-2.9.11
$ cat /home/vagrant/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/mysql-pr-2.9.11/lib/mysql-pr/packet.rb
#encoding: ascii-8bit

class MysqlPR
  class Packet
    # convert Numeric to LengthCodedBinary
# [ remaining lines in file not shown ]

The inserted line is: #encoding: ascii-8bit

then things worked.