Part-DB / Part-DB-server

Part-DB is an Open source inventory management system for your electronic components
https://docs.part-db.de/
GNU Affero General Public License v3.0
891 stars 96 forks source link

PartKeepr to Part-DB migration in docker #608

Closed pawlef closed 5 months ago

pawlef commented 5 months ago

Hi all, I would like to move from the discontinued Partkeeper to the promising Part-DB, but errors appear during import. According to the instructions, I exported the database from Partkeepr to an xml file. I placed the file in the directory and executed the command docker exec --user=www-data -it partdb bin/console partdb:migrations:import-partkeepr /var/www/html/var/db/pk.xml

I tried the latest and edge branch, but unfortunately without success. I am attaching my pk.xml file and a screenshot.

docker exec --user=www-data -it partdb bin/console partdb:migrations:import-partkeepr /var/www/html/var/db/pk.xml

! [NOTE] This command is still in development. If you encounter any problems, please report them to the issue tracker
! on GitHub.

[WARNING] This command will delete all existing data in the database (except users). Make sure that you have no
important data in the database before you continue!

Please type "DELETE ALL DATA" to continue. []:

DELETE ALL DATA

[INFO] Importing distributors...

[OK] Imported 5 distributors.

[INFO] Importing part measurement units...

[OK] Imported 1 measurement units.

[INFO] Importing manufacturers...

[OK] Imported 288 manufacturers.

[INFO] Importing categories...

[OK] Imported 272 categories.

[INFO] Importing Footprints...

[OK] Imported 44 footprints.

[INFO] Importing storage locations...

[OK] Imported 32 storage locations.

pk.tgz.txt

{"message":"User Deprecated: Since symfony/doctrine-bridge 6.3: Registering \"App\EventSubscriber\LogSystem\EventLoggerSubscriber\" as a Doctrine subscriber is deprecated. Register it as a listener instead, using e.g. the #[AsDoctrineListener] or #[AsDocumentListener] attribute.","context":{"exception":{"class":"ErrorException","message":"User Deprecated: Since symfony/doctrine-bridge 6.3: Registering \"App\EventSubscriber\LogSystem\EventLoggerSubscriber\" as a Doctrine subscriber is deprecated. Register it as a listener instead, using e.g. the #[AsDoctrineListener] or #[AsDocumentListener] attribute.","code":0,"file":"/var/www/html/vendor/symfony/doctrine-bridge/ContainerAwareEventManager.php:211","trace":["/var/www/html/vendor/symfony/doctrine-bridge/ContainerAwareEventManager.php:102","/var/www/html/vendor/doctrine/dbal/src/Connection.php:387","/var/www/html/vendor/doctrine/dbal/src/Connection.php:453","/var/www/html/vendor/doctrine/dbal/src/Connection.php:411","/var/www/html/vendor/doctrine/dbal/src/Connection.php:318","/var/www/html/src/Doctrine/Purger/ResetAutoIncrementORMPurger.php:126","/var/www/html/src/Services/ImportExportSystem/PartKeeprImporter/PKImportHelper.php:46","/var/www/html/src/Command/Migrations/ImportPartKeeprCommand.php:82","/var/www/html/vendor/symfony/console/Command/Command.php:326","/var/www/html/vendor/symfony/console/Application.php:1096","/var/www/html/vendor/symfony/framework-bundle/Console/Application.php:126","/var/www/html/vendor/symfony/console/Application.php:324","/var/www/html/vendor/symfony/framework-bundle/Console/Application.php:80","/var/www/html/vendor/symfony/console/Application.php:175","/var/www/html/vendor/symfony/runtime/Runner/Symfony/ConsoleApplicationRunner.php:49","/var/www/html/vendor/autoload_runtime.php:29","/var/www/html/bin/console:18"]}},"level":200,"level_name":"INFO","channel":"deprecation","datetime":"2024-05-02T14:54:36.877203+00:00","extra":{"token":null,"command":{"name":"partdb:migrations:import-partkeepr","arguments":{"command":"partdb:migrations:import-partkeepr","file":"/var/www/html/var/db/pk.xml"}}}} {"message":"User Deprecated: Since symfony/doctrine-bridge 6.3: Registering \"App\EventSubscriber\LogSystem\LogDBMigrationSubscriber\" as a Doctrine subscriber is deprecated. Register it as a listener instead, using e.g. the #[AsDoctrineListener] or #[AsDocumentListener] attribute.","context":{"exception":{"class":"ErrorException","message":"User Deprecated: Since symfony/doctrine-bridge 6.3: Registering \"App\EventSubscriber\LogSystem\LogDBMigrationSubscriber\" as a Doctrine subscriber is deprecated. Register it as a listener instead, using e.g. the #[AsDoctrineListener] or #[AsDocumentListener] attribute.","code":0,"file":"/var/www/html/vendor/symfony/doctrine-bridge/ContainerAwareEventManager.php:211","trace":["/var/www/html/vendor/symfony/doctrine-bridge/ContainerAwareEventManager.php:102","/var/www/html/vendor/doctrine/dbal/src/Connection.php:387","/var/www/html/vendor/doctrine/dbal/src/Connection.php:453","/var/www/html/vendor/doctrine/dbal/src/Connection.php:411","/var/www/html/vendor/doctrine/dbal/src/Connection.php:318","/var/www/html/src/Doctrine/Purger/ResetAutoIncrementORMPurger.php:126","/var/www/html/src/Services/ImportExportSystem/PartKeeprImporter/PKImportHelper.php:46","/var/www/html/src/Command/Migrations/ImportPartKeeprCommand.php:82","/var/www/html/vendor/symfony/console/Command/Command.php:326","/var/www/html/vendor/symfony/console/Application.php:1096","/var/www/html/vendor/symfony/framework-bundle/Console/Application.php:126","/var/www/html/vendor/symfony/console/Application.php:324","/var/www/html/vendor/symfony/framework-bundle/Console/Application.php:80","/var/www/html/vendor/symfony/console/Application.php:175","/var/www/html/vendor/symfony/runtime/Runner/Symfony/ConsoleApplicationRunner.php:49","/var/www/html/vendor/autoload_runtime.php:29","/var/www/html/bin/console:18"]}},"level":200,"level_name":"INFO","channel":"deprecation","datetime":"2024-05-02T14:54:36.878986+00:00","extra":{"token":null,"command":{"name":"partdb:migrations:import-partkeepr","arguments":{"command":"partdb:migrations:import-partkeepr","file":"/var/www/html/var/db/pk.xml"}}}} {"message":"Error thrown while running command \"partdb:migrations:import-partkeepr '/var/www/html/var/db/pk.xml'\". Message: \"An exception occurred while executing a query: SQLSTATE[23000]: Integrity constraint violation: 19 UNIQUE constraint failed: parts.ipn\"","context":{"exception":{"class":"Doctrine\DBAL\Exception\UniqueConstraintViolationException","message":"An exception occurred while executing a query: SQLSTATE[23000]: Integrity constraint violation: 19 UNIQUE constraint failed: parts.ipn","code":19,"file":"/var/www/html/vendor/doctrine/dbal/src/Driver/API/SQLite/ExceptionConverter.php:41","trace":["/var/www/html/vendor/doctrine/dbal/src/Connection.php:1939","/var/www/html/vendor/doctrine/dbal/src/Connection.php:1881","/var/www/html/vendor/doctrine/dbal/src/Statement.php:194","/var/www/html/vendor/doctrine/dbal/src/Statement.php:249","/var/www/html/vendor/doctrine/orm/src/Persisters/Entity/BasicEntityPersister.php:285","/var/www/html/vendor/doctrine/orm/src/UnitOfWork.php:1183","/var/www/html/vendor/doctrine/orm/src/UnitOfWork.php:447","/var/www/html/vendor/doctrine/orm/src/EntityManager.php:403","/var/www/html/src/Services/ImportExportSystem/PartKeeprImporter/PKPartImporter.php:121","/var/www/html/src/Command/Migrations/ImportPartKeeprCommand.php:142","/var/www/html/src/Command/Migrations/ImportPartKeeprCommand.php:95","/var/www/html/vendor/symfony/console/Command/Command.php:326","/var/www/html/vendor/symfony/console/Application.php:1096","/var/www/html/vendor/symfony/framework-bundle/Console/Application.php:126","/var/www/html/vendor/symfony/console/Application.php:324","/var/www/html/vendor/symfony/framework-bundle/Console/Application.php:80","/var/www/html/vendor/symfony/console/Application.php:175","/var/www/html/vendor/symfony/runtime/Runner/Symfony/ConsoleApplicationRunner.php:49","/var/www/html/vendor/autoload_runtime.php:29","/var/www/html/bin/console:18"],"previous":{"class":"Doctrine\DBAL\Driver\PDO\Exception","message":"SQLSTATE[23000]: Integrity constraint violation: 19 UNIQUE constraint failed: parts.ipn","code":19,"file":"/var/www/html/vendor/doctrine/dbal/src/Driver/PDO/Exception.php:28","trace":["/var/www/html/vendor/doctrine/dbal/src/Driver/PDO/Statement.php:132","/var/www/html/vendor/doctrine/dbal/src/Statement.php:190","/var/www/html/vendor/doctrine/dbal/src/Statement.php:249","/var/www/html/vendor/doctrine/orm/src/Persisters/Entity/BasicEntityPersister.php:285","/var/www/html/vendor/doctrine/orm/src/UnitOfWork.php:1183","/var/www/html/vendor/doctrine/orm/src/UnitOfWork.php:447","/var/www/html/vendor/doctrine/orm/src/EntityManager.php:403","/var/www/html/src/Services/ImportExportSystem/PartKeeprImporter/PKPartImporter.php:121","/var/www/html/src/Command/Migrations/ImportPartKeeprCommand.php:142","/var/www/html/src/Command/Migrations/ImportPartKeeprCommand.php:95","/var/www/html/vendor/symfony/console/Command/Command.php:326","/var/www/html/vendor/symfony/console/Application.php:1096","/var/www/html/vendor/symfony/framework-bundle/Console/Application.php:126","/var/www/html/vendor/symfony/console/Application.php:324","/var/www/html/vendor/symfony/framework-bundle/Console/Application.php:80","/var/www/html/vendor/symfony/console/Application.php:175","/var/www/html/vendor/symfony/runtime/Runner/Symfony/ConsoleApplicationRunner.php:49","/var/www/html/vendor/autoload_runtime.php:29","/var/www/html/bin/console:18"],"previous":{"class":"PDOException","message":"SQLSTATE[23000]: Integrity constraint violation: 19 UNIQUE constraint failed: parts.ipn","code":23000,"file":"/var/www/html/vendor/doctrine/dbal/src/Driver/PDO/Statement.php:130","trace":["/var/www/html/vendor/doctrine/dbal/src/Driver/PDO/Statement.php:130","/var/www/html/vendor/doctrine/dbal/src/Statement.php:190","/var/www/html/vendor/doctrine/dbal/src/Statement.php:249","/var/www/html/vendor/doctrine/orm/src/Persisters/Entity/BasicEntityPersister.php:285","/var/www/html/vendor/doctrine/orm/src/UnitOfWork.php:1183","/var/www/html/vendor/doctrine/orm/src/UnitOfWork.php:447","/var/www/html/vendor/doctrine/orm/src/EntityManager.php:403","/var/www/html/src/Services/ImportExportSystem/PartKeeprImporter/PKPartImporter.php:121","/var/www/html/src/Command/Migrations/ImportPartKeeprCommand.php:142","/var/www/html/src/Command/Migrations/ImportPartKeeprCommand.php:95","/var/www/html/vendor/symfony/console/Command/Command.php:326","/var/www/html/vendor/symfony/console/Application.php:1096","/var/www/html/vendor/symfony/framework-bundle/Console/Application.php:126","/var/www/html/vendor/symfony/console/Application.php:324","/var/www/html/vendor/symfony/framework-bundle/Console/Application.php:80","/var/www/html/vendor/symfony/console/Application.php:175","/var/www/html/vendor/symfony/runtime/Runner/Symfony/ConsoleApplicationRunner.php:49","/var/www/html/vendor/autoload_runtime.php:29","/var/www/html/bin/console:18"]}}},"command":"partdb:migrations:import-partkeepr '/var/www/html/var/db/pk.xml'","message":"An exception occurred while executing a query: SQLSTATE[23000]: Integrity constraint violation: 19 UNIQUE constraint failed: parts.ipn"},"level":500,"level_name":"CRITICAL","channel":"console","datetime":"2024-05-02T14:54:40.614389+00:00","extra":{"token":null,"command":{"name":"partdb:migrations:import-partkeepr","arguments":{"command":"partdb:migrations:import-partkeepr","file":"/var/www/html/var/db/pk.xml"}}}} 14:54:40 CRITICAL [console] Error thrown while running command "partdb:migrations:import-partkeepr '/var/www/html/var/db/pk.xml'". Message: "An exception occurred while executing a query: SQLSTATE[23000]: Integrity constraint violation: 19 UNIQUE constraint failed: parts.ipn" ["exception" => Doctrine\DBAL\Exception\UniqueConstraintViolationException^ { …},"command" => "partdb:migrations:import-partkeepr '/var/www/html/var/db/pk.xml'","message" => "An exception occurred while executing a query: SQLSTATE[23000]: Integrity constraint violation: 19 UNIQUE constraint failed: parts.ipn"] ["token" => null,"command" => ["name" => "partdb:migrations:import-partkeepr","arguments" => ["command" => "partdb:migrations:import-partkeepr","file" => "/var/www/html/var/db/pk.xml"]]] {"message":"Command \"partdb:migrations:import-partkeepr '/var/www/html/var/db/pk.xml'\" exited with code \"19\"","context":{"command":"partdb:migrations:import-partkeepr '/var/www/html/var/db/pk.xml'","code":19},"level":100,"level_name":"DEBUG","channel":"console","datetime":"2024-05-02T14:54:40.623217+00:00","extra":{"token":null,"command":{"name":"partdb:migrations:import-partkeepr","arguments":{"command":"partdb:migrations:import-partkeepr","file":"/var/www/html/var/db/pk.xml"}}}}

In ExceptionConverter.php line 41:

An exception occurred while executing a query: SQLSTATE[23000]: Integrity constraint violation: 19 UNIQUE constraint failed: parts.ipn

In Exception.php line 28:

SQLSTATE[23000]: Integrity constraint violation: 19 UNIQUE constraint failed: parts.ipn

In Statement.php line 130:

SQLSTATE[23000]: Integrity constraint violation: 19 UNIQUE constraint failed: parts.ipn

jbtronics commented 5 months ago

You probably have two different parts in your partkeepr database which have the same internal part number (IPN). For partkeepr that does not matter, but for Part-DB that is intended as a custom internal identifier for this specific part and it must be unique across the database.

So you should fix that in your data before proceeding and change (or remove) the IPN field of one of the parts before continuing with the import to Part-DB.

Either you do the changes in PartKeepr or you do it in the XML file directly.

pawlef commented 5 months ago

Thank you for your very quick help. I found and removed duplicates in the xml file.