search

SAP / sap-commerce-db-sync

SAP Commerce extensions to perform table-to-table replication in single-directionally manner between two SAP Commerce instances or between SAP Commerce and an external database.
Apache License 2.0
29 stars 12 forks source link

Migration (SAP HANA) shows only " 0 / xx | completed in xxx ms" #12

Closed Otiom closed 4 months ago

Otiom commented 11 months ago

Our onPrem DB is HANA DB. Platform version is 2205

We run the migration tool , but after 2 second we were done. Log showed this entries (always 0 /xx | completed) :

2023-12-20T10:56:08.671Z | units->units | 0 / 115 | Completed in 48.07 ms 2023-12-20T10:56:08.722Z | pricerows->pricerows | 0 / 34858 | Completed in 82.82 ms 2023-12-20T10:56:08.796Z | composedtypes->composedtypes | 0 / 1535 | Completed in 101.3 ms 2023-12-20T10:56:08.815Z | basestore2languagerel->basestore2languagerel | 0 / 2 | Completed in 59.10 ms 2023-12-20T10:56:08.990Z | credentials6612sn->credentials6612sn | 0 / 1 | Completed in 189.1 ms 2023-12-20T10:56:09.053Z | endpoints6610sn->endpoints6610sn | 0 / 4 | Completed in 233.5 ms 2023-12-20T10:56:09.109Z | products->products | 0 / 182320 | Completed in 87.27 ms

When searching for „ERROR“ during the migration time i find : ERROR [hybrisHTTP35] [DefaultDatabaseCopyTaskRepository] {} Couldn't fetch migrationId due to: SQLServerException: Invalid object name 'MIGRATIONTOOLKIT_TABLECOPYSTATUS'. INFO [hybrisHTTP30] [MigrationHybrisMSSqlPlatform] {} Executed 115 SQL command(s) with 0 error(s)

When searching for the Migration id in the opensearch i find this as the last entries: INFO [MigrationMonitor] [CustomClusterDatabaseCopyScheduler] {migrationID=af48d6fb-b10c-4457-8bf5-f2fe5f653d53} 710/710 tables migrated. 0 failed. State: PROCESSED

INFO [MigrationMonitor] [CustomClusterDatabaseCopyScheduler] {migrationID=af48d6fb-b10c-4457-8bf5-f2fe5f653d53} Migration finished (PROCESSED) in 00:00:31.866

INFO [MigrationMonitor] [CustomClusterDatabaseCopyScheduler] {migrationID=af48d6fb-b10c-4457-8bf5-f2fe5f653d53, pipeline=solrindexoperation->solrindexoperation, clusterID=3} 0/99299 processed. Completed in {70.53 ms}. Last Update: {2023-12-20T10:56:12.891}

Do you have any idea on the MIGRATIONTOOLKIT_TABLECOPYSTATUS error ?

Thanks, Kai

lnowakowski commented 11 months ago

Hello, this error message is irrelevant here, it just indicates that MIGRATIONTOOLKIT_TABLECOPYSTATUS table was not created yet (this was done based on your log here: Executed 115 SQL command(s) with 0 error(s)), we'll remove or decrease level of this message, as it's confusing. Still I'm trying to figure out what could be an actual root cause of your issue, that all tables were processed but no data migrated.

Otiom commented 10 months ago

@lnowakowski my collegue used the same code in the S1 environment (before it was D1). In the S1 environment, the migration did not have this issue anymore, but it was the internal issue 62, which seems fixed in the develop branch.

Maybe it was in D1 related to the limited resources ?

PS: i just remembered something else: the developer was doing that initially with a source system that was on version 2105. When we did now the migration, we did this with a version of 2205.

PPS: We did not do the migration of media and cronjobs table in the first round; when we did this today with the classcast fix, then the images were transferred. After that we did the cronjobs and the initial issue happened again (0 of 670 were migrated ). Anything that I can check or where I should add aditional logs ?

thendricks0 commented 7 months ago

We have a similar problem on 2205 with a HANA DB:

2024-04-12T14:25:28.193Z | configitems->configitems | 0 / 0 | Completed in 0
2024-04-12T14:25:28.210Z | metainformations->metainformations | 0 / 1 | In progress...
2024-04-12T14:25:28.214Z | numberseries->numberseries | 0 / 303 | In progress...
2024-04-12T14:25:28.219Z | ydeployments->ydeployments | 0 / 507 | In progress...
2024-04-12T14:25:28.223Z | aclentries->aclentries | 0 / 7926 | In progress...
2024-04-12T14:25:28.597Z | ydeployments->ydeployments | 507 / 507 | Completed in 342.2 ms
2024-04-12T14:25:28.749Z | metainformations->metainformations | 1 / 1 | Completed in 142.8 ms
2024-04-12T14:25:29.411Z | numberseries->numberseries | 303 / 303 | Completed in 657.3 ms
2024-04-12T14:25:29.428Z | aclentries->aclentries | 7926 / 7926 | Completed in 1.174 s

These are all the tables that get copied even though the source database as products with 200k rows. There are a lot of missing tables in general.

When trying to migrate schema it doesn't generate any SQL but shows an info message about duplicate tables.

How should I debug the issue?

lnowakowski commented 7 months ago

Try running following Groovy code: defaultDataCopyItemProvider.get(migrationContext).each { i -> println "${i.sourceItem} -> ${i.targetItem}" } It will display all eligible table pairs for migration respecting current setup.

If you don't see all tables there check

thendricks0 commented 7 months ago

Oh yes the schema was missing! The db.url of the source server had no schema parameter. But apparently in our case the schema is the same as the db.username of the connection.

I found the correct schema with this SQL queries within the HAC select * from schemas And to verify the schema contains the hybris tables I want select * from tables where SCHEMA_NAME ='<schema_name>'

Thank you for the hint with the schema!

lnowakowski commented 4 months ago

No further action needed