String-Value overwritten with integer (sorting-value)

[DavidBruchmann]

There is an issue with the Datahandler and I couldn't find yet the reason. I have two queries and backtraces where the first pair is right but overwritten by the second one:

`INSERT INTO tx_myextension_domain_model_spezialisierung (spezialisierung,pid,crdate,cruser_id,tstamp) VALUES ('Katathym-imaginative Verfahren','154','1492972609','19','1492972609');

debug_backtrace => ' call_user_func#21 // {closure}# // TYPO3\CMS\Backend\Http\Application->run#20 // TYPO3\CMS\Core\Core\Bootstrap->handleRequest#94 // TYPO3\CMS\Backend\Http\BackendModuleRequestHandler->handleRequest#302 // TYPO3\CMS\Backend\Http\BackendModuleRequestHandler->dispatchModule#92 // TYPO3\CMS\Extbase\Core\Bootstrap->run#214 // TYPO3\CMS\Extbase\Core\Bootstrap->handleRequest#193 // TYPO3\CMS\Extbase\Mvc\Web\BackendRequestHandler->handleRequest#206 // TYPO3\CMS\Extbase\Mvc\Dispatcher->dispatch#32 // TYPO3\CMS\Extbase\Mvc\Controller\ActionController->processRequest#86 // TYPO3\CMS\Extbase\Mvc\Controller\ActionController->callActionMethod#176 // call_user_func_array#283 // Cobweb\ExternalImport\Controller\DataModuleController->synchronizeAction# // Cobweb\ExternalImport\Importer->synchronizeData#199// Cobweb\ExternalImport\Importer->handleData#358 // Cobweb\ExternalImport\Importer->storeData#566 // TYPO3\CMS\Core\DataHandling\DataHandler->;process_datamap#1378 // TYPO3\CMS\Core\DataHandling\DataHandler->insertDB#1330 // TYPO3\CMS\Core\Database\DatabaseConnection->exec_INSERTquery#6793 // TYPO3\CMS\Core\Database\DatabaseConnection->debug#224'

UPDATE tx_myextension_domain_model_spezialisierung SET spezialisierung='512' WHERE uid=1;

debug_backtrace => ' call_user_func#21 // {closure}# // TYPO3\CMS\Backend\Http\Application->run#20 // TYPO3\CMS\Core\Core\Bootstrap->handleRequest#94 // TYPO3\CMS\Backend\Http\BackendModuleRequestHandler->handleRequest#302 // TYPO3\CMS\Backend\Http\BackendModuleRequestHandler->dispatchModule#92 // TYPO3\CMS\Extbase\Core\Bootstrap->run#214 // TYPO3\CMS\Extbase\Core\Bootstrap->handleRequest#193 // TYPO3\CMS\Extbase\Mvc\Web\BackendRequestHandler->handleRequest#206 // TYPO3\CMS\Extbase\Mvc\Dispatcher->dispatch#32 // TYPO3\CMS\Extbase\Mvc\Controller\ActionController->processRequest#86 // TYPO3\CMS\Extbase\Mvc\Controller\ActionController->callActionMethod#176 // call_user_func_array#283 // Cobweb\ExternalImport\Controller\DataModuleController->synchronizeAction# // Cobweb\ExternalImport\Importer->synchronizeData#199// Cobweb\ExternalImport\Importer->handleData#358 // Cobweb\ExternalImport\Importer->storeData#566 // TYPO3\CMS\Core\DataHandling\DataHandler->;process_datamap#1378 // TYPO3\CMS\Core\DataHandling\DataHandler->getSortNumber#1081 // TYPO3\CMS\Core\DataHandling\DataHandler->resorting#6957 // TYPO3\CMS\Core\Database\DatabaseConnection->exec_UPDATEquery#7045 // TYPO3\CMS\Core\Database\DatabaseConnection->debug#271' (1212 chars)' `

The incoming values are right and I tried it with deactivating any sorting-fields of the table too but I still have the issue that the first value is reproducible overwritten with the sorting-value "512". I don't know how the second query with the update can be suppressed or corrected. I'm not sure if the fault is in the datahandler / core or in external_import or even my extension.

When calling the import-function a second time the wrong value is corrected again but I don't want to explain the customer that he's to call the import twice, especially as it's only the first part of a larger import (where I still have to track down other issues).

[fsuter]

That behaviour does ring a bell, but I can't remember the details. I'm pretty sure I already encountered a situation where the sorting field's value was written into the label field. Right now I don't remember what the problem was and whether it was strictly related to External Import or a more general issue. Maybe it will come back to me.

Could you post the full TCA for your table? Maybe it will trigger some memory.

[DavidBruchmann]

For the tca see the file in the next comment.

Thanks for the fast answer. The mapping for additionalFields is working fine, so the issue is not related to this. The strange thing is that the described fault is only in one of several rows, in the example-data it was the last record, which in datahandler becomes the first one.

[DavidBruchmann]

tca.php.txt here the tca as file.

[DavidBruchmann]

Having deactivated the "sorting" field in the TCA I get the right results now. So it's a matter of documentation, might be that I missed corresponding hints from your doc, I never checked now. I cleared all caches including autoloader and Cache in typo3temp and finally it's not overwritten. Responsible for the issue is in the datahandler-class the block starting with line 1073 where inside "if ($sortRow)" "$this->getSortNumber()" is called. So, avoid "sort" if it's not really only a sorting-field with integer but use "default_sortby" instead.

[DavidBruchmann]

$this->getSortNumber() can be found 2 times below:

                        if ($OK) {
                            $sortRow = $GLOBALS['TCA'][$table]['ctrl']['sortby'];
                            // Points to a page on which to insert the element, possibly in the top of the page
                            if ($pid_value >= 0) {
                                // If this table is sorted we better find the top sorting number
                                if ($sortRow) {
                                    $fieldArray[$sortRow] = $this->getSortNumber($table, 0, $pid_value);
                                }
                                // The numerical pid is inserted in the data array
                                $fieldArray['pid'] = $pid_value;
                            } else {
                                // points to another record before ifself
                                // If this table is sorted we better find the top sorting number
                                if ($sortRow) {
                                    // Because $pid_value is < 0, getSortNumber returns an array
                                    $tempArray = $this->getSortNumber($table, 0, $pid_value);
                                    $fieldArray['pid'] = $tempArray['pid'];
                                    $fieldArray[$sortRow] = $tempArray['sortNumber'];
                                } else {
                                    // Here we fetch the PID of the record that we point to...
                                    $tempdata = $this->recordInfo($table, abs($pid_value), 'pid');
                                    $fieldArray['pid'] = $tempdata['pid'];
                                }
                            }
                        }

[fsuter]

Glad that you sorted this out. I remember that this was indeed the issue I stumbled upon back then: it's very bad idea to use the sorting field for anything else, as this field is filled automatically by the DataHandler class. Note that this has nothing to do with External Import BTW.

[DavidBruchmann]

Yeah sure, it never has to do with external_import, that's clear. Nevertheless a hint in the documentation might be helpful I suppose. Thanks for you concerns ;-)

[glucka]

I had today the same issue by import the language overlays. Typo3 9.5.11 After 3-4 hours find this ticket :) Thanks @DavidBruchmann !!! Changing 'sortby' to 'default_sortby' solve my problem too!

[DavidBruchmann]

great that you could limit it to 4 hours by finding my post, @glucka ;-)