cobwebch / external_import

Other
20 stars 16 forks source link

External Import does not work as scheduled task #283

Closed hoermannklaus closed 1 year ago

hoermannklaus commented 1 year ago

I use the extension "external_import" in version 6.3.2, as well as the extensions "svconnector" (4.0.0) and "svconnector_json" (3.0.0).

I have a file "partner.json" which imports custom Partner data into my custom extension table. I defined the TCA properties, and when I run the import manually using the backend module "Data Import" everything works fine. Also when I run the scheduled task manually everything works fine.

But when the scheduler runs "automatically" (e.g. 2:20 in the night), I get an error in the "External Import -> Log". The scheduled task itself does not have an error message. But when I look into the "External Import -> Log" I see this: image

The first thing which struck me, was that in the column "User" only a dash "-" is contained. When I look a the import results using the "List" module I see this: image

"Wert is nicht erlaubt" (German) means "Value is not allowed". The inserted user has the uid 0, but my "cli" user has the uid 3. Not sure if this is a problem.

The second thing which is of interest is the "Message" field. It says: "The import was aborted. Please see the relevant messages." Where are the relevant messages?

Long story short: Do you have an idea, what the problem could be? Any tipps or hints on how to investigate this error further?

Thanks

fsuter commented 1 year ago

Hi. This is weird, you are right that the use should be "cli". I have no idea why this happens.

About the messages, you can check if there's something in the sys_log table. Otherwise you may want to log all messages from External Import in a dedicated log file (see example configuration on page: https://docs.typo3.org/p/cobweb/external_import/6.3/en-us/Installation/Configuration/Index.html?highlight=logger).

HTH

hoermannklaus commented 1 year ago

Thanks for your quick reply.

TLDNR: It works now!

First I added the logging configuration as you suggested. This did not result in any messages in my case.

Then I started executing the External Import using the CLI. That's where I discovered that it makes a difference whether to use the typo3 or the typo3cms (TYPO3 console) command.

I generally use the typo3cms scheduler:run command in my crontab to execute the scheduler using the CLI. When I tried to execute this, I got an error. When I used typo3 scheduler:run the task was executed successfully.

It seems that the typo3cms command reacts differently than the typo3 command to PHP WARNINGS.

Digging deeper I managed to find the "problem". I implemented a custom ExternalImport DataHandler, where I assigned a value to an array part, which did not exist. PHP although creates the array and sets the values, but a WARNING was thrown.

image

A warning seems to make the typo3console break, but not the typo3 command.

I then fixed the PHP code, creating these arrays if they did not exist, and now it works also using the TYPO3 Console.

Thanks

fsuter commented 1 year ago

Wow, that's very good to know!

I'm glad you could find out what the problem was, nice debugging.