techdivision / import

This is a library that provides generic functionalities for the implementation of imports. In addition to maximum performance and optimized memory consumption, Pacemaker can also be used to implement imports in distributed scenarios that place the highest demands on speed and stability.
https://pacemaker.techdivision.com
MIT License
128 stars 23 forks source link

Add functionality to pass an exit code when stopping the application #203

Open DanieliMi opened 3 years ago

DanieliMi commented 3 years ago

Currently the exit code is always 1 when calling \TechDivision\Import\ApplicationInterface::stop. It would be nice to have an option to stop the application with exit code 0.

amenk commented 3 years ago

would exit(0) work :D ?

amenk commented 3 years ago

We could either introduce a new type of exception or replace https://github.com/techdivision/import-app-simple/blob/master/src/Simple.php#L667

by

return $ase->getCode() ?? 1;

so we can throw such an exception with a custom code

amenk commented 3 years ago

Actually it might be better to introduce a new exception type, because in such cases of allowed stops, we do not need to print the full exception.

Our use case is that we have empty imports in some cases - which are okay - and we do not need to generate an artefact. That's why we simply need to stop the process.

DanieliMi commented 3 years ago

I think it would be nice to be able to pass an exit code to \TechDivision\Import\ApplicationInterface::stop and we could then pass the code to \TechDivision\Import\Exceptions\ApplicationStoppedException which is used as exit code.

DanieliMi commented 3 years ago

I image it something like this: https://github.com/techdivision/import-app-simple/blob/master/src/Simple.php#L638-L668

        } catch (ApplicationStoppedException $ase) {
            // rollback the transaction, if single transaction mode has been configured
            if ($this->getConfiguration()->isSingleTransaction()) {
                $this->getImportProcessor()->getConnection()->rollBack();
            }

            // invoke the event that has to be fired after the application rollbacked the
            // transaction (if single transaction mode has been activated)
            $this->getEmitter()->emit(EventNames::APP_PROCESS_TRANSACTION_FAILURE, $this, $ase);

            // finally, if a PID has been set (because CSV files has been found),
            // remove it from the PID file to unlock the importer
            $this->unlock();

            // track the time needed for the import in seconds
            $endTime = microtime(true) - $startTime;

            // If the exit code is above 0 it's an error that we have to handle like it else it is an expected stop and we gracefully end
            if ($ase->getCode() > 0) {
                // log a message that the file import failed
                foreach ($this->systemLoggers as $systemLogger) {
                    $systemLogger->error($ase->__toString());
                }

                // log a message that import has been finished
                $this->getSystemLogger()->warning(sprintf('Can\'t finish import with serial %s in %f s', $this->getSerial(), $endTime));

                // log the exception message as warning
                $this->log($ase->getMessage(), LogLevel::WARNING);

                // return the exit code of the exception
                return $ase->getCode();
            } else {
                // log a info message that import has been finished
                $this->getSystemLogger()->info($ase->getMessage());
                $this->getSystemLogger()->info(sprintf('Execution time for operation with serial %s in %f s', $this->getSerial(), $endTime));

                // invoke the event that has to be fired before the application has the transaction
                // committed successfully (if single transaction mode has been activated)
                $this->getEmitter()->emit(EventNames::APP_PROCESS_TRANSACTION_SUCCESS, $this);
            }
        }
amenk commented 3 years ago

https://github.com/techdivision/import/pull/211 https://github.com/techdivision/import-app-simple/pull/29

amenk commented 3 years ago

Made some pull requests to have something to discuss, but we should also avoid the rollback on finish. Or we find another way to exit the app if nothing is to do, for example with a series of return statements? @DanieliMi

amenk commented 3 years ago

I think this is a better approach here - so we do not need a PR to the Pacemaker-Core

We define a new LocalFinishedException()

@DanieliMi

public function process()
{
    try {
        $this->export(time(), 1);
    } catch (LocalFinishedException $e) {
        echo $e->getMessage();
        return;
    }
}

public function getArtefacts() {

   ....

    if (empty($artefacts)) {
        throw new LocalFinishedException('Found no simple products to delete.');
    }
}
DanieliMi commented 3 years ago

I don't think this will work because after this there will be following operations executed which will result in Exceptions in this case so we have to stop the whole application.

amenk commented 3 years ago

true, understood

amenk commented 3 years ago

@wagnert what do you think?

We have process chains which should be stopped if initial jobs do not produce output, but this should have a success exit-code. Is there a better way to solve that or do you think our approach is generally feasible?

Currently we use vendor/bin/import-simple .... || true in our bash script to avoid stopping the general chain, but this also covers up other problems.