Open HeicoSindelItShapes opened 2 years ago
Additional question: is the data-importer designed for importing bulk data (mostly 20.000 to 100.000 objects; sometimes up to 1.000.000 objects)? which type (sftp, push,...) is recommended when a daily update is necessary?
The reason pimcores SftpLoader doesn't work is because it uses the following copy-command that copies the files within the same sftp connection. It doesn't download the file to the local directory.
$filesystem = new Filesystem(new SftpAdapter($connectionProvider, '/'));
try {
$filesystem->copy($this->remotePath, $this->importFilePath);
} catch (FilesystemException $e) {
// Error stuff
}
Since we're developing our own specialized SftpLoader (overriding the old one) we could instead create one SftpAdapter and one LocalFilesystemAdapter and transfer the stream from sftp to the local filesystem:
$fileSystemSFTP = new Filesystem(new SftpAdapter($connectionProvider, '/'));
$adapter = new LocalFilesystemAdapter('/');
$filesystemLocal = new Filesystem($adapter);
try {
$readStream = $fileSystemSFTP->readStream($file->path());
$filesystemLocal->writeStream($this->importFilePath, $readStream);
return $this->importFilePath;
} catch (FilesystemException $e) {
// Error stuff
}
would you like to contribute that? would be great. thx!
Expected behavior
A Json-File is stored on a SFTP-Server. When starting the import (manually) the file should be copied from the SFTP-Server in a tmp-folder to load it from there and import the data in pimcore
Actual behavior
After starting the job, an error ("Could not copy from remote location") appears
Steps to reproduce