Open hotrush opened 6 years ago
EIO
needs a little care some times, you can set a call invoker (React\Filesystem\CallInvokerInterface
) when you try the call invokers in this package to see if it the errors go away.
@WyriHaximus unfortunately can not make it works. Try my snippet with eio and periodic timer)
@hotrush will do it together with https://github.com/reactphp/filesystem/issues/18#issuecomment-419674559
I'm experiencing a similar issue with a simple create temp file, write, rename cycle within a periodic timer. Every few iterations eio_open returns with -1. I have trier various Invokers without luck. As per the initial reporter, I can see file handles been open in lsof output and eventually the number of open files limit is hit.
If I run a similar loop calling eio_* functions directly I cannot reproduce the issue.
Issue occurring on AWS EC2 Linux template, running the same code on MacOSX does not show the issue.
Pecl package eio version 2.0.4 and ev 1.0.4 installed on both systems. Running php 7.2.12
Failing code snippet:
$loop = React\EventLoop\Factory::create();
$filesystem = \React\Filesystem\Filesystem::create($loop, ['invoker' => 'React\Filesystem\QueuedInvoker']);
$saveHandler = $loop->addPeriodicTimer(4, function() use ($filesystem) {
$filename = tempnam("/tmp", ".foobar.tmp");
$file = $filesystem->file($filename);
echo "Created Temp file" . PHP_EOL;
$file->putContents(date("c"))
->then(function () use ($file, $filename, $filesystem) {
echo "Written to Temp file" . PHP_EOL;
$file->rename("/tmp/foo.bar")
->then(function (\React\Filesystem\Node\FileInterface $newfile) use ($filename, $filesystem)
{
echo "Renamed tmp file" . PHP_EOL;
}, function ($error) {
throw $error;
});
}, function ($error) {
throw $error;
});
});
/** GO! */
$loop->run();
EIO Code working snippet:
<?php
while(1)
{
$path = tempnam("/tmp", "eio.tmp");
eio_open($path, 65, 496, 0 , function ($path, $fd) {
if ($fd === -1) {
echo "fail";
}
else {
eio_write($fd, "Hello");
eio_close($fd);
eio_rename($path, "/tmp/eio.txt");
echo "done";
}
});
eio_event_loop();
usleep(10000);
}
I realised my eio example wasn't the best and I re-wrote it to work the same way as react filesystem and can replicate the issue.
The call to eio_open is occasionally returning FALSE instead of the resource but then successfully calling the callback when it opens the file. In the basic eio example this then can continue to write to the file. However react filesystem cannot because the open promise has been rejected due to the failed call to eio_open().
Reading the doco http://php.net/manual/en/function.eio-open.php I'm not sure the check for the return value from eio_open is correct. The documentation doesn't define what the return result should be from the function, it only specifies that a failure would be a -1 in the result code sent to the callback
https://github.com/reactphp/filesystem/blob/master/src/Eio/Adapter.php#L407
An update on this, I've been running for the last 5 days with
if (!call_user_func_array($function, $args)) {
$name = $function;
if (!is_string($function)) {
$name = get_class($function);
}
$exception = new RuntimeException('Unknown error calling "' . $name . '"');
$exception->setArgs($args);
$deferred->reject($exception);
};
replaced by
call_user_func_array($function, $args)
So far, no leaking open file handles, files are getting successfully created, opened, written to, and renamed with not error seen.
The only downside to this is that you can't get a reference to the request if you wanted to cancel the request. Not sure if anyone actually want's to do such a thing (eg cancelling an open file request, there is a good chance the file will be opened before the request is cancelled). However if libeio sometimes returns failure on eio_open() but then successfully completes the action you have to handle it the best way.
For completeness I compiled pecl eio with the current version of libeio instead of the version from 2012 currently in the codebase, no change.
Eio is quite a diva. I've been running for a few days react/fs with eio, with a periodic timer (2mins interval) to write data to a single file (always overwriting). Every 3rd or 4th time I got an unknown function eio_open
error, all day and night long. To not get my logs clogged, I've switched to the child process adapter for now.
In my newish project I'm using pthreads and thus am using my pthreads adapter (with an improved react/fs (#45)).
I have the same error: "Unknown error calling "eio_open".
From documentation
eio_open() returns file descriptor in result argument of callback on success; otherwise, result is equal to -1.
In my case eio sometimes return false instead of descriptor of file.
I also found an error when not the whole file is read:
create file
php -r 'foreach(range(0, 1000000) as $i) { print $i . PHP_EOL; }' > static/test.txt
create script
<?php
use \React\Promise\Deferred;
$loader = require __DIR__ . '/vendor/autoload.php';
#$loop = React\EventLoop\Factory::create();
$loop = new React\EventLoop\ExtEvLoop();
$filesystem = \React\Filesystem\Filesystem::create($loop);
$file = $filesystem->file('static/test.txt');
$file->exists()
->then(function () use ($file) {
return $file->open('r');
})
->then(function ($stream) use ($loop) {
$d = new Deferred();
$stream->on('data', function ($data) {
print $data;
});
$stream->on('end', function ($data) use ($stream, $d) {
$stream->close();
$d->resolve('finish');
});
return $d->promise();
})
->then(function () use ($loop) {
$loop->stop();
})
->otherwise(function ($e) {
var_dump($e);
});
$loop->run();
php7.3 3.php
1
2
3
...
375148
375149
3751root@7cf1e7fada2c:/app/adv#
I create separate topic https://github.com/reactphp/filesystem/issues/73 about this bug.
I faced an issue with eio adapter. Simple snippet to reproduce:
First tick written ok, others cause an error:
Opened files number growing and when reach system limits causing other errors e.g. DNS query failed or ssl handshake failed.
Here is another strange thing -
eio_open
causes an error, butls -l /proc/23566/fd
shows opened files, e.g. (but no real files created):It works fine at my local deepin linux (based on debian 8), but can not make it work at any server with ubuntu 16.04/18.04. Any ideas how to debug it?