krakjoe / parallel

A succinct parallel concurrency API for PHP8
Other
1.48k stars 96 forks source link

SIGSEGV, Segmentation fault. #315

Closed testAccountDeltas closed 4 months ago

testAccountDeltas commented 4 months ago

This error appears on php 8.3.8 with the latest sources parallel

Starting program: H:\php\Debug_TS\php.exe 1.php
[New Thread 1948.0x430c]
[New Thread 1948.0x1ec0]
[New Thread 1948.0x30a8]
[New Thread 1948.0x3a54]
[New Thread 1948.0x1680]
[New Thread 1948.0x4f4]
[New Thread 1948.0x1c5c]
[New Thread 1948.0x34b0]
[New Thread 1948.0x2094]
[New Thread 1948.0xae4]
[New Thread 1948.0x48dc]
[New Thread 1948.0x15a0]
[New Thread 1948.0x1704]
[New Thread 1948.0x2450]
[New Thread 1948.0x33c]
[New Thread 1948.0x4a88]
[New Thread 1948.0x457c]
[New Thread 1948.0x4280]
[New Thread 1948.0x1d2c]
[Thread 1948.0x3a54 exited with code 0]
[Thread 1948.0x1680 exited with code 0]
[Thread 1948.0x4f4 exited with code 0]
[Thread 1948.0x1c5c exited with code 0]
[Thread 1948.0x34b0 exited with code 0]
[Thread 1948.0x2094 exited with code 0]
[Thread 1948.0xae4 exited with code 0]
[Thread 1948.0x48dc exited with code 0]
[Thread 1948.0x15a0 exited with code 0]
[Thread 1948.0x1704 exited with code 0]
[Thread 1948.0x2450 exited with code 0]
[Thread 1948.0x33c exited with code 0]
[Thread 1948.0x4a88 exited with code 0]
[Thread 1948.0x457c exited with code 0]
[Thread 1948.0x4280 exited with code 0]
[Thread 1948.0x1d2c exited with code 0]
[New Thread 1948.0x4380]
[New Thread 1948.0x21a4]
[New Thread 1948.0x533c]
VirtualProtect() failed [87] The parameter is set incorrectly
VirtualProtect() failed [87] The parameter is set incorrectly
VirtualProtect() failed [87] The parameter is set incorrectly
VirtualProtect() failed [87] The parameter is set incorrectly
[New Thread 1948.0x4670]
VirtualProtect() failed [87] The parameter is set incorrectly
VirtualProtect() failed [87] The parameter is set incorrectly
VirtualProtect() failed [87] The parameter is set incorrectly
VirtualProtect() failed [87] The parameter is set incorrectly
VirtualProtect() failed [87] The parameter is set incorrectly
VirtualProtect() failed [87] The parameter is set incorrectly
VirtualProtect() failed [87] The parameter is set incorrectly
VirtualProtect() failed [87] The parameter is set incorrectly
VirtualProtect() failed [87] The parameter is set incorrectly
VirtualProtect() failed [87] The parameter is set incorrectly
VirtualProtect() failed [87] The parameter is set incorrectly
VirtualProtect() failed [87] The parameter is set incorrectly
VirtualProtect() failed [87] The parameter is set incorrectly
VirtualProtect() failed [87] The parameter is set incorrectly
VirtualProtect() failed [87] The parameter is set incorrectly
VirtualProtect() failed [87] The parameter is set incorrectly
VirtualProtect() failed [87] The parameter is set incorrectly
VirtualProtect() failed [87] The parameter is set incorrectly

Thread 24 received signal SIGSEGV, Segmentation fault.
[Switching to Thread 1948.0x4670]
0x00007ffda491537c in zend_update_class_constants () from H:\php\Debug_TS\php8ts_debug.dll
(gdb)backtrace
#0 0x00007ffda491537c in zend_update_class_constants () from H:\php\Debug_TS\php8ts_debug.dll
#1 0x00007ffda49153e1 in zend_update_class_constants () from H:\php\Debug_TS\php8ts_debug.dll
#2 0x00007ffda4920909 in zend_parse_arg_str_or_long_slow@@32 ()
 from H:\php\Debug_TS\php8ts_debug.dll
#3 0x00007ffda4917421 in php8ts_debug!object_init_ex () from H:\php\Debug_TS\php8ts_debug.dll
#4 0x00007ffda49d5a11 in zend_throw_conflicting_coercion_error()
 from H:\php\Debug_TS\php8ts_debug.dll
#5 0x00007ffda498e959 in php8ts_debug!execute_ex () from H:\php\Debug_TS\php8ts_debug.dll
#6 0x00007ffda5283ecb in php_version_compare () from H:\php\Debug_TS\php8ts_debug.dll
#7 0x00007ffda5284588 in php_version_compare () from H:\php\Debug_TS\php8ts_debug.dll
#8 0x00007ffde49a4bae in pthread_attr_setscope () from H:\php\Debug_TS\pthreadVC3.dll
#9 0x00007ffe03021bb2 in ucrtbase!_configthreadlocale () from C:\Windows\System32\ucrtbase.dll
#10 0x00007ffe04d37344 in KERNEL32!BaseThreadInitThunk () from C:\Windows\System32\kernel32.dll
#11 0x00007ffe051e26b1 in ntdll!RtlUserThreadStart () from C:\Windows\SYSTEM32\ntdll.dll
#12 0x0000000000000000 in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)
(gdb)

The code provided is for informational purposes only and is not a test code. The problem is mainly connecting the GoogleTranslate library

Example code ```php use Stichoza\GoogleTranslate\GoogleTranslate; function GoogleTranslateData($langSoruce, $langTarget, $data) { try { $tr = new GoogleTranslate(); if($langSoruce && is_string($langSoruce) && trim($langSoruce)) { $tr->setSource($langSoruce); } if($langTarget && is_string($langTarget) && trim($langTarget)) { $tr->setTarget($langTarget); return trim($tr->translate($data)); } else { return false; } } catch (Exception $e) { echo 'Caught exception (GoogleTranslateDataTR): ', $e->getMessage(), "\n"; } return false; } ``` class func GoogleTranslateData ```php public function getTranslationCategories($threadsParseCount, $streamsCount, callable $callback) { $langTranslationTest = []; $categoriesList = $this->getTranslationCategoriesList($threadsParseCount); foreach($categoriesList['translation'] as $langTarget => $langSoruce) { (new ThreadArray($streamsCount))->start($categoriesList['categories'], function($max, $current, $categoryValue, $callback, $langSoruce, $langTarget) { $nameCategoryFile = md5($categoryValue); if(!$this->distCategoryISFile($langTarget, $nameCategoryFile)) { $dataTranslate = GoogleTranslateData($langSoruce, $langTarget, $categoryValue); if($dataTranslate !== false) { $dataWrite = [ 'valueOrig' => $categoryValue, 'value' => $dataTranslate ]; $this->distCategoryFileWrite($langTarget, $nameCategoryFile, $dataWrite); if(is_callable($callback)) { $callback($max, $current, ['name' => $categoryValue], ['name' => $dataTranslate]); } } } return false; }, $callback, $langSoruce, $langTarget); } } ```
all other code ThreadArray ```php trait filesDistTranslationFunction { // Получим полный путь до папки категорий public function distCategoriesLangFilesDir($lang) { return $this->filesDir('distCategoriesLangFiles_' . $lang); } // Получим полный путь до файла категории public function distCategoryFilePath($lang, $item) { return $this->fileDir('distCategoriesLangFiles_' . $lang, $item); } // Проверим существует ли файл категории public function distCategoryISFile($lang, $item) { return $this->fileDirIs('distCategoriesLangFiles_' . $lang, $item); } // Читаем файл категории public function distCategoryFile($lang, $filePath) { return $this->fileRead('distCategoriesLangFiles_' . $lang, $filePath); } // Для записи в файл категории public function distCategoryFileWrite($lang, $item, $data, $lastmod = false) { return $this->fileWrite('distCategoriesLangFiles_' . $lang, $item, $data, $lastmod); } // Получим файлы из папки категории public function distCategoriesFiles($lang) { return iterator_to_array(GetFilesGenerator('/.json/', true, $this->distCategoriesLangFilesDir($lang))); } private function getTranslationCategoriesList($streamsCount) { if(is_array($categories = (new ThreadArray($streamsCount))->start($this->distFiles(), function($max, $current, $filePath) { $dataFile = $this->distFile($filePath); $categories = []; if(isset($dataFile['categories']) && is_array($dataFile['categories'])) { $categories = $dataFile['categories']; if(isset($dataFile['categories']) && (bool)$dataFile['categoriesNewFirstNameSkipTranslation']) { if(count($categories)) { $categories = array_splice($categories, 1); } } } $langList = isset($dataFile['langTranslation']) ? $dataFile['langTranslation'] : []; if(is_string($langList)) { $langList = (array)$langList; } if(!is_array($langList)) { $langList = []; } $langList = array_flip($langList); foreach($langList as $indexLang => $indexValue) { $langList[$indexLang] = isset($dataFile['langPage']) ? $dataFile['langPage'] : false; } return [ 'translation' => $langList, 'categories' => $categories, ]; }))) { $resultCategories = [ 'translation' => [], 'categories' => [] ]; foreach($categories as $itemDataArray) { foreach($itemDataArray as $itemData) { if(!count($itemData['categories'])) { continue; } $resultCategories['translation'] = array_merge($resultCategories['translation'], $itemData['translation']); $resultCategories['categories'] = array_merge($resultCategories['categories'], $itemData['categories']); } } $resultCategories['categories'] = array_unique($resultCategories['categories']); } else { $resultCategories = []; } return $resultCategories; } public function getTranslationCategories($threadsParseCount, $streamsCount, callable $callback) { $langTranslationTest = []; $categoriesList = $this->getTranslationCategoriesList($threadsParseCount); foreach($categoriesList['translation'] as $langTarget => $langSoruce) { (new ThreadArray($streamsCount))->start($categoriesList['categories'], function($max, $current, $categoryValue, $callback, $langSoruce, $langTarget) { $nameCategoryFile = md5($categoryValue); if(!$this->distCategoryISFile($langTarget, $nameCategoryFile)) { $dataTranslate = GoogleTranslateData($langSoruce, $langTarget, $categoryValue); if($dataTranslate !== false) { $dataWrite = [ 'valueOrig' => $categoryValue, 'value' => $dataTranslate ]; $this->distCategoryFileWrite($langTarget, $nameCategoryFile, $dataWrite); if(is_callable($callback)) { $callback($max, $current, ['name' => $categoryValue], ['name' => $dataTranslate]); } } } return false; }, $callback, $langSoruce, $langTarget); } } public function distTranslation($threadsParseCount, $streamsCount, callable $callback) { $categoriesLang = $this->getTranslationCategories($threadsParseCount, $streamsCount, $callback); // ... other code } // ... other code } ``` ```php use parallel\Runtime; use parallel\Future; use parallel\Error\Error; class ThreadError { public $message = ''; public $trace = ''; public function __construct($message, $trace) { $this->message = $message; $this->trace = $trace; } public function getError() { return "Error: {$this->message}\nTrace: {$this->trace}"; } } class Thread { private $runtimes = []; private $errorHandler; public static function errorHandle($error) { echo $error->getError(); } public function __construct(callable $errorHandler = null) { $this->errorHandler = is_null($errorHandler) ? 'Thread::errorHandle' : $errorHandler; } public function runReturn(callable $task, callable $taskResult, ...$args) { $item = (new Runtime(__DIR__ . '/include.php'))->run(function ($task, $errorHandler, $taskResult, ...$args) { ini_set('memory_limit', '-1'); set_time_limit(0); set_error_handler(function ($errno, $errstr, $errfile, $errline) { throw new ErrorException($errstr, $errno, 0, $errfile, $errline); }); try { $value = $task(...$args); if (is_callable($taskResult)) { $taskResult($value); } return $value; } catch (Throwable $e) { ($errorHandler)(new ThreadError($e->getMessage(), $e->getTraceAsString())); } finally { restore_error_handler(); } }, [$task, $this->errorHandler, $taskResult, ...$args]); $this->runtimes[] = $item; return $item; } public function run(callable $task, ...$args) { return $this->runReturn($task, function () {}, ...$args); } public function wait() { $resultAll = []; foreach ($this->runtimes as $index => $future) { while (!$future->done()) { usleep(10000); } $result = $future->value(); if (!($result instanceof ThreadError)) { $resultAll[] = $result; } unset($this->runtimes[$index]); } $this->runtimes = array_values($this->runtimes); return $resultAll; } public function killRuntimes() { foreach ($this->runtimes as &$future) { if($future instanceof parallel\Future) { $future->kill(); } } $this->runtimes = []; } public function __destruct() { $this->runtimes = []; } } class ThreadArray extends Thread { private $streamsCount; private $usleep; public function __construct(int $streamsCount = 5, int $usleep = 0, callable $errorHandler = null) { parent::__construct($errorHandler); $this->streamsCount = $streamsCount; $this->usleep = $usleep; } public function set_streams_count(int $streamsCount) { $this->streamsCount = $streamsCount; } public function get_streams_count() { return $this->streamsCount; } public function start($ArrayLines, callable $callback, ...$args) { $ArrayLines = (array) $ArrayLines; $maxIndex = count($ArrayLines); if (!$maxIndex) return []; $currentIndex = new parallel\Sync(0); foreach (array_chunk($ArrayLines, ceil($maxIndex / $this->streamsCount)) as $index => $chunk) { $this->run(function ($chunk, $maxIndex, $callback, $usleep, ...$args) use ($currentIndex) { $results = []; $start = true; foreach ($chunk as $ItemArray) { if(!$start && $usleep) usleep($usleep); $currentIndex->set((int)$currentIndex->get() + 1); if($value = $callback($maxIndex, (int)$currentIndex->get(), $ItemArray, ...$args)) { $results[] = $value; } $start = false; } return $results; }, $chunk, $maxIndex, $callback, $this->usleep, ...$args); } return $this->wait(); } } ```
testAccountDeltas commented 4 months ago

The bug is really a call to the GoogleTranslate library. I have now rewritten the GoogleTranslate functionality and I don’t have a single error.

new code code ![Снимок экрана (889)](https://github.com/krakjoe/parallel/assets/37482617/29f1777b-5a3e-4399-b4d7-992a3b7e19c2) ```php public function getTranslationCategories($threadsParseCount, $streamsCount, callable $callback) { $langTranslationTest = []; $categoriesList = $this->getTranslationCategoriesList($threadsParseCount); foreach($categoriesList['translation'] as $langTarget => $langSoruce) { (new ThreadArray($streamsCount))->start($categoriesList['categories'], function($max, $current, $categoryValue, $callback, $langSoruce, $langTarget) { $nameCategoryFile = md5($categoryValue); if(!$this->distCategoryISFile($langTarget, $nameCategoryFile)) { try { if($dataTranslate = googleTranslation($langSoruce, $langTarget, $categoryValue, $lastDetectedSource)) { $dataWrite = [ 'valueOrig' => $categoryValue, 'value' => $dataTranslate ]; $this->distCategoryFileWrite($langTarget, $nameCategoryFile, $dataWrite); if(is_callable($callback)) { $callback($max, $current, ['name' => $categoryValue], ['name' => $dataTranslate]); } } } catch (Exception $e) { echo 'Caught exception (GoogleTranslateDataTR): ', $e->getMessage(), "\n"; } } return false; }, $callback, $langSoruce, $langTarget); } } ``` google ```php class RateLimitException extends ErrorException { } class LargeTextException extends ErrorException { } class TranslationRequestException extends ErrorException { } class TranslationDecodingException extends ErrorException { } function googleTranslation($source, $target, $text, &$lastDetectedSource) { $lastDetectedSource = null; if(!is_string($target) || !($target = trim($target))) { $target = 'auto'; } if(!is_string($source) || !($target = trim($target))) { $source = 'ru'; } $queryUrl = preg_replace('/%5B\d+%5D=/', '=', http_build_query(array_merge([ 'client' => 'gtx', 'hl' => 'en', 'dt' => [ 't', // Translate 'bd', // Full translate with synonym ($bodyArray[1]) 'at', // Other translate ($bodyArray[5] - in google translate page this shows when click on translated word) 'ex', // Example part ($bodyArray[13]) 'ld', // I don't know ($bodyArray[8]) 'md', // Definition part with example ($bodyArray[12]) 'qca', // I don't know ($bodyArray[8]) 'rw', // Read also part ($bodyArray[14]) 'rm', // I don't know 'ss' // Full synonym ($bodyArray[11]) ], 'sl' => null, // Source language 'tl' => null, // Target language 'q' => null, // String to translate 'ie' => 'UTF-8', // Input encoding 'oe' => 'UTF-8', // Output encoding 'multires' => 1, 'otf' => 0, 'pc' => 1, 'trs' => 1, 'ssel' => 0, 'tsel' => 0, 'kc' => 1, 'tk' => null, ], [ 'sl' => $source, 'tl' => $target, 'tk' => googleTranslationGenerateToken($source, $target, $text), 'q' => $text ]))); $url = 'https://translate.google.com/translate_a/single?' . $queryUrl; $data = curlRequest($url, false, false); switch($data->code) { case 200: $bodyJson = preg_replace(['/,+/', '/\[,/', '/\xc2\xa0/'], [',', '[', ' '], $data->body); try { $responseArray = json_decode($bodyJson, true, flags: JSON_THROW_ON_ERROR); } catch (JsonException) { throw new TranslationDecodingException('Data cannot be decoded or it is deeper than the recursion limit'); } if (empty($responseArray[0])) { return false; } $detectedLanguages = []; foreach ($responseArray as $item) { if (is_string($item)) $detectedLanguages[] = $item; } if (isset($responseArray[count($responseArray) - 2][0][0])) { $detectedLanguages[] = $responseArray[count($responseArray) - 2][0][0]; } foreach ($detectedLanguages as $lang) { if ((bool) preg_match('/^([a-z]{2,3})(-[A-Za-z]{2,4})?$/', $lang)) { $lastDetectedSource = $lang; break; } } if (is_string($responseArray)) { $output = $responseArray; } elseif (is_array($responseArray[0])) { $output = (string) array_reduce($responseArray[0], static function ($carry, $item) { $carry .= $item[0]; return $carry; }); } else { $output = (string) $responseArray[0]; } return $output; break; case 429: case 503: throw new RateLimitException($e->getMessage(), $e->getCode()); break; case 413: throw new LargeTextException($e->getMessage(), $e->getCode()); default: throw new TranslationRequestException($e->getMessage(), $e->getCode()); } return false; } function googleTranslationGenerateToken(string $source, string $target, string $text): string { $tkk = ['406398', 2087938574]; for ($d = [], $e = 0, $f = 0; $f < mb_strlen($text); $f++) { $g = mb_ord(mb_substr($text, $f, 1)); if ($g < 128) { $d[$e++] = $g; } else { if ($g < 2048) { $d[$e++] = $g >> 6 | 192; } else { if ($g & 64512 === 55296 && $f + 1 < mb_strlen($text) && (mb_ord(mb_substr($text, $f + 1, 1)) & 64512) === 56320) { $g = 65536 + (($g & 1023) << 10) + (mb_ord(mb_substr($text, ++$f, 1)) & 1023); $d[$e++] = $g >> 18 | 240; $d[$e++] = $g >> 12 & 63 | 128; } else { $d[$e++] = $g >> 12 | 224; } $d[$e++] = $g >> 6 & 63 | 128; } $d[$e++] = $g & 63 | 128; } } $a = $tkk[0]; foreach ($d as $value) { $a += $value; $a = googleTranslationGenerateToken_rl($a, '+-a^+6'); } $a = googleTranslationGenerateToken_rl($a, '+-3^+b+-f'); $a ^= $tkk[1]; if ($a < 0) { $a = ($a & 2147483647) + 2147483648; } $a = fmod($a, 1000000); return $a . '.' . ($a ^ $tkk[0]); } function googleTranslationGenerateToken_rl(int $a, string $b): int { for ($c = 0; $c < strlen($b) - 2; $c += 3) { $d = $b[$c + 2]; $d = $d >= 'a' ? ord($d[0]) - 87 : (int) $d; $d = $b[$c + 1] === '+' ? googleTranslationGenerateToken_unsignedRightShift($a, $d) : $a << $d; $a = $b[$c] === '+' ? ($a + $d & 4294967295) : $a ^ $d; } return $a; } function googleTranslationGenerateToken_unsignedRightShift(int $a, int $b): int { if ($b >= 32 || $b < -32) { $m = (int) ($b / 32); $b -= ($m * 32); } if ($b < 0) { $b += 32; } if ($b === 0) { return (($a >> 1) & 0x7fffffff) * 2 + (($a >> $b) & 1); } if ($a < 0) { $a >>= 1; $a &= 2147483647; $a |= 0x40000000; $a >>= ($b - 1); } else { $a >>= $b; } return $a; } ```
realFlowControl commented 4 months ago

Hey @testAccountDeltas,

glad that you could solve it. I am sorry to tell you, that I have not been able to reproduce the error so far. As your problem seems gone, I'll close this ticket.

testAccountDeltas commented 4 months ago

Well, if we assume that the solution to the problem is solved by rewriting the library functionality, you are right.