pluginsGLPI / datainjection

Plugin to import CSV files into GLPI
http://plugins.glpi-project.org/#/plugin/datainjection
GNU General Public License v2.0
52 stars 51 forks source link

No more progress bar after upgrade to 2.7.1 #191

Closed sbraz closed 8 months ago

sbraz commented 4 years ago

Hi, I've recently upgraded GLPI from 9.3.1 to 9.4.5 and the injection plugin from 2.6.3 to 2.7.1 and there is no more progress shown when injecting a file.

The GET call to clientinjection.form.php blocks for several seconds (until the injection is complete) instead of displaying a page where the progress would be checked asynchronously.

I think my issue is similar to #152 but there have been several bugfix releases for GLPI since.

stonebuzz commented 4 years ago

Hi @sbraz

can you give us more informations :

I do not reproduce your problem in my environment.

Best regards

sbraz commented 4 years ago

I'm running PHP 7.3.15 (via mod_php on Apache) on Red Hat 7 and I installed GLPI and the plugin using Remi's packages.

stonebuzz commented 4 years ago

Can you check glpi and apache logs ?

Best regards

stonebuzz commented 4 years ago

And enable console from your browser and retry , to know if you have error from javascript for example

sbraz commented 4 years ago

The only thing I see in Apache's error log doesn't seem to have anything to do with the problem, it's an error that is also present for other pages:

PHP Warning:  preg_match(): Allocation of JIT memory failed, PCRE JIT will be disabled. This is likely caused by security restrictions. Either grant PHP permission to allocate executable memory, or set pcre.jit=0 in /usr/share/php/Zend/Loader/StandardAutoloader.php on line 259, referer: https://xxx/glpi/plugins/datainjection/front/clientinjection.form.php

I don't see anything being added to event.log, cron.log and php-errors.log files in /var/log/glpi at the time of the problem.

I did see an error in the console on one computer running Firefox but I don't see it any more and I still have the problem. The message was:

ReferenceError: templateResult is not defined clientinjection.form.php:213:13
    <anonyme> clientinjection.form.php:213
    jQuery 4
        i
        fireWith
        ready
        K

The issue is also present on Internet Explorer.

I'm pretty sure that this is not a client error, here are my logs that show that the GET call takes 25 esconds (the date is the start of the request):

[02/Mar/2020:15:49:14 +0100] "GET /glpi/plugins/datainjection/front/clientinjection.form.php HTTP/1.1" 200 30855
[02/Mar/2020:15:49:39 +0100] "POST /glpi/plugins/datainjection/ajax/results.php HTTP/1.1" 200 867

Once the GET is done, there is an AJAX call to results.php. What happens on your side, when everything works correctly? I assume the GET call returns immediately and you see several AJAX calls to results.php?

stonebuzz commented 4 years ago

Before import, plugin create progressbar, process import injection and foreach lines manage, change the progressbar position, after, plugin call result.php via an ajax call to get result.

When plugin update progressbar position, glpi flush the system write buffers of PHP with

ob_flush
//and
flush()

To be sure is not related to

PHP Warning:  preg_match(): Allocation of JIT memory failed, PCRE JIT will be disabled. This is likely caused by security restrictions. Either grant PHP permission to allocate executable memory, or set pcre.jit=0 in /usr/share/php/Zend/Loader/StandardAutoloader.php on line 259, referer: https://xxx/glpi/plugins/datainjection/front/clientinjection.form.php

can you try this

add pcre.jit=0 to php.ini file in [Pcre]

restart php / apache and retry

Best regards

sbraz commented 4 years ago

OK I get it: you're not sending the whole page at once and my browser starts displaying it but it stops after the header: image

I've looked at the DOM and I do see several instance of this, stemming from the calls to self::progressBar:

<script type="text/javascript">
//<![CDATA[

$('#doaction_progress_text').text("Importation du fichier... 84% (6 secondes)");

//]]>
</script>

It used to work with GLPI 9.3, have there been any changes to the progress bar functions between 9.3 and 9.4? I have also upgraded PHP from 7.2 to 7.3, could there have been any changes to the flush functions?

In any case, this seems like a very unusual way to do things. What happens if the request takes so long that the server times out? Wouldn't it be better to poll some URL via AJAX requests?

add pcre.jit=0 to php.ini file in [Pcre]

I've done it, this wasn't the cause of the problem.

sbraz commented 4 years ago

The following code works as expected:

<?php
for  ($i = 0; $i<10; $i++){
  echo "test<br>";
  if (function_exists("ob_flush")
    && (ob_get_length() !== false)) {
      ob_flush();
  } 
  flush();
  sleep(1);
}

Since this is more or less what GLPI does, I suspect that the problem has to do with the JS code not running as the page loads.

EDIT: even if I change the DOM for each iteration of the loop, it still works, I don't see why GLPI fails whereas my code works.

stonebuzz commented 4 years ago

Can you tryr this :

undo your changes on glpi_flush function and just add on top

header( 'Content-type: text/html; charset=utf-8' );

like this

   /**
    * Flushes the system write buffers of PHP and whatever backend PHP is using (CGI, a web server, etc).
    * This attempts to push current output all the way to the browser with a few caveats.
    * @see https://www.sitepoint.com/php-streaming-output-buffering-explained/
   **/
   static function glpi_flush() {
      header( 'Content-type: text/html; charset=utf-8' );
      if (function_exists("ob_flush")
          && (ob_get_length() !== false)) {
         ob_flush();
      }

      flush();
   }
sbraz commented 4 years ago

It doesn't change anything. It doesn't seem to be a very good idea to add headers in the body of the request anyway.