Making sure everything has been executed before closing the server.

[rjworks]

I have this in my onDisable()

$this->getMySQLProvider()->getConnector()->executeChange(Queries::UPDATE_ALL, [ "uuid" => $this->uuid, "name" => $this->name, "balance" => $this->balance, ]); $this->getMySQLProvider()->getConnector()->waitAll();

However, the data doesn't update when I close the sever properly with /stop. Is there a way to make sure every player's data has been updated before fully closing the server? I've even used the waitAll() function.

[SOF3]

Are you calling close() on DataConnector after waitAll?

[rjworks]

Are you calling close() on DataConnector after waitAll?

Yes.

[Ifera]

Well thats a you problem then

Oops my bad.. misread that

[rjworks]

it is?

[SOF3]

Well thats a you problem then man_facepalming

why would that be a problem?

[SOF3]

Please provide full steps to reproduce this issue, including the version you have been using.

waitAll blocks the thread until there are no pending queries in the pool, regardless whether it is in server startup phase or shutdown phase.

I am not sure if this issue has been caused by the Snooze update. @CortexPE any thoughts? Might the notifier consume the queries too early and schedule the next query asynchronously such that waitAll returns before the latter query gets scheduled?

[rjworks]

I am using version 3.3.0. Update: I dropped the table and now everything is updating fine on onDisable(); however, I do not know why the previous issue was happening and deleting the table in production to fix encounters like this doesn't sound very good to me.

     * PrisonPlayer constructor.
     *
     * @param string $uuid
     * @param string $name
     * @param int $balance
     */
    public function __construct(string $uuid, string $name, int $balance)
    {
        self::$instance = $this;
        $this->uuid = $uuid;
        $this->name = $name;
        $this->balance = $balance;
    }`
`public function update(): void
    {
        MainClass::getInstance()->getMySQLProvider()->getConnector()->executeChange(Queries::UPDATE_LAHAT, [
            "uuid" => $this->uuid,
            "name" => $this->name,
            "balance" => $this->balance
        ]);
        MainClass::getInstance()->getMySQLProvider()->getConnector()->waitAll();
    }
}```

```    /**
     * MemberPlayer constructor.
     *
     * @param \member\MainClass $plugin
     */
    public function __construct(MainClass $plugin)
    {
        $this->plugin = $plugin;
        Server::getInstance()->hasWhitelist(true);
        Server::getInstance()->getLogger()->warning("[Loading data starting] Whitelist is now on");
        $plugin->getMySQLProvider()->getConnector()->executeSelect(Queries::LOAD_ALL, [], function (array $rows): void {
            $count = 0;
            foreach ($rows as $row) {
                $this->players[$row["UUID"]] = new PrisonPlayer($row["UUID"], $row["NAME"],
                    $row["BALANCE"]);
                $count ++;
                $this->uuids[$row["UUID"]] = array("name" => $row["NAME"], "uuid" => $row["UUID"], "balance" => $row{"BALANCE"});
                $this->plugin->getLogger()->warning("Loading " . $count . " players");
            }`

        $this->plugin->getMySQLProvider()->getConnector()->waitAll();
    }```

`onEnable(){
 $this->player= new MemberPlayer($this);
}`
`onDisable(){
$this->update();
$this->close();
}`§

[SOF3]

I am not sure if this issue is really valid since your "dropping table fixes it" sounds suspicious. I'll leave this to @CortexPE since the snooze update very likely altered behaviour here.

[CortexPE]

it does work as intended... test code:

class Main extends PluginBase{
    private $db = null;
    private $fin = [];

    public function onEnable() : void{
        $this->db = libasynql::create($this, [
            "type" => "sqlite",
            "sqlite" => [
                "file" => "test.sqlite"
            ],
            "worker-limit" => 1
        ], ["sqlite" => "test.sql"]);
        for($i = 0; $i < 100000; $i++)$this->db->executeSelect("test", ["str" => (string)count($this->fin)], function(array $rows):void{
            $this->fin[] = array_values($rows)[0];
        });
        $this->getServer()->shutdown();
    }

    public function onDisable() : void{
        if(!($this->db instanceof DataConnector))return;
        $this->getLogger()->notice("Finished until " . count($this->fin));
        $t = microtime(true);
        $this->db->waitAll();
        $this->getLogger()->notice("Unfinished query time: " . (microtime(true) - $t) . "s");
        $this->getLogger()->notice("Final count " . count($this->fin));
        $this->db->close();
    }
}

test.sql:

-- #!sqlite

-- #{ test
--    :str string
SELECT hex(:str);
-- #}

output:

[20:22:47] [Server thread/INFO]: Enabling testPlugin v0.0.1
... unrelated ...
[20:22:50] [Server thread/DEBUG]: Disabling all plugins
[20:22:50] [Server thread/INFO]: Disabling testPlugin v0.0.1
[20:22:50] [Server thread/NOTICE]: [testPlugin] Finished until 0
[20:22:58] [Server thread/NOTICE]: [testPlugin] Unfinished query time: 7.4843888282776s
[20:22:58] [Server thread/NOTICE]: [testPlugin] Final count 100000

the plugin shutdown is delayed by 7.4 seconds as you can see on the console logger, after executing 100,000 queries and forcibly waiting for all of them to finish.

[CortexPE]

since he did mention

I dropped the table and now everything is updating fine on onDisable(); however, I do not know why the previous issue was happening and deleting the table in production to fix encounters like this doesn't sound very good to me.

I think it may be a schema issue? or maybe an un-noticed faulty constraint that possibly always evaluate false causing the database itself to not update at all.

The chances are pretty slim but it is possible.

[SOF3]

Closing issue as @rjworks is unable to provide a full reducible example. Please comment below if any further information can be provided.

[rjworks]

It is happing again. What information do I need to give you to better understand this possible issue?

[SOF3]

It is happing again. What information do I need to give you to better understand this possible issue?

A fully reproducible example. Ideally a plugin + data folder (or MySQL data folder) setup such that starting from this state reproduces the issue every time.

[rjworks]

mysql data folder? my mysql.sql config?

[SOF3]

No. I want all the relevant files, including those data files in MySQL's data directory, usually in /var/lib/mysql or something when the server starts, and a whole plugin that can reproduce the issue.

[rjworks]

i'm using sqlite now and still this issue happens. Is there any other way to reproduce other than giving the whole plugin..

[SOF3]

This is the question I am asking you...

[rjworks]

is it possible that many rows updating is causing this

[SOF3]

Test it yourself. I do not have your code nor your database, and I have never reproduced this. You are the most likely person to be able to find a minimal reproducible example.

[rjworks]

it does work as intended... test code:

class Main extends PluginBase{
  private $db = null;
  private $fin = [];

  public function onEnable() : void{
      $this->db = libasynql::create($this, [
          "type" => "sqlite",
          "sqlite" => [
              "file" => "test.sqlite"
          ],
          "worker-limit" => 1
      ], ["sqlite" => "test.sql"]);
      for($i = 0; $i < 100000; $i++)$this->db->executeSelect("test", ["str" => (string)count($this->fin)], function(array $rows):void{
          $this->fin[] = array_values($rows)[0];
      });
      $this->getServer()->shutdown();
  }

  public function onDisable() : void{
      if(!($this->db instanceof DataConnector))return;
      $this->getLogger()->notice("Finished until " . count($this->fin));
      $t = microtime(true);
      $this->db->waitAll();
      $this->getLogger()->notice("Unfinished query time: " . (microtime(true) - $t) . "s");
      $this->getLogger()->notice("Final count " . count($this->fin));
      $this->db->close();
  }
}

test.sql:

-- #!sqlite

-- #{ test
--    :str string
SELECT hex(:str);
-- #}

output:

[20:22:47] [Server thread/INFO]: Enabling testPlugin v0.0.1
... unrelated ...
[20:22:50] [Server thread/DEBUG]: Disabling all plugins
[20:22:50] [Server thread/INFO]: Disabling testPlugin v0.0.1
[20:22:50] [Server thread/NOTICE]: [testPlugin] Finished until 0
[20:22:58] [Server thread/NOTICE]: [testPlugin] Unfinished query time: 7.4843888282776s
[20:22:58] [Server thread/NOTICE]: [testPlugin] Final count 100000

the plugin shutdown is delayed by 7.4 seconds as you can see on the console logger, after executing 100,000 queries and forcibly waiting for all of them to finish.

which build is this???

[CortexPE]

Latest

On Sat, Jul 25, 2020, 10:26 Arjay notifications@github.com wrote:

it does work as intended... test code:

class Main extends PluginBase{ private $db = null; private $fin = [];

public function onEnable() : void{ $this->db = libasynql::create($this, [ "type" => "sqlite", "sqlite" => [ "file" => "test.sqlite" ], "worker-limit" => 1 ], ["sqlite" => "test.sql"]); for($i = 0; $i < 100000; $i++)$this->db->executeSelect("test", ["str" => (string)count($this->fin)], function(array $rows):void{ $this->fin[] = array_values($rows)[0]; }); $this->getServer()->shutdown(); }

public function onDisable() : void{ if(!($this->db instanceof DataConnector))return; $this->getLogger()->notice("Finished until " . count($this->fin)); $t = microtime(true); $this->db->waitAll(); $this->getLogger()->notice("Unfinished query time: " . (microtime(true) - $t) . "s"); $this->getLogger()->notice("Final count " . count($this->fin)); $this->db->close(); } }

test.sql:

-- #!sqlite -- #{ test-- :str stringSELECT hex(:str);-- #}

output:

[20:22:47] [Server thread/INFO]: Enabling testPlugin v0.0.1 ... unrelated ... [20:22:50] [Server thread/DEBUG]: Disabling all plugins [20:22:50] [Server thread/INFO]: Disabling testPlugin v0.0.1 [20:22:50] [Server thread/NOTICE]: [testPlugin] Finished until 0 [20:22:58] [Server thread/NOTICE]: [testPlugin] Unfinished query time: 7.4843888282776s [20:22:58] [Server thread/NOTICE]: [testPlugin] Final count 100000

the plugin shutdown is delayed by 7.4 seconds as you can see on the console logger, after executing 100,000 queries and forcibly waiting for all of them to finish.

which build is this???

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/poggit/libasynql/issues/36#issuecomment-663795941, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEHQQFHPD7ODZHDJGTFOFQ3R5I7FHANCNFSM4N5WF7OA .

[rjworks]

Latest … On Sat, Jul 25, 2020, 10:26 Arjay @.***> wrote: it does work as intended... test code: class Main extends PluginBase{ private $db = null; private $fin = []; public function onEnable() : void{ $this->db = libasynql::create($this, [ "type" => "sqlite", "sqlite" => [ "file" => "test.sqlite" ], "worker-limit" => 1 ], ["sqlite" => "test.sql"]); for($i = 0; $i < 100000; $i++)$this->db->executeSelect("test", ["str" => (string)count($this->fin)], function(array $rows):void{ $this->fin[] = array_values($rows)[0]; }); $this->getServer()->shutdown(); } public function onDisable() : void{ if(!($this->db instanceof DataConnector))return; $this->getLogger()->notice("Finished until " . count($this->fin)); $t = microtime(true); $this->db->waitAll(); $this->getLogger()->notice("Unfinished query time: " . (microtime(true) - $t) . "s"); $this->getLogger()->notice("Final count " . count($this->fin)); $this->db->close(); } } test.sql: -- #!sqlite -- #{ test-- :str stringSELECT hex(:str);-- #} output: [20:22:47] [Server thread/INFO]: Enabling testPlugin v0.0.1 ... unrelated ... [20:22:50] [Server thread/DEBUG]: Disabling all plugins [20:22:50] [Server thread/INFO]: Disabling testPlugin v0.0.1 [20:22:50] [Server thread/NOTICE]: [testPlugin] Finished until 0 [20:22:58] [Server thread/NOTICE]: [testPlugin] Unfinished query time: 7.4843888282776s [20:22:58] [Server thread/NOTICE]: [testPlugin] Final count 100000 the plugin shutdown is delayed by 7.4 seconds as you can see on the console logger, after executing 100,000 queries and forcibly waiting for all of them to finish. which build is this??? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#36 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEHQQFHPD7ODZHDJGTFOFQ3R5I7FHANCNFSM4N5WF7OA .

135 dev build??

[CortexPE]

I just clone the repo

On Mon, Jul 27, 2020, 06:37 Arjay notifications@github.com wrote:

Latest … <#m3561115580394982117> On Sat, Jul 25, 2020, 10:26 Arjay @.***> wrote: it does work as intended... test code: class Main extends PluginBase{ private $db = null; private $fin = []; public function onEnable() : void{ $this->db = libasynql::create($this, [ "type" => "sqlite", "sqlite" => [ "file" => "test.sqlite" ], "worker-limit" => 1 ], ["sqlite" => "test.sql"]); for($i = 0; $i < 100000; $i++)$this->db->executeSelect("test", ["str" => (string)count($this->fin)], function(array $rows):void{ $this->fin[] = array_values($rows)[0]; }); $this->getServer()->shutdown(); } public function onDisable() : void{ if(!($this->db instanceof DataConnector))return; $this->getLogger()->notice("Finished until " . count($this->fin)); $t = microtime(true); $this->db->waitAll(); $this->getLogger()->notice("Unfinished query time: " . (microtime(true) - $t) . "s"); $this->getLogger()->notice("Final count " . count($this->fin)); $this->db->close(); } } test.sql: -- #!sqlite -- #{ test-- :str stringSELECT hex(:str);-- #} output: [20:22:47] [Server thread/INFO]: Enabling testPlugin v0.0.1 ... unrelated ... [20:22:50] [Server thread/DEBUG]: Disabling all plugins [20:22:50] [Server thread/INFO]: Disabling testPlugin v0.0.1 [20:22:50] [Server thread/NOTICE]: [testPlugin] Finished until 0 [20:22:58] [Server thread/NOTICE]: [testPlugin] Unfinished query time: 7.4843888282776s [20:22:58] [Server thread/NOTICE]: [testPlugin] Final count 100000 the plugin shutdown is delayed by 7.4 seconds as you can see on the console logger, after executing 100,000 queries and forcibly waiting for all of them to finish. which build is this??? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#36 (comment) https://github.com/poggit/libasynql/issues/36#issuecomment-663795941>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEHQQFHPD7ODZHDJGTFOFQ3R5I7FHANCNFSM4N5WF7OA .

135 dev build??

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/poggit/libasynql/issues/36#issuecomment-664049114, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEHQQFGZODKU7E2DVHXVQ33R5SV3RANCNFSM4N5WF7OA .

[rjworks]

should waitAll() only be used in onDisable since it will make the main thread go to sleep or would it be safe to use it every 30 minutes for auto save?

[SOF3]

should waitAll() only be used in onDisable since it will make the main thread go to sleep or would it be safe to use it every 30 minutes for auto save?

There is no point blocking the main thread if the main thread isn't immediately doing something destructive to your queries.

[rjworks]

let’s say I have 30plus queries happening every 30 minutes for an auto save, i don’t have to use waitAll() to make sure everyone query gets executed and saved fine?

[SOF3]

The only thing that waitAll() does is to block the main thread before all queries have been dispatched. It does nothing to guarantee that all queries are executed. In onDisable, waitAll merely forces the server to wait for all queries to execute before continuing the shutdown, which happens on the main thread.