Concurrency issue causing ArrayIndexOutOfBoundsException

[devbaraus]

Every time I initialize the server after a few seconds it isn't accessible anymore. If I reload the plugin it comes back but then it turns off again.

[Dantevg]

That's weird, is there anything in the console? Could you share your config?

[devbaraus]

config.yml

# Configuration file for WebStats
# See https://github.com/Dantevg/WebStats/wiki/Config-file for more info

# Port to use for the web server. Make sure that this port corresponds
# to the one set in the web page js script.
port: 25566

# A list of tables to show on the webpage. Default is to show a single unnamed
# table which contains all columns, sorted by player name in ascending order.
tables:
  - # The name of the table. This will be displayed above it on the webpage.
    # Leave empty for no title / name.
    name:

    # Only these columns will appear, in the order specified.
    # When commented out, all columns will be displayed, in alphabetical order (default).
    # The 'Player' column will always be displayed, as the first column.
    columns: ['Health', 'Max Health', 'Skills Power']

    # Which column to sort by
    sort-column: Player

    # Whether to sort ascending (smallest first) or descending (largest first)
    sort-direction: ascending

# These columns will not be displayed on individual player's rows, but on a
# separate 'Server' row.
server-columns: []

# Whether to store IP to player name mapping in 'ip-to-names.yml',
# used for matching your IP to your player name
store-player-ips: true

# Whether to also serve the web page from within the plugin. (not only the stats)
# Disable this when you want to use your own external web server, enable this
# when you do not have one.
serve-webpage: false

# When serve-webpage is enabled, this is the tab title of the webpage.
#webpage-title: "Web Stats"

# Scoreboard objectives to use. When '*' is present, uses all objectives.
# (make sure to enclose the '*' in quotes)
objectives:
  - 'ts_Deaths'

# Database credentials for MySQL database connectivity and placeholder storage
#database:
#  hostname: localhost
#  username: DATABASE USERNAME
#  password: DATABASE PASSWORD
#  config: # uncomment from here to use MySQL database connectivity
#    - database: DATABASE NAME
#      table: TABLE NAME
#      convert: # Some examples, refer to documentation for explanations
#        - [rename, from-column, to-column]
#        - [json, column]
#        - [key-value, key-column, value-column]
#        - [uuid, column]

# Configuration for the PlaceholderAPI connectivity (uncomment to use)
placeholders:
 '%aureliumskills_power%': Skills Power
 '%player_max_health%': Max Health
 '%player_health%': Health

# Set to true to store placeholders in a file, for when players go offline. (uncomment to use)
# Mutually exclusive with `store-placeholders-database`
store-placeholders-in-file: true

# Alternatively, the database name to store placeholders in. (uncomment to use)
# Make sure to specify the database hostname, username and password above
# Mutually exclusive with `store-placeholders-in-file`
#store-placeholders-database: WebStats_placeholders

# Configuration for the Discord webhook function (uncomment to use)
# Make sure to set the webhook URL
#discord-webhook:
#  # The webhook URL. Get this from Discord
#  url: ""
#  
#  # The update interval, in minutes
#  update-interval: 5
#  
#  # Show the top n rows/players
#  display-count: 10
#  
#  # Message title, text to be displayed before the embeds
#  title:
#  
#  # Each item in this list is one embed
#  embeds:
#    - # The title of the embed. Leave empty for no title
#      title:
#      
#      # Which column to sort by
#      sort-column: Player
#      
#      # Whether to sort ascending (smallest first) or descending (largest first)
#      sort-direction: ascending
#      
#      # Which columns to display. Defaults to using the columns defined at the
#      # top of this file in a single embed, or all columns otherwise.
#      # Note that Discord displays max 3 columns next to each other (including
#      # the "Player" column which is always present)
#      columns: [Deaths]
#      
#      # A further example
#    - title: Best miners
#      sort-column: Mine All Diamond
#      sort-direction: ascending
#      columns: [Mine All Diamond, Mine Ancient Debris]

# Whether to save placeholders before any plugins are disabled on server shutdown
# This is a fix for error "java.lang.IllegalStateException: zip file closed."
# You can leave this off if you don't experience this error.
save-placeholders-on-plugin-disable: false

[devbaraus]

It sends nothing to the console, only when I reload the plugin. It works for some time and stops.

[5:32:07 PM INFO] [WebStats] Reload: disabling plugin
[5:32:07 PM INFO] [WebStats] Stopping web server
[5:32:07 PM INFO] [WebStats] Saved placeholders
[5:32:07 PM INFO] [WebStats] Saved player ip-to-names mappings
[5:32:07 PM INFO] [WebStats] Reload: re-enabling plugin
[5:32:07 PM INFO] [WebStats] Loaded 20 ip-to-names mappings
[5:32:07 PM INFO] [WebStats] Enabling scoreboard source
[5:32:07 PM INFO] [WebStats] Enabling placeholder source
[5:32:07 PM INFO] [WebStats] Enabling placeholder storage
[5:32:08 PM INFO] [WebStats] Web server started on port 25566
[5:32:08 PM INFO] [WebStats] Reload complete

[Dantevg]

When the web server crashes it somehow never outputs anything to the console, so that's probably what happens here too. If that is indeed what's happening, you should be able to do /webstats export and see all stats in plugins/WebStats/stats.csv. (that means that everything behind the webserver is still fully working)

I'll see if I can get the web server to spit out its exceptions so that I can see what is wrong.

[Dantevg]

While getting all webserver exceptions to be logged I also fixed a mistake in that code, it may now even work for you already.

Here's the latest build, could you post whether it gives an error? WebStats-1.8.3-SNAPSHOT+mc1.18+.jar.zip

[devbaraus]

Now it throws an error.

https://sentry.io/share/issue/2328d1ffb7474f878502e193e32acfc8/

[Dantevg]

Based on looking at similar issues on the web (particularly this one: https://github.com/montlikadani/TabList/issues/304), it seems to be a concurrency issue. I've changed the http request code to always gather the stats on the main thread, hopefully it works correctly now: WebStats-1.8.3-SNAPSHOT+mc1.18+.jar.zip

[devbaraus]

Well it still doesn't work sometimes.

I was wondering if you could make so the placeholder.csv was updated from 5 to 5 minutes. Then I could read the file externally.

[Dantevg]

That is weird, does it still give the same error or another one?

I've compiled a new version, you can set save-placeholders-interval: 5 in the config to let it save the placeholders every 5 minutes. Here's the jar: WebStats-1.8.3-SNAPSHOT+mc1.18+.jar.zip

[devbaraus]

It gives no error. I don't know whats happening.

One question, what's the difference between placeholders.csv and stats.csv? Maybe it's better to export from 5 to 5 using stats.csv.

[devbaraus]

Idk why but after I changed some placeholders and reloaded the plugin my placeholders.csv got messed, but stats.csv is fine.

[Dantevg]

• placeholders.csv contains all placeholders that you specified under placeholders: in config.yml. It is used for remembering placeholders when players go offline (since many placeholders only work on online players). The file normally gets the latest placeholders written to it every time the server closes.
• stats.csv contains only the stats you define in the header of the csv file. It gets the stats appended to it every time you execute /webstats export. It's meant for creating nice graphs of the stats over time. (I've personally set it up to run the export command every night)

I see now in your screenshot that old placeholders that are no longer in config.yml remain in placeholders.csv, I'll fix that soon. You probably renamed the placeholder columns to remove the starting capital, I guess?

So at the moment, you cannot use stats.csv as a normal output since it will have the stats appended. I can make a new command (something like /webstats export overwrite) to let the file contain only the current stats. Do you have a way to schedule commands automatically (so I only need to add the command), or so you need the plugin to have an auto scheduling function for the export?

Weird that it stops working without any error. Is it still the exact same behaviour as initially?

[devbaraus]

Well, I don't have a way to schedule commands, if you could change that save-placeholders-interval: 5 to export-interval=5 or something like it I'd appreciate.

[Dantevg]

I'll see what I can do! In the meantime, could you say if the behaviour is still the same as before the attempted fix? (apart from the errors that disappeared)

[devbaraus]

So, it doesn't log any error, just sometimes the page doesn't work or takes too long to load. With the stats.csv I made an API using fastapi https://mine.devbaraus.ddns.net/stats/?sort=-money&query=online==True

[Dantevg]

I just got around to implementing your suggestion. You'll want to set these values in config.yml (and save-placeholders-interval is gone):

# Whether to keep old statistics in stats.csv when executing '/webstats export'.
# Disabling this will override older saved statistics in stats.csv when exporting.
export-cumulative: false

# The interval in which to export all stats to stats.csv, in minutes.
# Set to 0 to disable (default).
export-interval: 5

I also did some thinking about the source of the bug, but without any error messages it's really hard to figure out. Regardless, here's the new jar: WebStats-1.8.3-SNAPSHOT+mc1.18+.jar.zip

Could you let me know if it works?