Bundle failure when attempting to fetch (Scope of CustomRealms plugins)

[mattmazzola]

I don't think this is a bug with the core library but misconfiguration of consuming package that caused an error during attempt to bundle. This brought up some other questions about scope of CustomRealms. I am new to Minecraft and plugin development but have experience using TypeScript so I was interested in this library for developing a plugin although I wanted to first confirm what the recommend approach is for my goal.

I want a plugin to communicate with an external service over HTTP as seen in this diagram:

1	2	3	4
Player	Minecraft Client	PaperMC Server with CustomRealms Plugin	External Service

This would allow the plugin to be much lighter and stay focused on Minecraft API where other logic can go in the service, can be developed independently, and process work loads that we wouldn't want running directly in the plugin code and same server process.

When the user types some command, say /view, the plugin registered callback would be executed which sends a request to external service. The plugin would translate the response data into Minecraft commands which the display data to the Minecraft user.

I was attempting to use fetch but came across this error during bundling.

============================================================
Bundling JavaScript code using Webpack
============================================================
assets by status 184 KiB [cached] 2 assets
orphan modules 322 KiB [orphan] 125 modules
runtime modules 6.73 KiB 9 modules
cacheable modules 531 KiB
  modules with errors 429 bytes [errors]
    optional modules 78 bytes [optional]
      node:process 39 bytes [optional] [built] [code generated] [1 error]
      node:stream/web 39 bytes [optional] [built] [code generated] [1 error]
    9 modules
  modules by path ./ 530 KiB
    modules by path ./node_modules/fetch-blob/ 9.74 KiB 3 modules
    ./src/main.ts + 120 modules 321 KiB [built] [code generated]
    ./node_modules/formdata-polyfill/esm.min.js 2.32 KiB [built] [code generated]
    ./node_modules/node-fetch/src/utils/multipart-parser.js + 1 modules 12.1 KiB [built] [code generated]
    ./node_modules/web-streams-polyfill/dist/ponyfill.es2018.js 185 KiB [built] [code generated]

WARNING in ./node_modules/fetch-blob/streams.cjs 29:19-36
Module not found: Error: Can't resolve 'buffer' in 'E:\repos\minecraft-plugin-01\node_modules\fetch-blob'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
        - add a fallback 'resolve.fallback: { "buffer": require.resolve("buffer/") }'
        - install 'buffer'
If you don't want to include a polyfill, you can use an empty module like this:
        resolve.fallback: { "buffer": false }
 @ ./node_modules/fetch-blob/index.js 6:0-22
 @ ./node_modules/node-fetch/src/body.js 11:0-30 135:13-17
 @ ./node_modules/node-fetch/src/index.js 15:0-47 170:13-18 294:2-15
 @ ./src/main.ts 3:0-31 46:37-42

There is the surface question of: What would you recommend to solve this particular bundle error?

I assume I need to manually include a polyfill for Buffer. I'm not familiar with the "resolve.fallback" or what to edit. Perhaps this is in the package.json of the "node-fetch" module?

As I understand. CustomRealms is a wrapper around existing Minecraft APIs so the core code can only leverage what is exposed by the bukkit runtime. I thought sending HTTP requests was relatively trivial (Maybe this is bad assumption?) and worried even if I managed to solve this problem I would risk hitting more issues like this as I continue development and wondered about alternatives. I thought I should ask for recommendations

To alleviate this concern do you think I would be better off with an alternate plugin development library or is this something CustomRealms plans to increase support for?

[AL1L]

Unfortunately, CustomRealms does not currently support node modules such at node-fetch, buffer, crypto, http, events, etc, but it's something we want to add. The JavaScript engine that CR is running on is the pure V8 engine with nothing added other than Bukkit interfaces. I think it'd be a great idea to add the Fetch API into CustomRealms.

This issue is a better fit for https://github.com/customrealms/bukkit-runtime, @connerdouglass can move it.

I do like the idea of moving to a NodeJS runtime rather than just V8, but not sure how the integration between Java and it would work if it would at all.

[AL1L]

A "workaround" of sorts is to use the Java apis and Java.resolve('com.package.Name) in CR to make HTTP requests. Although that'll be pretty difficult and pretty clunky.

[connerdouglass]

@mattmazzola Yeah great question, and I think @AL1L covered most of my thoughts. For your plugin concept, you're currently probably better off writing it in Java directly (sadly). CustomRealms' JavaScript runtime is missing a lot of the built-in NodeJS functionality that a lot of libraries on NPM depend on.

It's a pretty high priority for us to add it, but it will take some time to complete!

I just opened this issue to track our progress toward NodeJS compatibility: https://github.com/customrealms/bukkit-runtime/issues/6

[AL1L]

Here's a really bad workaround to make GET requests

const URL = Java.resolve('java.net.URL');
const BufferedReader = Java.resolve('java.io.BufferedReader');
const InputStreamReader = Java.resolve('java.io.InputStreamReader');

async function getRequest(uri: string) {
    const url = new URL(uri);
    const con = url.openConnection();

    // These dont work...cant figure out why
    // con.setRequestMethod('GET');
    // con.setRequestProperty('User-Agent', 'CustomRealms/0.2.0');

    const reader = new BufferedReader(
        new InputStreamReader(con.getInputStream()));

    let inputLine;
    let content = [];
    while ((inputLine = reader.readLine()) != null) {
        content.push(inputLine);
    }
    reader.close();

    return content.join('\n');
}

Some stuff could be added to check and return the status code, but it does the job. Just might spam the console if anything goes wrong (404 errors break it).

[mattmazzola]

The JavaScript engine that CR is running on is the pure V8 engine with nothing added other than Bukkit interfaces

For your plugin concept, you're currently probably better off writing it in Java directly (sadly). CustomRealms' JavaScript runtime is missing a lot of the built-in NodeJS functionality that a lot of libraries on NPM depend on.

Ok! Now that you mention it, I do remember seeing a video mention that console.log had to be manually implemented although I didn't connect that meant node packages wouldn't function. Makes sense now.

Thanks for the replies clarifying and for the honest recommendations. Given the popularity of node I do think it's great that you are considering using it as runtime although I imagine that is a LOT of work. I think our team will start development soon so we will likely use Java as you suggested so we don't have to worry about timing of features and such.

It looks like a great project to make Minecraft plugin development more accessible. I hope it goes well. 👍