Feature: modules

[evgenykuzyakov]

Modules are pieces of code, that doesn't render a component. Instead they can return an arbitrary value that can be used by a widget or another module. They can be used to implement reusable functions, data processing functions, fetching logic, wrappers, helpers, etc. Pretty much libraries or modules that you usually import.

Example

Module to fetch an NFT image. It returns an object that contains a single function.

function getNftImageUrl(contractId, tokenId) {
  const nftMetadata = Near.view(contractId, "nft_metadata");
  const nftToken = Near.view(contractId, "nft_token", {
    token_id: tokenId,
  });

  let imageUrl = null;

  if (nftMetadata && nftToken) {
    let tokenMetadata = nftToken.metadata;
    let tokenMedia = tokenMetadata.media || "";

    const ownerId = nftToken.owner_id["Account"]
      ? nftToken.owner_id["Account"]
      : nftToken.owner_id;

    imageUrl =
      tokenMedia.startsWith("https://") ||
      tokenMedia.startsWith("http://") ||
      tokenMedia.startsWith("data:image")
        ? tokenMedia
        : nftMetadata.base_uri
        ? `${nftMetadata.base_uri}/${tokenMedia}`
        : tokenMedia.startsWith("Qm")
        ? `https://ipfs.near.social/ipfs/${tokenMedia}`
        : tokenMedia;

    if (
      !tokenMedia &&
      tokenMetadata.reference &&
      nftMetadata.base_uri === "https://arweave.net"
    ) {
      const res = fetch(`${nftMetadata.base_uri}/${tokenMetadata.reference}`);
      imageUrl = res.body.media;
    }
  }

  return imageUrl;
}

return { getNftImageUrl };

Then you can implement a widget that renders this NFT. You can start by importing the module.

const { getNftImageUrl } = require("mob.near/module/NftImageLoader");

const nft = props.nft ?? {
  contractId: props.contractId,
  tokenId: props.tokenId,
};
const contractId = nft.contractId;
const tokenId = nft.tokenId;
const className = props.className ?? "img-fluid";
const style = props.style;
const alt = props.alt;
const thumbnail = props.thumbnail;
const fallbackUrl = props.fallbackUrl;

const imageUrl = getNftImageUrl(contractId, tokenId) || fallbackUrl;

return (
  <img
    className={className}
    style={style}
    src={
      thumbnail ? `https://i.near.social/${thumbnail}/${imageUrl}` : imageUrl
    }
    alt={alt}
  />
);

[marcinbodnar]

@evgenykuzyakov I'm handling this issue. I took a closer look at the VM code and Widget component.

This issue contains two separate things:

1. Changes to Viewer so it will handle Module in the UI/UX - basically showing Module properly, separating it visually from Components, an additional option to create Module, etc.
2. Changes VM regarding Widget component to handle require - we will need to extend the acorn parser functionality - acorn supports plugins.

Changes to the viewer will not be problematic, I already prepare a basic implementation to see how it can be handled, and it shouldn't be any problems. Number 2. also shouldn't be a problem.

[evgenykuzyakov]

I'm thinking we should introduce require method to the VM. It should take the path to the source of the module and execute it in a child VM instance. The child VM doesn't not have to have all the functions of the parent VM, it probably should be configurable. E.g. no Near.call methods, but by default we can provide most of the functionality.

So the parent VM should fetch the source of the require module and feed it to the children VM. There are probably more nuances related to access and how it's implemented that I can't see right now

[marcinbodnar]

Thanks @evgenykuzyakov - I will take a closer look at it

[marcinbodnar]

@evgenykuzyakov It took me a while to figure out how VM is working. Most of the things are clear to me, but it's possible that I'm missing some nuances.

I prepared the solution: https://github.com/NearSocial/VM/pull/16 it's not final, but I would like to know what you think about the approach and also a few decisions that need to be made.

The way it's working:

• conts module = require(...) is parsed to VariableDeclaration, where the right side is a CallExpression, and as a result, it's trying to callFunction
• inside callFunction the callee is checked, and when it's require the vm.getModule method is called
• vm.getModule is getting module code, parse it and change type to Module
• after that, we are back in the executeStatement where value from executeExpression is checked, if it's a Module then parsed code is passed to executeStatement -

Recursive modules are working fine.

I'm not sure about a few things:

1. At this moment left side of VariableDeclaration is ignored, so it's not possible to use aliases or assign it to a different name - it needs to be resolved, for example, we can parse the module code, take a function declaration, remove the name, and assign to the left side of VariableDeclaration creating function expression.
2. I'm not sure if we want to introduce a new type Module - I added it, but we can handle it without it, not sure about that
3. How about two functions in one module, do we want to handle it?
4. Do we want to handle the arrow function as a module?

[evgenykuzyakov]

Sorry, missed your reply.

Current implementation works more like include from C, which injects the code into the current code. I might be a good option as well.

I was originally thinking that we would spawn a new VM instance when the require is called. This way the module doesn't have access to our state and it's somewhat static. The new VM should have an argument indicating it's a module, so it may disable some VM methods, e.g. we don't want it to have its own state and refresh. Ideally, it should only have functions, but it needs to be able to fetch other modules. If a method needs to have access to a state, it should get it from our VM.

So for now maybe we should rename require into VM.include or something like this.

[marcinbodnar]

@evgenykuzyakov Right, it's more like an include.

I finished preparing changes to Viewer to handle modules, and I will try another attempt to implement it more like JS require. I have an idea of how it can work.

[marcinbodnar]

@evgenykuzyakov This is what I came up with https://github.com/NearSocial/VM/pull/16:

• when the require function is called
• we are adding the require function to the component code
• this function wraps the module code with the module prop and returns module.

I did try with many different approaches. I tried to add a semi-global require function, that will fetch the module code every time it's called, but VM doesn't support the Function object so it's not able to fetch the code and create a function from the string.

I also tried a solution where require function is a method in the VM and it's fetching the code and saves it to the module object - every time the function that is already inside the module.exports is called we are just executing this function. But it also didn't work well with the VM limitations.

The current solution is the simplest one, and the one that it's working the best, basically identical like the JS require. The only main difference is that instead of one require function, we need to add a separate require function for every require call but that's because we are not able to simulate the global require function.