search

subsquid-labs / squid-substrate-template

Squid project template for Substrate-based chains
MIT License
33 stars 50 forks source link

Crustio Tutorial Error when running $npx squid-substrate-typegen typegen.json #15

Closed MaEdAlpha closed 2 years ago

MaEdAlpha commented 2 years ago

Hi there,

Going through your docs, and testing the Crustio tutorial. When I try to execute

npx squid-substrate-typegen typegen.json

I get the following error.

Error: Type MerchantLedger is not defined at OldTypeRegistry.buildNamedType (/home/ksm-node/Github/mu-squid/node_modules/@subsquid/substrate-metadata/lib/old/typeRegistry.js:284:15) at OldTypeRegistry.buildScaleType (/home/ksm-node/Github/mu-squid/node_modules/@subsquid/substrate-metadata/lib/old/typeRegistry.js:208:29) at OldTypeRegistry.use (/home/ksm-node/Github/mu-squid/node_modules/@subsquid/substrate-metadata/lib/old/typeRegistry.js:78:35) at /home/ksm-node/Github/mu-squid/node_modules/@subsquid/substrate-metadata/lib/chainDescription.js:529:42 at Array.forEach (<anonymous>) at /home/ksm-node/Github/mu-squid/node_modules/@subsquid/substrate-metadata/lib/chainDescription.js:496:31 at /home/ksm-node/Github/mu-squid/node_modules/@subsquid/substrate-metadata/lib/chainDescription.js:596:21 at Array.forEach (<anonymous>) at FromOld.forEachPallet (/home/ksm-node/Github/mu-squid/node_modules/@subsquid/substrate-metadata/lib/chainDescription.js:593:45) at FromOld.storage (/home/ksm-node/Github/mu-squid/node_modules/@subsquid/substrate-metadata/lib/chainDescription.js:492:14)

belopash commented 2 years ago

Hello, we have added built-in typesBundle for crust, so you don't need seperate typesBundle.json. Try to set "typesBundle": "crust" in your typegen.json file

UPD: It doesn't work too because crust have updated their types

belopash commented 2 years ago

Here are types that is missing in tutorial typesBundle.json "MerchantLedger": { "reward": "Balance", "collateral": "Balance" }, UsedInfo: { used_size: "u64", reported_group_count: "u32", groups: "BTreeMap<SworkerAnchor, bool>" },

RaekwonIII commented 2 years ago

Hey @MaEdAlpha , thanks for flagging this up. The template got updated with newer libraries and added functionality causes the error because the typesBundle is also out of date. Going to amend the documentation.

Edit: docs update, can you please try again and let us know? https://docs.subsquid.io/tutorial/create-a-simple-squid#generate-typescript-interfaces

MaEdAlpha commented 2 years ago

Hey @MaEdAlpha , thanks for flagging this up. The template got updated with newer libraries and added functionality causes the error because the typesBundle is also out of date. Going to amend the documentation.

Edit: docs update, can you please try again and let us know? https://docs.subsquid.io/tutorial/create-a-simple-squid#generate-typescript-interfaces

Definitely. Also, while going through your site, I noticed the Discord invite link no longer works. Is there still a Discord channel?

MaEdAlpha commented 2 years ago

Confirmed template in your tutorial docs works. Thanks, much appreciated. I'll go through the rest of the walkthrough and let you know if there are any issues. Cheers

RaekwonIII commented 2 years ago

Definitely. Also, while going through your site, I noticed the Discord invite link no longer works. Is there still a Discord channel?

Thanks, we are actually developing a completely new home page, I guess we completely forgot about that. Here is a Discord link, please join the server! https://discord.gg/dxR4wNgdjV

We are planning as of now to add a section dedicated to developers.

We also have a Telegram chat for Devs: https://t.me/HydraDevs As of now it's the quickest way to get support, but we plan to move this sort of interaction to Discord. 😉

MaEdAlpha commented 2 years ago

Hey, i'll post the processor.ts in my next comment. I modified for your docs (if you want) below. I think i'm missing some addons in my IDE (fresh VSCode install on Rpi4), so I added some changes for importing as well.

While I have you, just wanted to say thanks, this is an awesome tool. Mind if I can pick your brain later? Will attempt to setup my own squid; I want to access on the Kusama network the remark call function on the runtime module system.

Since it's not an event, I'm curious how to do this. I looked at the processor.ts library and see there are some preHooks()/postHooks() so I will play around with those.

Cheers

/////////////////////////////////////////////////////////////// Mods ///////////////////////////////////////////////////////////////

workReport.extrinisicId = extrinsic?.id; should be workReport.*extrinsicId* = extrinsic?.id;

import {Account, WorkReport, JoinGroup, StorageOrder} from "./model/gen" switched to import {Account} from "./model/generated/account.model"; import { WorkReport } from '.model/generated/workReport.model'; etc.. etc...

Also, {Store} from '@subsquid/substrate-processor'; was missing.

Lastly, I used the http://localhost:4350/graphql to make my queries. Every time I ran the server, it opened on that port. Docs say localhost:3000.

MaEdAlpha commented 2 years ago 
import {SubstrateProcessor, EventHandlerContext, Store} from '@subsquid/substrate-processor';
import  {Account} from './model/generated/account.model';

import  {WorkReport} from './model/generated/workReport.model';

import  {JoinGroup} from './model/generated/joinGroup.model';

import  {StorageOrder} from './model/generated/storageOrder.model';

import * as crustTypes from '@crustio/type-definitions'

const processor = new SubstrateProcessor('crust_example')
processor.setDataSource({
    archive: 'https://crust.indexer.gc.subsquid.io/v4/graphql',
    chain: 'wss://rpc-crust-mainnet.decoo.io'
});
processor.setBlockRange({from: 583000});
processor.setTypesBundle(crustTypes);
processor.addEventHandler('market.FileSuccess', fileSuccess);
processor.addEventHandler('swork.JoinGroupSuccess', joinGroupSuccess);
processor.addEventHandler('swork.WorksReportSuccess', workReportSuccess);

processor.run();

function stringifyArray(list: any[]): any[] {
  let listStr : any[] = [];
  list = list[0]
  for (let vec of list){
    for (let i = 0; i < vec.length; i++){
      vec[i] = String(vec[i]);
    }
    listStr.push(vec);
  }
  return listStr
}

async function joinGroupSuccess({
  store,
  event,
  block,
  extrinsic,
}: EventHandlerContext): Promise<void> {
  const memberId = String(event.params[0].value);
  const account = await getOrCreate(store, Account, memberId);

  const joinGroup = new JoinGroup();

  joinGroup.id = event.id;
  joinGroup.member = account;
  joinGroup.owner = String(event.params[1].value);

  joinGroup.blockHash = block.hash;
  joinGroup.blockNum = block.height;
  joinGroup.createdAt = new Date(block.timestamp);
  joinGroup.extrinsicId = extrinsic?.id;
  await store.save(account);
  await store.save(joinGroup);
}

async function fileSuccess({
  store,
  event,
  block,
  extrinsic,
}: EventHandlerContext): Promise<void> {
    const accountId = String(event.params[0].value);
    const account = await getOrCreate(store, Account, accountId);
    const storageOrder = new StorageOrder();
    storageOrder.id = event.id;
    storageOrder.account = account;
    storageOrder.fileCid =  String(event.params[1].value);
    storageOrder.blockHash = block.hash;
    storageOrder.blockNum = block.height;
    storageOrder.createdAt = new Date(block.timestamp);
    storageOrder.extrinsicId = extrinsic?.id;
    await store.save(account);
    await store.save(storageOrder);
}

async function workReportSuccess({
  store,
  event,
  block,
  extrinsic,
}: EventHandlerContext): Promise<void> {
  const accountId = String(event.params[0].value);
  const accountPr = getOrCreate(store, Account, accountId);
  const addedFilesObjPr = extrinsic?.args.find(arg => arg.name === "addedFiles");
  const deletedFilesObjPr = extrinsic?.args.find(arg => arg.name === "deletedFiles");

  const [account,addFObj,delFObj] = await Promise.all([accountPr,addedFilesObjPr,deletedFilesObjPr]);

  const workReport = new WorkReport();

  workReport.addedFiles = stringifyArray(Array(addFObj?.value))
  workReport.deletedFiles = stringifyArray(Array(delFObj?.value))
  if ((workReport.addedFiles.length > 0) || (workReport.deletedFiles.length > 0))
  { workReport.account = account;

  workReport.id = event.id;
  workReport.blockHash = block.hash;
  workReport.blockNum = block.height;
  workReport.createdAt = new Date(block.timestamp);
  workReport.extrinsicId = extrinsic?.id;

  await store.save(account);
  await store.save(workReport);
  }
}

async function getOrCreate<T extends {id: string}>(
    store: Store,
    entityConstructor: EntityConstructor<T>,
    id: string
): Promise<T> {
    let e = await store.get<T>(entityConstructor, {
        where: { id },
    })
    if (e == null) {
        e = new entityConstructor()
        e.id = id
    }
    return e
}
type EntityConstructor<T> = {
    new (...args: any[]): T
}
MaEdAlpha commented 2 years ago

Definitely. Also, while going through your site, I noticed the Discord invite link no longer works. Is there still a Discord channel?

Thanks, we are actually developing a completely new home page, I guess we completely forgot about that. Here is a Discord link, please join the server! https://discord.gg/dxR4wNgdjV

We are planning as of now to add a section dedicated to developers.

We also have a Telegram chat for Devs: https://t.me/HydraDevs As of now it's the quickest way to get support, but we plan to move this sort of interaction to Discord. 😉

will do, thanks RaekwonIII, I'll find you help in the Telegram chat. Cheers

RaekwonIII commented 2 years ago

Hey, i'll post the processor.ts in my next comment. I modified for your docs (if you want) below. I think i'm missing some addons in my IDE (fresh VSCode install on Rpi4), so I added some changes for importing as well.

I can't speak for certain about addons, but it should be a reload issue or something. Plus, if you try and import from "./model", even if intellisense gets mad, you should be able to compile.

  • In the docs, there is an extra i in the extrinsic field property in all the function examples (joinGroup, storageOrder, workReport).

Corrected that. It does not impact the functioning of the APUI, because it's consistent in the schema, in the model and in code usage, but I agree, it's bad 😆

Also, {Store} from '@subsquid/substrate-processor'; was missing.

It already is part of the squid-template but thanks for spotting the mistake in the final code! 🙌

Lastly, I used the http://localhost:4350/graphql to make my queries. Every time I ran the server, it opened on that port. Docs say localhost:3000.

Corrected that!

MaEdAlpha commented 2 years ago

Cheers, wasn't trying to be anal on the remarks. first time 'contributing' to something lol. Thanks, will close this. Thanks again!