162 update packages

[sroertgen]

I migrated to Gatsby v4 and updated some packages.

I also deleted some packages that weren't used anymore.

Gatsby v4 needs at least Node v14, so I updated that in the relevant parts (so this might also close #151 ) and noted that as well in the README.

Gatsby changed the order of execution in its build methods, so I had to move the createTypes call out of the schemaCustomization call, because it gets now called before the sourceNodes call. This has the side effect that the languages var is not yet filled with the apparent languages in the turtle files.

To overcome this issue I just moved the createTypes method to the sourceNodes call. I think this is the biggest change here. Apart from that its mostly just updating packages and a few syntax changes, where packages changed something.

I also changed the informations in the package.json file since it was still the default values from the gatsby starter default.

EDIT: I suggest that we delete all the dependabot branches and Pull requests if this PR goes through.

[sroertgen]

Hey @literarymachine ,

thanks for trying out! I can't reproduce your error. I did a fresh git clone of the repo switched to this branch and followed the instructions in the README:

git clone https://github.com/skohub-io/skohub-vocabs.git
cd skohub-vocabs/
git switch 162-update-packages
git status
nvm use 14
npm i
cp .env.example .env
cp test/data/systematik.ttl data/
npm run build

This built everything without an error. Maybe there was still some cache or anything somewhere? The GitHub Actions also went through without an error.

Regarding this warning:

warn Calling `createTypes` in the `sourceNodes` API is deprecated. Please use: `createSchemaCustomization`.

I noticed as well. And just like you I did not find any solution right now. What is your suggestion? Keep everything as is till we find one or update and live with the warning for now till we find a solution?

[acka47]

@literarymachine Will you be able to review this or should we just move forward with deploying this (after functional review of yourse)?

[dr0i]

Thx @sroertgen ! ... and sorry for the late response

It's building fine here, too. Reduces vulnerabilities from:

found 345 vulnerabilities (12 low, 225 moderate, 107 high, 1 critical)

to

30 vulnerabilities (12 moderate, 17 high, 1 critical)

As @acka47 noted in https://github.com/skohub-io/skohub.io/issues/16#issuecomment-1036083470 we are going to merge and deploy this. Will do this at Monday - for Murphy's Law and today's Friday.

[dr0i]

uh - interestingly, I got the same error experienced by @literarymachine when building at stage server. And now, mysteriously, I get the same error locally. Changed nothing. ?!.

[dr0i]

Problem still here. Did a new fresh build exactly as noted by @sroertgen at https://github.com/skohub-io/skohub-vocabs/pull/163#issuecomment-981454997. Has it something to do with the exact node and npm version? Mine are:

Now using node v14.17.6 (npm v7.0.15)

[sroertgen]

Hey @dr0i,

thanks for picking this up again :)

Did this error occur after you switched node versions?

I remember that at one point I also had this error. The reason seemed to be that gatsby still used some cached graphql definitions from somewhere and because the GraphQL spec changed between the versions. That threw the error.

Can you please try again, ensuring the following directories are removed:

• .cache

• public

• node_modules

• please also check if the dist/build-dir contains nothing but the index.html (don't know if it is much important, but that should give you a more or less clean repo)

Then please try again:

npm i npm run build

[dr0i]

@sroertgen did what you said, same result. Also, did totally new clones like you mentioned before. Also used different node/npm versions. It's not working here. Curious why it works for you AND also for github. Cannot peek into github actions logs (not available atm). Have you tried it yourself right from start cloning? What's your output for node --version; npm --version ?

[sroertgen]

Hey @dr0i ,

What an annoying issue!

Have you tried it yourself right from start cloning?

Yes, just before I wrote my last comment, I tested with a fresh clone. Just did it again today ^^

❯ node --version
v14.17.6
❯ npm --version
8.1.2

I added a commit to log the node and npm version on the github action. And now it also fails here. I guess I will have to take a look into this again.

[sroertgen]

This is really strange, using https://github.com/nektos/act to test github actions locally, it also runs through, though it uses the same node and npm versions GitHub Actions does here.

[sroertgen]

Hey @dr0i ,

I debugged a little bit and was finally to reproduce it (sometimes) on an Ubuntu VM. The error seems to be that sometimes (how weird is that?) the turtle files are not found. I added a log message to show found turtle files in the last commit.

Surprisingly now the GitHub Action went through again, though I just added a console.log(...). Now the Turtle file was found:

[
  {
    id: '4af430c2-28e5-590a-a7b1-c785242938d2',
    children: [],
    parent: null,
    internal: {
      contentDigest: '2de422071e0fbdd5208b7184ae0da40d',
      type: 'File',
      mediaType: 'text/turtle',
      description: 'File "data/systematik.ttl"',
      owner: 'gatsby-source-filesystem',
      counter: 37
    },
    sourceInstanceName: 'data',
    relativePath: 'systematik.ttl',
    extension: 'ttl',
    prettySize: '97 kB',
    modifiedTime: '2022-02-23T08:38:55.800Z',
    accessTime: '2022-02-23T08:38:55.800Z',
    changeTime: '2022-02-23T08:38:55.800Z',
    birthTime: '2022-02-23T08:38:55.800Z',
    root: '/',
    dir: '/home/runner/work/skohub-vocabs/skohub-vocabs/data',
    base: 'systematik.ttl',
    ext: '.ttl',
    name: 'systematik',
    absolutePath: '/home/runner/work/skohub-vocabs/skohub-vocabs/data/systematik.ttl',
    relativeDirectory: '',
    dev: 2049,
    mode: 33188,
    nlink: 1,
    uid: 1001,
    rdev: 0,
    blksize: 4096,
    ino: 1063432,
    size: 97012,
    blocks: [19](https://github.com/skohub-io/skohub-vocabs/runs/5300543154?check_suite_focus=true#step:9:19)2,
    atimeMs: 1645605535799.883,
    mtimeMs: 1645605535799.883,
    ctimeMs: 1645605535799.883,
    birthtimeMs: 1645605535799.883,
    atime: '[20](https://github.com/skohub-io/skohub-vocabs/runs/5300543154?check_suite_focus=true#step:9:20)[22](https://github.com/skohub-io/skohub-vocabs/runs/5300543154?check_suite_focus=true#step:9:22)-02-[23](https://github.com/skohub-io/skohub-vocabs/runs/5300543154?check_suite_focus=true#step:9:23)T08:[38](https://github.com/skohub-io/skohub-vocabs/runs/5300543154?check_suite_focus=true#step:9:38):[55](https://github.com/skohub-io/skohub-vocabs/runs/5300543154?check_suite_focus=true#step:9:55).800Z',
    mtime: '2022-02-23T08:38:55.800Z',
    ctime: '2022-02-23T08:38:55.800Z',
    birthtime: '2022-02-23T08:38:55.800Z'
  }
]

Will have to look at if there is something wrong with gatsby's gatsby-source-filesystem plugin. That will be fun...

[dr0i]

Smells like some asynchronous (or not flushed) file writing - like trying to check them before they are really available in the fs.

[sroertgen]

Hey @dr0i,

would you mind checking again? The only way I was able to remove this issue, was to "outsource" the node creation to the onPreBootstrap API. Now I'm searching recursively all the ttl-files in the data folder using fs and creating nodes out of them. This way also all the languages are found, so the GraphQL type definitions can be build correctly.

This might not be the most gatsby-like way, but I was not able to figure out, why gatsby sometimes find the ttl-files and sometimes not.

Regarding the mentioned deprecation warning of the createTypes-action: I suggest we live with it for now. In the official documentation it is still mentioned to be ok to call this action during sourceNodes: https://www.gatsbyjs.com/docs/reference/config-files/actions/#createTypes

Sorry that this took so long. But it was a hell to debug this. Fingers crossed that it works on all machines now.

PS: Since this PR is now a bit older, I updated some more packages

[dr0i]

Thx @sroertgen , this time it's running fine - tested on two machines. So, @literarymachine you could review, if you have time? @acka47 can you do funtional review?

[acka47]

I tried this out with my HCRT fork but it doesn't work. The SkoHub server responds with a 202 and a build URI but the build does not seem to really execute, see my last try at https://test.skohub.io/build/?id=7e6fc25d-8d0f-4258-99c6-f3e4ae0549e6

[dr0i]

Had to add "nvm use $nodeversion" to make the start script using proper nvm version. Seems to work now.

[acka47]

Can confirm, see https://test.skohub.io/build/?id=0f71bfa3-ee55-402b-89c3-32f30ffcaa6c#. +1

[dr0i]

Merged and deployed. Great and many thx again @sroertgen !

[dr0i]

This branch seems to bring up erratic problems reported in https://github.com/skohub-io/skohub-vocabs/issues/182#issuecomment-1106438520.