Deploy subgraph on protected Graph and IPFS endpoints

[air3ijai]

Hello, we are started to use Graph node and IPFS in some of our projects which are deployed in the cloud and behind ALB.

Architecture

Subgraph deployment is usually done via GitHub Actions

                 Graph node
               /
GitHub --> ALB
               \
                 IPFS

Deploy

In our pipeline we have two steps which should be protected using authentication

graph create ...
graph deploy ...

Pipeline

name: Build and deploy Subgraph

on:
  push:
    tags:
      - 'prod-v*'
  workflow_dispatch:

jobs:
  run:
    runs-on: ubuntu-latest
    env:
      INDEX_NODE_ENDPOINT: index.domain.tld
      IPFS_ENDPOINT: ipfs.domain.tld
      SUBGRAPH_NAME: prod/subgraph
    steps:
        - uses: actions/checkout@v2
        - uses: actions/setup-node@v2
          with:
              node-version: 12

        - name: Install graph-cli
          run: yarn global add @graphprotocol/graph-cli@v0.23.2

        - name: Install dependencies
          run: yarn install

        - name: Codegen
          run: yarn codegen

        - name: Build
          run: yarn build

        - name: Graph create
          run: graph create --node {{ env.INDEX_NODE_ENDPOINT }}

        - name: Graph deploy
          run: |
               graph deploy ${{ env.SUBGRAPH_NAME }} \
                 --version-label $GITHUB_REF_NAME \
                 --node ${{ env.INDEX_NODE_ENDPOINT }} \
                 --ipfs ${{ env.IPFS_ENDPOINT }}

Authentication

I found the following PR - Make graph auth and --access-token useful for IPFS as well #184 but not sure if it is related just for hosted service.

The easiest way for us is to protect ALB using custom HTTP header, but maybe there some other options?

Workaround

We considering to use a very simple workaround, but it anyway will require a public access to IPFS node and curl usage

# Build and upload build results to an IPFS node
yarn build --ipfs ${{ env.IPFS_ENDPOINT }}" >build.log

# Get IPFS_HASH
echo "IPFS_HASH=$(awk -F': ' '/Build completed/ {print $2}' build.log)" >> $GITHUB_ENV

# Graph create
curl -X POST ${{ env.INDEX_NODE_ENDPOINT }} \
  -H 'header: password' \
  -H 'Content-Type: application/json' \
  -d '{"jsonrpc": "2.0", "id": "2", "method": "subgraph_create", "params": {"name": "${{ env.SUBGRAPH_NAME }}"}}'

# Graph deploy
curl -X POST ${{ env.INDEX_NODE_ENDPOINT }} \
  -H 'header: password' \
  -H 'Content-Type: application/json' \
  -d '{"jsonrpc": "2.0", "id": "2", "method": "subgraph_deploy", "params": {"name": "${{ env.SUBGRAPH_NAME }}", "ipfs_hash": "${{ env.IPFS_HASH }}"}}'

Questions

1. Can we send a custom HTTP header when we access IPFS and Graph node using graph-cli when create and deploy subgraph?
2. Is there a way to use access-token on self-deployed IPFS and Graph node?
3. If we can use access-token how it can be configured on IPFS and Graph node side?

[azf20]

Hi! i think access-token is not currently supported in a self-deployed graph-node & ipfs setup

[air3ijai]

Maybe something about basic authentication - Added basic url auth support #629?

We can implement basic auth on ALB side but can we specify it via graph-cli for create and deploy subgraph?

[air3ijai]

@juanmardefago, maybe you can provide any details about your PR?

[air3ijai]

In-url passwords doesn't work - graphprotocol/graph-cli@0.28.1 (example-subgraph)

yarn graph build --ipfs https://user:password@domain.com

ALB return 404 for non-authenticated requests. Same request with curl return 404 from IPFS endpoint.

✔ Compile subgraph
  Copy subgraph file build/schema.graphql
  Write subgraph file build/Factory/abis/BFactory.json
  Write subgraph file build/DTFactory/abis/DTFactory.json
  Write subgraph file build/Metadata/abis/Metadata.json
  Write subgraph file build/FixedRateExchange/abis/FixedRateExchange.json
  Write subgraph file build/FixedRateExchange/abis/ERC20.json
  Write subgraph file build/FixedRateExchange/abis/ERC20SymbolBytes.json
  Write subgraph file build/FixedRateExchange/abis/ERC20NameBytes.json
  Write subgraph file build/Dispenser/abis/Dispenser.json
  Write subgraph file build/Pool/abis/BPool.json
  Write subgraph file build/Pool/abis/BToken.json
  Write subgraph file build/Pool/abis/ERC20.json
  Write subgraph file build/Pool/abis/ERC20SymbolBytes.json
  Write subgraph file build/Pool/abis/ERC20NameBytes.json
  Write subgraph file build/DataToken/abis/DataTokenTemplate.json
  Write subgraph manifest build/subgraph.yaml
✔ Write compiled subgraph to build/
  Add file to IPFS build/schema.graphql
✖ Failed to upload subgraph to IPFS: Failed to upload file to IPFS: Server responded with 404
Error: Failed to upload file to IPFS: Server responded with 404
    at Compiler._uploadToIPFS (/home/ubuntu/subgraph/node_modules/@graphprotocol/graph-cli/src/compiler.js:619:13)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

[air3ijai]

We use the following workaround

1. Enable basic authentication on ALB for IPFS and Index node

2. Disable IPFS endpoint authentication on ALB Ingres Controller

3. Build subgraph and upload it to the IPFS - get Deployment ID hash

4. Deploy subgraph to the Index node using curl with basic authentication

5. Enable IPFS endpoint authentication on ALB Ingres Controller

It means that IPFS endpoint become unprotected during deploy and just for ~ 4 minutes in case of 1 subgraph.

[air3ijai]

Recently we tested two PR's

1. feat: add headers flag to the deploy command #905
2. fix: basic auth #927

Accordingly to the tests we did:

1. Inline basic authentication works only for Index node
2. Sending headers flag works only with IPFS node

But we successfully combined them

Create Subgraph

npm exec -- graph create subgraph-test \
  --node https://used:pass@rinkeby.index.domain.tld

Created subgraph: subgraph-test

Deploy Subgraph

npm exec -- graph deploy subgraph-test subgraph.yaml \
  --version-label subgraph-test \
  --headers "{\"Authorization\": \"Basic (username:password)to-base64\"}" \
  --ipfs https://ipfs.domain.tld \
  --node https://user:pass@rinkeby.index.domain.tld

✔ Upload subgraph to IPFS

Build completed: QmPL3wQiwtuTgN8kZkEuTMpHGzZL5evs1UpXKAWWdFF7TX

Deployed to https://rinkeby.index.domain.tld:8000/subgraphs/name/subgraph-test/graphql

Subgraph endpoints:
Queries (HTTP):     https://rinkeby.index.domain.tld:8000/subgraphs/name/subgraph-test

It means, that starting from graph-cli@0.33.0 we can use IPFS/Index nodes endpoints protected with basic authentication and deploy Subgraph by passing credentials.

[endersonmaia]

@air3ijai it worked here with graph-cli:0.33.1 and --headers parameter

but graph-node indexer won't work with HTTP Basic Auth on the URL, how did you manage it ?

[air3ijai]

@endersonmaia, I can't test it right now but let's see my recent messages - fix: basic auth #927.

As it was also mentioned by the @islishude, his PR works only with the jsonrpc(graph-node).

I confirmed that in our tests - graph create works well. It means that graph deploy also should work and it was conformed in our latest test in this issue.

During all tests we used AWS ALB with basic auth configured as it is mentioned here - amazon alb and basic auth.

[endersonmaia]

@air3ijai let me see if I understood, the --headers will by applied to --ipfs AND --node ? Is that it ?

my problem now is with the graph-node, as I explain in this issue https://github.com/graphprotocol/graph-node/issues/3981

[air3ijai]

@endersonmaia, not so, at least this we discovered during our experiments above

Accordingly to the tests we did:

1. Inline basic authentication works only for Index node
2. Sending headers flag works only with IPFS node

Graph node authentication - inline basic authentication

--node https://user:pass@rinkeby.index.domain.tld

IPFS authentication - sending headers

--headers "{\"Authorization\": \"Basic (username:password)to-base64\"}" \
--ipfs https://ipfs.domain.tld

[air3ijai]

Related to the case you described it is slightly different from our one

1. We run Graph node in Kubernetes
2. We run IPFS node in Kubernetes
3. Graph node have a direct access to the IPFS node in the Kubernetes without any authentication

                 Graph node
               /   |
GitHub --> ALB     |
               \   |
                 IPFS

So, we should distinguish here 2 steps 1. Deploy subgraph to the graph node using graph-cli You can do this based on this guide and it was tested and should work well

2. Indexing subgraph on graph node As you described, when you run graph node you should also specify how it should access IPFS endpoint. We use an environment variable for that -e ipfs=<HOST>:<PORT>, as it is described in the documentation. In Kubernetes manifest it looks like the following

   - name: ipfs
       value: http://ipfs.dev.svc.cluster.local:5001

So, there is no any authentication in our case and also looks like it is not supported yet and this is what do you exactly need.

As a simple workaround you can use

1. Run own IPFS node - good solution and works fast
2. Run a proxy in front of Infura and it will pass authentication for you - better to run IPFS node?

[rguichard]

I also have a protected graph-node and ipfs (behind a Kubernetes Nginx Ingress Controller). I tried to pass authentication headers to IPFS call but it doesn't work.

$ graph --version
0.40.0
$ graph deploy subgraph --node https://foo:bar@thegraph.tld --headers "{\"Authorization\": \"Basic Zm9vOmJhcgo=\"}" --ipfs https://ipfs.tld

Error: Failed to upload file to IPFS: <html>
<head><title>401 Authorization Required</title></head>
<body>
<center><h1>401 Authorization Required</h1></center>
<hr><center>nginx</center>
</body>
</html>

What am I missing ?

[air3ijai]

Can you please try to do the following tests

1. Check if credentials passed in the headers works as expected

   curl -v -H 'Authorization: Basic Zm9vOmJhcgo=' https://ipfs.tld

   We should get 404 code, but it will be not from Nginx, but from IPFS.

2. Check if we can build subgraph and upload it to IPFS

   graph build --headers "{\"Authorization\": \"Basic Zm9vOmJhcgo=\"}" --ipfs https://ipfs.tld

   We should see that subgraph was built and successfully uploaded to the IPFS.

[qiaopengjun5162]

Error: Failed to upload file to IPFS: fetch failed

So what is the solution to this? Are there any specific steps to follow?

[air3ijai]

So what is the solution to this? Are there any specific steps to follow?

App command should be found inside that issue.

1. Which graph CLI version do you use?
2. Which commands are you trying to tun?
3. How your IPFS/Graph nodes are protected?

[qiaopengjun5162]

graph --version @graphprotocol/graph-cli/0.78.0 darwin-arm64 node-v22.1.0

The above problem has been solved, and a new problem has arisen

[saihaj]

graph --version @graphprotocol/graph-cli/0.78.0 darwin-arm64 node-v22.1.0

The above problem has been solved, and a new problem has arisen

for better debugging on graph CLI you can provide a debug flag to get some more logs DEBUG=1 graph deploy

but I also noticed that you get back a 504 from the graph node which does make sense here cause you have a local IPFS node and you are pushing subgraph to the studio graph node deploy router

[qiaopengjun5162]

graph deploy --studio nftmarkethub --ipfs http://127.0.0.1:5001 DEBUG=1 Which version label to use? (e.g. "v0.0.1"): v0.0.1 Error: ENOENT: no such file or directory, open 'DEBUG=1' Code: ENOENT

graph deploy --studio nftmarkethub --ipfs http://127.0.0.1:5001 --debug › Error: Nonexistent flag: --debug › See more help with --help

[qiaopengjun5162]

[qiaopengjun5162]

The problem has been solved, thanks