Error adding migrations

[subwolf]

subwolf@localhost:~/storjnet-info/migrations$ go run main.go up
# command-line-arguments
./main.go:90:26: cannot use dbLogger literal (type dbLogger) as type pg.QueryHook in argument to db.baseDB.AddQueryHook:
        dbLogger does not implement pg.QueryHook (wrong type for AfterQuery method)
                have AfterQuery(context.Context, *pg.QueryEvent) (context.Context, error)
                want AfterQuery(context.Context, *pg.QueryEvent) error

[3bl3gamer]

Should be fixed since https://github.com/3bl3gamer/storjnet-info/commit/91dae6392fefe331e9cca8cd39f43725f0b5fe8d

Required go-pg* versions were messed up a bit before.

[subwolf]

New paths do not exist. Got:

subwolf@localhost:~/storjnet-info/migrations$ sudo go run main.go
main.go:12:2: cannot find package "github.com/go-pg/migrations/v7" in any of:
        /usr/lib/go-1.10/src/github.com/go-pg/migrations/v7 (from $GOROOT)
        /home/subwolf/go/src/github.com/go-pg/migrations/v7 (from $GOPATH)
main.go:13:2: cannot find package "github.com/go-pg/pg/v9" in any of:
        /usr/lib/go-1.10/src/github.com/go-pg/pg/v9 (from $GOROOT)
        /home/subwolf/go/src/github.com/go-pg/pg/v9 (from $GOPATH)

So ran:

subwolf@localhost:~/storjnet-info/migrations$ go get github.com/go-pg/migrations/v7
package github.com/go-pg/migrations/v7: cannot find package "github.com/go-pg/migrations/v7" in any of:
        /usr/lib/go-1.10/src/github.com/go-pg/migrations/v7 (from $GOROOT)
        /home/subwolf/go/src/github.com/go-pg/migrations/v7 (from $GOPATH)

https://github.com/go-pg/migrations/v7 is 404.

[3bl3gamer]

Are you using Go v1.10? This package relies on go modules which were introduced in v1.11

[subwolf]

subwolf@localhost:~/storjnet-info/migrations$ go version
go version go1.10.4 linux/amd64

I'll upgrade.

[subwolf]

Improving! Missing a table or two.

subwolf@localhost:~/storjnet-info/migrations$ sudo go run main.go
2019/08/31 11:00:22 48.252798ms
SELECT count(*) FROM "pg_tables" WHERE (schemaname = 'storjinfo') AND (tablename = 'gopg_migrations')
panic: table "storjinfo.gopg_migrations" does not exists; did you run init?

goroutine 1 [running]:
main.main()
        /home/subwolf/storjnet-info/migrations/main.go:96 +0x445
exit status 2

[3bl3gamer]

For some reason go-pg/migrations requires explicit init before migrating (it will be sudo go run main.go init in your case).

Updated README https://github.com/3bl3gamer/storjnet-info/blob/master/README.md#db-setup

[3bl3gamer]

Oh, and go run main.go will not include .go files with migrations so it won't migrate anything. It should be like go run *.go.

[subwolf]

I'm doing something silly, I've been researching but no luck.

2019/09/09 04:57:18 3.255838ms
CREATE SCHEMA IF NOT EXISTS storjinfo
2019/09/09 04:57:18 268.891_s
CREATE TABLE storjinfo.gopg_migrations (
        id serial,
        version bigint,
        created_at timestamptz
)
panic: ERROR #42501 permission denied for schema storjinfo

[3bl3gamer]

Strange. Have you already created schema? Maybe it was somehow created from different user? Check \dn+, it should output smth like this:

storjinfo_db=> \dn+
                            List of schemas
   Name    |   Owner   |  Access privileges   |      Description       
-----------+-----------+----------------------+------------------------
 public    | postgres  | postgres=UC/postgres+| standard public schema
           |           | =UC/postgres         | 
 storjinfo | storjinfo |                      | 
(2 rows)

If owner of storjinfo schema is not storjinfo (user has the same name as schema) it may cause problem. In this case you can change schema owner with: ALTER SCHEMA storjinfo OWNER TO storjinfo or you can add privileges for storjinfo-user to existing schema: GRANT ALL ON SCHEMA storjinfo TO storjinfo

[subwolf]

I got the migrations to all run, starting http server I'm getting password authentication failed, even though there is a connection line in utils.go:33 that I have changed.

func makePGConnection() *pg.DB {
        db := pg.Connect(&pg.Options{User: "storjinfo", Password: "xxxxxxx", Database: "storjinfo_db"})
        // db.OnQueryProcessed(func(event *pg.QueryProcessedEvent) {
        //      query, err := event.FormattedQuery()

2019/09/21 04:09:10 ERRO: SERVER: FATAL #28P01 password authentication failed for user "storjinfo"

main.HandleIndex
        /home/subwolf/storjnet-info/server.go:340
main.wrap.func1
        /home/subwolf/storjnet-info/server.go:309
github.com/julienschmidt/httprouter.(*Router).ServeHTTP
        /home/subwolf/go/src/github.com/julienschmidt/httprouter/router.go:334
net/http.serverHandler.ServeHTTP
        /usr/lib/go-1.10/src/net/http/server.go:2697
net/http.(*conn).serve
        /usr/lib/go-1.10/src/net/http/server.go:1830
runtime.goexit
        /usr/lib/go-1.10/src/runtime/asm_amd64.s:2361

When I look up the location I find references to db.Model that must be in an outside library.

[subwolf]

Grrrrrr, deliberately changed the username but it was ignored.

unc makePGConnection() *pg.DB {
        db := pg.Connect(&pg.Options{User: "wtf", Password: "xxxx", Database: "storjinfo_db"})

And

2019/09/24 08:10:12 starting HTTP server on 0.0.0.0:9002
2019/09/24 08:10:20 ERRO: SERVER: FATAL #28P01 password authentication failed for user "storjinfo"

Explain that?

[3bl3gamer]

Are you really-really-100% sure you have saved changes and rebuild binary? (maybe file was changed in another repo copy, maybe go build was run in migrations folder instead of project root, etc.)

pg.Connect is used exactly twice: in utils.go and migrations/main.go. Credentials changes in utils.go resulted in role/password errors for me (I've checked).

P.S.: I've also found that migrations and main app used different default DB passwords, oops. Fixed it in https://github.com/3bl3gamer/storjnet-info/commit/e00c185045258bf01196ee7a647937f50bd96c84 Though it should not be related with last ignored-username problem.

[subwolf]

Ok I'm getting there, got the data collector to run but it's crashing out trying to collect Kademlia data. Could this be because Kademlia was completely removed recently?

2019/09/27 03:46:24 INFO: CHANS:  KAD: 0->0->0, SELF: 0->0
Error: transport error: write tcp 127.0.0.1:56310->127.0.0.1:7778: write: broken pipe
2019/09/27 03:46:29 transport error: write tcp 127.0.0.1:56310->127.0.0.1:7778: write: broken pipe

main.StartNodesKadDataFetcher
        /home/subwolf/storjnet-info/update.go:29
main.CMDRun
        /home/subwolf/storjnet-info/main.go:130

[subwolf]

Also, seeing a context deadline exceeded error, which also relates to fetching Kademlia data:

2019/10/02 04:18:32 INFO: DB-IDS: no old IDs
2019/10/02 04:18:37 INFO: CHANS:  KAD: 0->0->0, SELF: 0->0
2019/10/02 04:18:42 INFO: DB-KAD: no old KADs
Error: transport error: context deadline exceeded
2019/10/02 04:18:42 transport error: context deadline exceeded

main.StartNodesKadDataFetcher
        /home/subwolf/storjnet-info/update.go:29
main.CMDRun
        /home/subwolf/storjnet-info/main.go:130
github.com/spf13/cobra.(*Command).execute
        /home/subwolf/go/pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:762
github.com/spf13/cobra.(*Command).ExecuteC
        /home/subwolf/go/pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:852
github.com/spf13/cobra.(*Command).Execute
        /home/subwolf/go/pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:800
main.main
        /home/subwolf/storjnet-info/main.go:161
runtime.main
        /usr/lib/go-1.12/src/runtime/proc.go:200
runtime.goexit
        /usr/lib/go-1.12/src/runtime/asm_amd64.s:1337

[3bl3gamer]

transport error: write tcp 127.0.0.1:56310->127.0.0.1:7778: write: broken pipe

Have not seen such errors. Maybe, some problems in docker port forwarding? I run node without docker and storjnet connected to it successfully. If node wasn't running, I just got connect: connection refused.

Could this be because Kademlia was completely removed recently?

Unlikely. It was still used when you got that error. Even now, when Kademlia was actually removed for two days, storjnet can communicate with no-kademlia-v0.22-nodes via old v0.21.3 node.