fastrepl / canary

Algolia alternative for technical docs
https://getcanary.dev
Other
457 stars 10 forks source link

index operation in `after_batch` cause DB timeout #169

Open yujonglee opened 1 month ago

yujonglee commented 1 month ago

ideally, we should do this after_transaction?

[error] Postgrex.Protocol (#PID<0.512.0>) disconnected: ** (DBConnection.ConnectionError) client #PID<0.1810.0> timed out because it queued and checked out the connection for longer than 15000ms

#PID<0.1810.0> was at location:

    (stdlib 6.0.1) gen.erl:241: :gen.do_call/4
    (stdlib 6.0.1) gen_statem.erl:2633: :gen_statem.call/3
    (ssl 11.2.1) ssl_gen_statem.erl:1267: :ssl_gen_statem.call/2
    (mint 1.6.2) lib/mint/core/transport/ssl.ex:382: Mint.Core.Transport.SSL.recv/3
    (mint 1.6.2) lib/mint/http1.ex:519: Mint.HTTP1.recv/3
    (finch 0.19.0) lib/finch/http1/conn.ex:262: Finch.HTTP1.Conn.receive_response/8
    (finch 0.19.0) lib/finch/http1/conn.ex:131: Finch.HTTP1.Conn.request/8
    (finch 0.19.0) lib/finch/http1/pool.ex:60: anonymous fn/10 in Finch.HTTP1.Pool.request/6
    (nimble_pool 1.1.0) lib/nimble_pool.ex:462: NimblePool.checkout!/4
    (finch 0.19.0) lib/finch/http1/pool.ex:52: Finch.HTTP1.Pool.request/6
    (finch 0.19.0) lib/finch.ex:493: anonymous fn/4 in Finch.request/3
    (telemetry 1.3.0) /Users/yujonglee/dev/company/canary/core/deps/telemetry/src/telemetry.erl:324: :telemetry.span/3
    (req 0.5.6) lib/req/finch.ex:239: Req.Finch.run_finch_request/3
    (req 0.5.6) lib/req/finch.ex:71: Req.Finch.run/4
    (req 0.5.6) lib/req/request.ex:1103: Req.Request.run_request/1
    (req 0.5.6) lib/req/request.ex:1047: Req.Request.run/1
    (canary 0.1.0) lib/canary/index/trieve/client.ex:181: anonymous fn/3 in Canary.Index.Trieve.Actual.upsert_chunks/2
    (elixir 1.17.2) lib/enum.ex:4858: Enumerable.List.reduce/3
    (elixir 1.17.2) lib/enum.ex:2585: Enum.reduce_while/3
    (canary 0.1.0) lib/canary/sources/document_create.ex:190: Canary.Sources.Document.Create.after_batch/3

We don't get this super often. set timeout to 30s as a workaround for now.

yujonglee commented 1 month ago

https://github.com/ash-project/ash/issues/1564