search

PostHog / posthog

🦔 PostHog provides open-source product analytics, session recording, feature flagging and A/B testing that you can self-host.
https://posthog.com
Other
20.99k stars 1.25k forks source link

Paths query fails with CHQueryErrorUnknownTable #6658

Open sentry-io[bot] opened 2 years ago

sentry-io[bot] commented 2 years ago

Sentry Issue: POSTHOG-3ZT

ServerException: Code: 60.
DB::Exception: Table posthog.funnel_persons doesn't exist. Stack trace:

0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8b6cbba in /usr/bin/clickhouse
1. DB::Exception::Exception<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&&) @ 0x8c7f7e3 in /usr/bin/clickhouse
2. void std::__1::__optional_storage_base<DB::Exception, false>::__construct<int const&, char const (&) [23], std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(int const&, char const (&) [23], std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&&) @ 0xf593cb1 in /usr/bin/clickhouse
3. DB::DatabaseCatalog::getTableImpl(DB::StorageID const&, std::__1::shared_ptr<DB::Context>, std::__1::optional<DB::Exception>*) const @ 0xf586443 in /usr/bin/clickhouse
4. DB::DatabaseCatalog::getTable(DB::StorageID const&, std::__1::shared_ptr<DB::Context>) const @ 0xf58bf88 in /usr/bin/clickhouse
5. ? @ 0xfc1d699 in /usr/bin/clickhouse
6. DB::getDatabaseAndTablesWithColumns(std::__1::vector<DB::ASTTableExpression const*, std::__1::allocator<DB::ASTTableExpression const*> > const&, std::__1::shared_ptr<DB::Context>) @ 0xfc1dd20 in /usr/bin/clickhouse
7. DB::JoinedTables::resolveTables() @ 0xfa87710 in /usr/bin/clickhouse
8. DB::InterpreterSelectQuery::InterpreterSelectQuery(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IBlockInputStream> const&, std::__1::optional<DB::Pipe>, std::__1::shared_ptr<DB::IStorage> const&, DB::SelectQueryOptions const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&) @ 0xf8d41b3 in /usr/bin/clickhouse
9. DB::InterpreterSelectQuery::InterpreterSelectQuery(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context>, DB::SelectQueryOptions const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) @ 0xf8d351e in /usr/bin/clickhouse
10. DB::InterpreterSelectWithUnionQuery::InterpreterSelectWithUnionQuery(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context>, DB::SelectQueryOptions const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) @ 0xfa4c027 in /usr/bin/clickhouse
11. DB::InterpreterSelectWithUnionQuery::getSampleBlock(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context>, bool) @ 0xfa4e164 in /usr/bin/clickhouse
12. ? @ 0xfc1ce8f in /usr/bin/clickhouse
13. DB::getDatabaseAndTablesWithColumns(std::__1::vector<DB::ASTTableExpression const*, std::__1::allocator<DB::ASTTableExpression const*> > const&, std::__1::shared_ptr<DB::Context>) @ 0xfc1dd20 in /usr/bin/clickhouse
14. DB::JoinedTables::resolveTables() @ 0xfa87710 in /usr/bin/clickhouse
15. DB::InterpreterSelectQuery::InterpreterSelectQuery(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IBlockInputStream> const&, std::__1::optional<DB::Pipe>, std::__1::shared_ptr<DB::IStorage> const&, DB::SelectQueryOptions const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&) @ 0xf8d41b3 in /usr/bin/clickhouse
16. DB::InterpreterSelectQuery::InterpreterSelectQuery(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context>, DB::SelectQueryOptions const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) @ 0xf8d351e in /usr/bin/clickhouse
17. DB::InterpreterSelectWithUnionQuery::InterpreterSelectWithUnionQuery(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context>, DB::SelectQueryOptions const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) @ 0xfa4c027 in /usr/bin/clickhouse
18. DB::InterpreterSelectWithUnionQuery::getSampleBlock(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context>, bool) @ 0xfa4e164 in /usr/bin/clickhouse
19. ? @ 0xfc1ce8f in /usr/bin/clickhouse
20. DB::getDatabaseAndTablesWithColumns(std::__1::vector<DB::ASTTableExpression const*, std::__1::allocator<DB::ASTTableExpression const*> > const&, std::__1::shared_ptr<DB::Context>) @ 0xfc1dd20 in /usr/bin/clickhouse
21. DB::JoinedTables::resolveTables() @ 0xfa87710 in /usr/bin/clickhouse
22. DB::InterpreterSelectQuery::InterpreterSelectQuery(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IBlockInputStream> const&, std::__1::optional<DB::Pipe>, std::__1::shared_ptr<DB::IStorage> const&, DB::SelectQueryOptions const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&) @ 0xf8d41b3 in /usr/bin/clickhouse
23. DB::InterpreterSelectQuery::InterpreterSelectQuery(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context>, DB::SelectQueryOptions const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) @ 0xf8d351e in /usr/bin/clickhouse
24. DB::InterpreterSelectWithUnionQuery::InterpreterSelectWithUnionQuery(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context>, DB::SelectQueryOptions const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) @ 0xfa4c027 in /usr/bin/clickhouse
25. DB::InterpreterSelectWithUnionQuery::getSampleBlock(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context>, bool) @ 0xfa4e164 in /usr/bin/clickhouse
26. ? @ 0xfc1ce8f in /usr/bin/clickhouse
27. DB::getDatabaseAndTablesWithColumns(std::__1::vector<DB::ASTTableExpression const*, std::__1::allocator<DB::ASTTableExpression const*> > const&, std::__1::shared_ptr<DB::Context>) @ 0xfc1dd20 in /usr/bin/clickhouse
28. DB::JoinedTables::resolveTables() @ 0xfa87710 in /usr/bin/clickhouse
29. DB::InterpreterSelectQuery::InterpreterSelectQuery(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IBlockInputStream> const&, std::__1::optional<DB::Pipe>, std::__1::shared_ptr<DB::IStorage> const&, DB::SelectQueryOptions const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&) @ 0xf8d41b3 in /usr/bin/clickhouse
30. DB::InterpreterSelectQuery::InterpreterSelectQuery(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Con...
(2 additional frame(s) were not displayed)
...
  File "clickhouse_driver/client.py", line 453, in process_ordinary_query
    return self.receive_result(with_column_types=with_column_types,
  File "clickhouse_driver/client.py", line 117, in receive_result
    return result.get_result()
  File "clickhouse_driver/result.py", line 50, in get_result
    for packet in self.packet_generator:
  File "clickhouse_driver/client.py", line 133, in packet_generator
    packet = self.receive_packet()
  File "clickhouse_driver/client.py", line 150, in receive_packet
    raise packet.exception

CHQueryErrorUnknownTable: Code: 60.
DB::Exception: Table posthog.funnel_persons doesn't exist. Stack trace:

0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8b6cbba in /usr/bin/clickhouse
1. DB::Exception::Exception<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&&) @ 0x8c7f7e3 in /usr/bin/clickhouse
2. void std::__1::__optional_storage_base<DB::Exception, false>::__construct<int const&, char const (&) [23], std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(int const&, char const (&) [23], std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&&) @ 0xf593cb1 in /usr/bin/clickhouse
3. DB::DatabaseCatalog::getTableImpl(DB::StorageID const&, std::__1::shared_ptr<DB::Context>, std::__1::optional<DB::Exception>*) const @ 0xf586443 in /usr/bin/clickhouse
4. DB::DatabaseCatalog::getTable(DB::StorageID const&, std::__1::shared_ptr<DB::Context>) const @ 0xf58bf88 in /usr/bin/clickhouse
5. ? @ 0xfc1d699 in /usr/bin/clickhouse
6. DB::getDatabaseAndTablesWithColumns(std::__1::vector<DB::ASTTableExpression const*, std::__1::allocator<DB::ASTTableExpression const*> > const&, std::__1::shared_ptr<DB::Context>) @ 0xfc1dd20 in /usr/bin/clickhouse
7. DB::JoinedTables::resolveTables() @ 0xfa87710 in /usr/bin/clickhouse
8. DB::InterpreterSelectQuery::InterpreterSelectQuery(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IBlockInputStream> const&, std::__1::optional<DB::Pipe>, std::__1::shared_ptr<DB::IStorage> const&, DB::SelectQueryOptions const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&) @ 0xf8d41b3 in /usr/bin/clickhouse
9. DB::InterpreterSelectQuery::InterpreterSelectQuery(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context>, DB::SelectQueryOptions const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) @ 0xf8d351e in /usr/bin/clickhouse
10. DB::InterpreterSelectWithUnionQuery::InterpreterSelectWithUnionQuery(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context>, DB::SelectQueryOptions const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) @ 0xfa4c027 in /usr/bin/clickhouse
11. DB::InterpreterSelectWithUnionQuery::getSampleBlock(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context>, bool) @ 0xfa4e164 in /usr/bin/clickhouse
12. ? @ 0xfc1ce8f in /usr/bin/clickhouse
13. DB::getDatabaseAndTablesWithColumns(std::__1::vector<DB::ASTTableExpression const*, std::__1::allocator<DB::ASTTableExpression const*> > const&, std::__1::shared_ptr<DB::Context>) @ 0xfc1dd20 in /usr/bin/clickhouse
14. DB::JoinedTables::resolveTables() @ 0xfa87710 in /usr/bin/clickhouse
15. DB::InterpreterSelectQuery::InterpreterSelectQuery(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IBlockInputStream> const&, std::__1::optional<DB::Pipe>, std::__1::shared_ptr<DB::IStorage> const&, DB::SelectQueryOptions const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&) @ 0xf8d41b3 in /usr/bin/clickhouse
16. DB::InterpreterSelectQuery::InterpreterSelectQuery(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context>, DB::SelectQueryOptions const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) @ 0xf8d351e in /usr/bin/clickhouse
17. DB::InterpreterSelectWithUnionQuery::InterpreterSelectWithUnionQuery(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context>, DB::SelectQueryOptions const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) @ 0xfa4c027 in /usr/bin/clickhouse
18. DB::InterpreterSelectWithUnionQuery::getSampleBlock(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context>, bool) @ 0xfa4e164 in /usr/bin/clickhouse
19. ? @ 0xfc1ce8f in /usr/bin/clickhouse
20. DB::getDatabaseAndTablesWithColumns(std::__1::vector<DB::ASTTableExpression const*, std::__1::allocator<DB::ASTTableExpression const*> > const&, std::__1::shared_ptr<DB::Context>) @ 0xfc1dd20 in /usr/bin/clickhouse
21. DB::JoinedTables::resolveTables() @ 0xfa87710 in /usr/bin/clickhouse
22. DB::InterpreterSelectQuery::InterpreterSelectQuery(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IBlockInputStream> const&, std::__1::optional<DB::Pipe>, std::__1::shared_ptr<DB::IStorage> const&, DB::SelectQueryOptions const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&) @ 0xf8d41b3 in /usr/bin/clickhouse
23. DB::InterpreterSelectQuery::InterpreterSelectQuery(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context>, DB::SelectQueryOptions const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) @ 0xf8d351e in /usr/bin/clickhouse
24. DB::InterpreterSelectWithUnionQuery::InterpreterSelectWithUnionQuery(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context>, DB::SelectQueryOptions const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) @ 0xfa4c027 in /usr/bin/clickhouse
25. DB::InterpreterSelectWithUnionQuery::getSampleBlock(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context>, bool) @ 0xfa4e164 in /usr/bin/clickhouse
26. ? @ 0xfc1ce8f in /usr/bin/clickhouse
27. DB::getDatabaseAndTablesWithColumns(std::__1::vector<DB::ASTTableExpression const*, std::__1::allocator<DB::ASTTableExpression const*> > const&, std::__1::shared_ptr<DB::Context>) @ 0xfc1dd20 in /usr/bin/clickhouse
28. DB::JoinedTables::resolveTables() @ 0xfa87710 in /usr/bin/clickhouse
29. DB::InterpreterSelectQuery::InterpreterSelectQuery(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context>, std::__1::shared_ptr<DB::IBlockInputStream> const&, std::__1::optional<DB::Pipe>, std::__1::shared_ptr<DB::IStorage> const&, DB::SelectQueryOptions const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&) @ 0xf8d41b3 in /usr/bin/clickhouse
30. DB::InterpreterSelectQuery::InterpreterSelectQuery(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Con...
(6 additional frame(s) were not displayed)
...
  File "posthog/tasks/update_cache.py", line 89, in update_cache_item
    result = _calculate_by_filter(filter, key, team_id, cache_type)
  File "posthog/tasks/update_cache.py", line 165, in _calculate_by_filter
    result = insight_class(filter, Team(pk=team_id)).run(filter, Team(pk=team_id))
  File "ee/clickhouse/queries/paths/paths.py", line 64, in run
    results = self._exec_query()
  File "ee/clickhouse/queries/paths/paths.py", line 84, in _exec_query
    return sync_execute(query, self.params)
  File "ee/clickhouse/client.py", line 156, in sync_execute
    raise err
neilkakkar commented 2 years ago

Had a (quick) look: it's the caching thing, where the way we generate the Paths filter is wrong :/ - it has no funnel_filter anymore, which causes the dashboard refreshes to fail.

https://github.com/PostHog/posthog/blob/master/posthog/tasks/update_cache.py#L165

which should instead look like: https://github.com/PostHog/posthog/blob/master/ee/clickhouse/views/insights.py#L84

macobo commented 2 years ago

When fixing it's worth considering how to avoid adding insight-specific logic to the caching logic. Perhaps funnel_filter param shouldn't even exist in the first place?

neilkakkar commented 2 years ago

the reason we end up having something like that is because we can't have filters inside filters. Agree though, maybe there's a way to create the funnel Filter later using something like Filter(data=filter._data.get('funnel_filter'))

posthog-bot commented 10 months ago

This issue hasn't seen activity in two years! If you want to keep it open, post a comment or remove the stale label – otherwise this will be closed in two weeks.

neilkakkar commented 10 months ago

still valid