search

Sifchain / sa-eliza

Conversational Agent for Twitter and Discord
https://ai16z.github.io/eliza/
MIT License
0 stars 0 forks source link

Privately turn on test run in CI/CD and file tickets on whatever breaks #13

Open snobbee opened 1 week ago

snobbee commented 1 week ago

Privately turn on test run in CI/CD and file tickets on whatever breaks

pgoos commented 1 week ago

Log from CI: https://github.com/pgoos/eliza/actions/runs/11955668665/job/33328568537?pr=1

Run cd packages/core && pnpm test

> @ai1[6](https://github.com/pgoos/eliza/actions/runs/11955668665/job/33328568537?pr=1#step:8:7)z/eliza@0.1.3 test /home/runner/work/eliza/eliza/packages/core
> vitest run

 RUN  v2.1.5 /home/runner/work/eliza/eliza/packages/core

stdout | src/tests/goals.test.ts
Current directory: /home/runner/work/eliza/eliza/packages/core/src/test_resources
Trying to load env from: /home/runner/work/eliza/eliza/packages/core/.env.test

stdout | src/tests/relationships.test.ts
Current directory: /home/runner/work/eliza/eliza/packages/core/src/test_resources
Trying to load env from: /home/runner/work/eliza/eliza/packages/core/.env.test

stdout | src/tests/database.test.ts
Current directory: /home/runner/work/eliza/eliza/packages/core/src/test_resources
Trying to load env from: /home/runner/work/eliza/eliza/packages/core/.env.test

 ✓ src/tests/database.test.ts (8 tests) 9ms
 ✓ src/tests/relationships.test.ts (6 tests) 13ms
 ❯ src/tests/goals.test.ts (0 test)
stdout | src/tests/videoGeneration.test.ts
Current directory: /home/runner/work/eliza/eliza/packages/core/src/test_resources
Trying to load env from: /home/runner/work/eliza/eliza/packages/core/.env.test

stdout | src/tests/messages.test.ts
Current directory: /home/runner/work/eliza/eliza/packages/core/src/test_resources
Trying to load env from: /home/runner/work/eliza/eliza/packages/core/.env.test

stdout | src/tests/providers.test.ts
Current directory: /home/runner/work/eliza/eliza/packages/core/src/test_resources
Trying to load env from: /home/runner/work/eliza/eliza/packages/core/.env.test

 ✓ src/tests/videoGeneration.test.ts (3 tests) 11ms
 ✓ src/tests/providers.test.ts (5 tests) [7](https://github.com/pgoos/eliza/actions/runs/11955668665/job/33328568537?pr=1#step:8:8)ms
 ✓ src/tests/messages.test.ts (6 tests) 6ms
stdout | src/tests/evaluators.test.ts
Current directory: /home/runner/work/eliza/eliza/packages/core/src/test_resources
Trying to load env from: /home/runner/work/eliza/eliza/packages/core/.env.test

stdout | src/tests/token.test.ts
Current directory: /home/runner/work/eliza/eliza/packages/core/src/test_resources
Trying to load env from: /home/runner/work/eliza/eliza/packages/core/.env.test

stdout | src/tests/posts.test.ts
Current directory: /home/runner/work/eliza/eliza/packages/core/src/test_resources
Trying to load env from: /home/runner/work/eliza/eliza/packages/core/.env.test

 ✓ src/tests/posts.test.ts (3 tests) 5ms
 ✓ src/tests/evaluators.test.ts (4 tests) 5ms
stdout | src/tests/models.test.ts
Current directory: /home/runner/work/eliza/eliza/packages/core/src/test_resources
Trying to load env from: /home/runner/work/eliza/eliza/packages/core/.env.test

stdout | src/tests/defaultCharacters.test.ts
Current directory: /home/runner/work/eliza/eliza/packages/core/src/test_resources
Trying to load env from: /home/runner/work/eliza/eliza/packages/core/.env.test

 ✓ src/tests/models.test.ts (7 tests) 6ms
 ✓ src/tests/defaultCharacters.test.ts (11 tests) [8](https://github.com/pgoos/eliza/actions/runs/11955668665/job/33328568537?pr=1#step:8:9)ms
(node:9652) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
stdout | src/tests/cache.test.ts
Current directory: /home/runner/work/eliza/eliza/packages/core/src/test_resources
Trying to load env from: /home/runner/work/eliza/eliza/packages/core/.env.test

stdout | src/tests/env.test.ts
Current directory: /home/runner/work/eliza/eliza/packages/core/src/test_resources
Trying to load env from: /home/runner/work/eliza/eliza/packages/core/.env.test

stdout | src/tests/env.test.ts > Environment Setup > should verify .env.test file exists
Current working directory: /home/runner/work/eliza/eliza/packages/core
__dirname: /home/runner/work/eliza/eliza/packages/core/src/tests
Path /home/runner/work/eliza/eliza/packages/core/.env.test exists: true
Path /home/runner/work/eliza/eliza/packages/core/packages/core/.env.test exists: false
Path /home/runner/work/eliza/eliza/packages/core/.env.test exists: true
Path /home/runner/work/eliza/eliza/packages/core/src/.env.test exists: false
Path /home/runner/work/eliza/eliza/packages/core/src/tests/.env.test exists: false

 ✓ src/tests/env.test.ts (1 test) 4ms
 ❯ src/tests/cache.test.ts (0 test)
stdout | src/tests/token.test.ts > TokenProvider Tests > should fetch token security data
 ["◎ sqlite-vec extensions loaded successfully."] 

 ["◎ sqlite-vec extensions loaded successfully."] 

 ✓ SUCCESS
   Agent ID 
   ec6bb8be-2524-0c27-b174-2e0fa328[9](https://github.com/pgoos/eliza/actions/runs/11955668665/job/33328568537?pr=1#step:8:10)[10](https://github.com/pgoos/eliza/actions/runs/11955668665/job/33328568537?pr=1#step:8:11)2 

 ❯ src/tests/token.test.ts (1 test | 1 failed) 25ms
   × TokenProvider Tests > should fetch token security data 24ms
     → Endpoint URL must start with `http:` or `https:`.

⎯⎯⎯⎯⎯⎯ Failed Suites 2 ⎯⎯⎯⎯⎯⎯⎯

 FAIL  src/tests/cache.test.ts [ src/tests/cache.test.ts ]
ReferenceError: jest is not defined
 ❯ src/tests/cache.test.ts:9:5
      7|     let cache: CacheManager<MemoryCacheAdapter>;
      8| 
      9|     jest.useFakeTimers();
       |     ^
     10| 
     [11](https://github.com/pgoos/eliza/actions/runs/11955668665/job/33328568537?pr=1#step:8:12)|     beforeEach(() => {

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/3]⎯

 FAIL  src/tests/goals.test.ts [ src/tests/goals.test.ts ]
ReferenceError: CacheManager is not defined
 ❯ src/tests/goals.test.ts:34:23
     32| export const mockRuntime: IAgentRuntime = {
     33|     databaseAdapter: mockDatabaseAdapter as any,
     34|     cacheManager: new CacheManager(new MemoryCacheAdapter()),
       |                       ^
     35|     agentId: "qweqew-qweqwe-qweqwe-qweqwe-qweeqw",
     36|     serverUrl: "",

 Test Files  3 failed | 10 passed ([13](https://github.com/pgoos/eliza/actions/runs/11955668665/job/33328568537?pr=1#step:8:14))
      Tests  1 failed | 54 passed (55)
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[2/3]⎯

⎯⎯⎯⎯⎯⎯⎯ Failed Tests 1 ⎯⎯⎯⎯⎯⎯⎯

 FAIL  src/tests/token.test.ts > TokenProvider Tests > should fetch token security data
TypeError: Endpoint URL must start with `http:` or `https:`.
 ❯ assertEndpointUrl ../../node_modules/@solana/web3.js/src/connection.ts:362:11
 ❯ new Connection ../../node_modules/@solana/web3.js/src/connection.ts:3234:25
 ❯ src/tests/token.test.ts:21:13
     19| 
     20|         const walletProvider = new WalletProvider(
     21|             new Connection(runtime.getSetting("RPC_URL")),
       |             ^
     22|             new PublicKey(runtime.getSetting("WALLET_PUBLIC_KEY"))
     23|         );

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[3/3]⎯

   Start at  [14](https://github.com/pgoos/eliza/actions/runs/11955668665/job/33328568537?pr=1#step:8:15):50:21
   Duration  1.79s (transform 418ms, setup 202ms, collect 1.21s, tests 99ms, environment 3ms, prepare 1.26s)

Error: ReferenceError: jest is not defined
 ❯ src/tests/cache.test.ts:9:5

Error: ReferenceError: CacheManager is not defined
 ❯ src/tests/goals.test.ts:34:23

Error: TypeError: Endpoint URL must start with `http:` or `https:`.
 ❯ assertEndpointUrl ../../node_modules/@solana/web3.js/src/connection.ts:362:11
 ❯ new Connection ../../node_modules/@solana/web3.js/src/connection.ts:3234:25
 ❯ src/tests/token.test.ts:[21](https://github.com/pgoos/eliza/actions/runs/11955668665/job/33328568537?pr=1#step:8:22):13

 ELIFECYCLE  Test failed. See above for more details.
Error: Process completed with exit code 1.
pgoos commented 1 week ago

Seems like 3 tests are broken. I'll file tickets for them

snobbee commented 1 week ago

@pgoos you can find the broken tests result in this comment here https://github.com/ai16z/eliza/pull/463#issuecomment-2490964674

pgoos commented 4 days ago

New PR created: https://github.com/ai16z/eliza/pull/590 - all tests are passing now.