langflow-ai / langflow

Langflow is a low-code app builder for RAG and multi-agent AI applications. It’s Python-based and agnostic to any model, API, or database.
http://www.langflow.org
MIT License
35.42k stars 4.22k forks source link

fix: traces inputs and outputs not being sent to tracing services #4669

Closed rogeriochaves closed 3 days ago

rogeriochaves commented 6 days ago

Hey there, we found 3 issues that broke tracing a little bit since a few months after a langflow refactoring, this PR fixes two of them:

The issue with input values is that the ones from _inputs seems to be the ones actually holding the value in runtime and those were not being passed, also it's a bit weird and heavy for monitoring tools that it sends the component code every time, the Dynamic inputs didn't seem to be working well, so I've replaced with _inputs which does what we expected

Now the issue with output is that an async trace ending was introduced, but the reset_io was kept sync, as a result the outputs were being set back to {} before the trace actually could capture the output value

For the flow_name being none, we couldn't find an easy way to fix it, so we put just a fallback for langwatch in the meantime

Before

notice input_value and output being empty, but full code present

image

After

image
codspeed-hq[bot] commented 6 days ago

CodSpeed Performance Report

Merging #4669 will degrade performances by 38.94%

Comparing rogeriochaves:fix-traces-inputs-outputs (cbfc881) with main (8fb19ad)

Summary

⚡ 2 improvements
❌ 2 regressions
✅ 11 untouched benchmarks

:warning: Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Benchmark main rogeriochaves:fix-traces-inputs-outputs Change
test_build_flow 4,880.8 ms 410.3 ms ×12
test_successful_run_with_input_type_any 234.7 ms 324.5 ms -27.69%
test_successful_run_with_input_type_text 323 ms 229.9 ms +40.52%
test_successful_run_with_output_type_debug 140.1 ms 229.5 ms -38.94%
ogabrielluiz commented 6 days ago

Hey @rogeriochaves

How are you?

Thanks for this.

Does making _reset_io async solve the issue?

rogeriochaves commented 5 days ago

@ogabrielluiz going good, thanks for asking 😄

you mean running it from where it was but then also async? I haven't tried, but I wouldn't risk it, I think it could work some times and some times not due to race conditions, I think it's more reliable and also makes sense to happen after the end loops are done, also because nobody else call those functions, to just end without resetting

ogabrielluiz commented 5 days ago

@rogeriochaves I fixed the flow_name being always None. Can you validate, please? The PR LGTM.

rogeriochaves commented 4 days ago

hey @ogabrielluiz, the fix for the flow_name works but the last refactor is breaking any flow execution: https://app.warp.dev/block/k3mUoiYZYuxPKDByJRotQp

I think it's better to revert it back, or refactor in some other way and we are good to go