search

Netflix / metaflow-service

:rocket: Metadata tracking and UI service for Metaflow!
http://www.metaflow.org
Apache License 2.0
193 stars 71 forks source link

[Ready for review] Update SearchAPI to search for foreach variable name and value. #411

Closed darinyu closed 9 months ago

darinyu commented 10 months ago

Add the option to search for foreach variable name and value (based on scope in request).

In this PR, we will

  1. search in metadata table. We will look up all the tasks and metadata field="foreach-stack", we then filter based on the search term. Note that we do not have sub-process searching in this case because the qps and data sizes is expected to be low. We can add workers for this in the future if necessary.
  2. Allow control on artifact search and foreach variable search separately.

Note: git somehow forgets it is a rename in this PR, this commit shows the change in search.py better.

darinyu commented 10 months ago

I'd prefer @saikonen to review as well. I tried looking at the changes between the old file and the new one and they look OK (I think). I see you mostly added a new type of search (in metadata) but kept the rest the same.

yes. git somehow forgets it is a rename in this PR, this commit shows the change in search.py better.