search

jamesaoverton / cmi-pb-terminology

CMI-PB Controlled Terminology
0 stars 0 forks source link

Use metatable information to support autocomplete; use NULL as the metadata field for valid columns with no extra associated information #55

Closed lmcmicu closed 2 years ago

lmcmicu commented 2 years ago

Closes #51, closes #52 (please close them manually since this PR is not going into master).

lmcmicu commented 2 years ago

@beckyjackson please look at get_matching_values(config, table_name, column_name, leading_string="") in validate.py and let me know if the function signature will work for you or if it needs to be tweaked.

beckyjackson commented 2 years ago

Maybe I'm missing something, but I'm not able to get results from get_matching_values (just returns an empty list). I had to replace the code on lines 399-436 with what you sent me in Slack to get around multiprocessing, so maybe I'm missing a part of the loading.

Here's what I'm trying just to test getting values from the table column of the table table:

data = get_matching_values(config, "table", "table")
lmcmicu commented 2 years ago

Hi Becky, for the example you mentioned:

data = get_matching_values(config, "table", "table")

this will not return anything because the datatype is not defined in terms of an in() condition. See the issue description here: https://github.com/jamesaoverton/cmi-pb-terminology/issues/51#issue-1140116403

We are only returning values for:

  1. Datatypes defined in terms of "in"
  2. "from" structures
  3. "under" structures

As for table_name, it is defined in the datatype table as exclude(/\W/)

lmcmicu commented 2 years ago

To turn off the multiprocessing, set the MULTIPROCESSING flag to False (at this line). Sorry I should have set it to False by default. I'll do that now.

beckyjackson commented 2 years ago

Thanks - I got it working with Typeahead so the output looks good to me.

lmcmicu commented 2 years ago

Converting to draft since we need to modify the get_matching_values() function. Instead of only matching on values that begin with the given string parameter, we need to match on values that contain that parameter as a substring (which can be anywhere in the value). We should also perform the match in SQL instead of python. See this issue comment for more info.

jamesaoverton commented 2 years ago

@beckyjackson We are no longer wrapping SQL JSON value with json(...) (which was a mistake I didn't understand/notice), so lines like this https://github.com/ontodev/sprocket/blob/standalone-table/sprocket/render.py#L262 metadata = json.loads(meta[5:-1]) should be changed to metadata = json.loads(meta)