search

mahmoud / glom

☄️ Python's nested data operator (and CLI), for all your declarative restructuring needs. Got data? Glom it! ☄️
https://glom.readthedocs.io
Other
1.88k stars 61 forks source link

Should we use this or boltons.remap? #237

Closed matanox closed 9 months ago

matanox commented 2 years ago

Thanks for this great library. Just a question, how should one position this v.s. https://sedimental.org/remap.html when choosing api for data manipulation? they may seem to have some overlap on first look.

majidaldo commented 9 months ago

bump. same question. i can only think of glom having a "syntax" whereas remap sticks with arbitrary recursion.

mahmoud commented 9 months ago

Oh, great question! So my standard recommendation on this:

Remap performs a full traversal of a nested data structure, walking it like a tree. In contrast, glom only goes where it's told by the spec.

We've still got our eye on integrating remap-like features, but we haven't found an API we liked yet.

Does that make sense? I'd like to get a good answer to put in the FAQ doc :)

mahmoud commented 9 months ago

For example:

You run an error reporting service. Users send you an arbitrary dictionary of metadata related to the error. But you have a requirement that you don't store secrets.

Remap is a great way to traverse that full structure, look for all keys containing the substring "secret" and replace the value with "[REDACTED]". The output shape will be the same as the input shape.

Glom doesn't have a way to do that now. The output shape is predefined by the spec.

majidaldo commented 9 months ago

I feel like glom could be built ontop of remap; b/c remap is more general but glom is more specific. The glom specs are like the remap queries and visitors.

mahmoud commented 9 months ago

That'd be a fun challenge! But for day-to-day usage, it's rarely necessary to visit every element of a target structure. And design-wise remap's hooks don't have as much context as glom's plugin architecture. There's a reason we built (and still use!) remap, and then built glom :)

majidaldo commented 9 months ago

Thanks! Satisfactory answer for me at least to close the issue.

mahmoud commented 9 months ago

Great! I added a FAQ entry here: https://glom.readthedocs.io/en/latest/faq.html#should-i-use-glom-or-remap