atviriduomenys / spinta

Spinta is a framework to describe, extract and publish data (a DEP Framework).
MIT License
13 stars 4 forks source link

Command line tool for keymaps #461

Open sirex opened 1 year ago

sirex commented 1 year ago

When trying to understand an issue with data, it is often important to know, what is inside a keymap.db file. One can open keymap.db file, with an SQLite database manager and look what is inside, but since local identifiers are encoded with MsgPack and hashed with SHA1, it is not always easy to find what you are looking for.

It is possible, to get hashed value using following oneliner:

$ python -c 'import msgpack, hashlib; print(hashlib.sha1(msgpack.dumps(11, use_bin_type=True)).hexdigest())'
067d5096f219c64b53bb1c7d5e3754285b565a47

But this is not very convenient and user friendly.

So it would be nice to have something like this:

$ spinta keymap get --local datasets/gov/rc/ar/savivaldybes/Savivladybe 0c5a1536-9668-4406-b0cf-552aa25d8671
11

Here, we given a model name, and an global id and command returns local id.

Alternatively, we can do same thing in reverse:

$ spinta keymap get --global datasets/gov/rc/ar/savivaldybes/Savivladybe 11
0c5a1536-9668-4406-b0cf-552aa25d8671

Command with --global flag, should cast 11 to integer if property in model.ref is integer type.

sirex commented 12 months ago
python - ~/.local/share/spinta/keymap.db datasets/gov/example/Data 264ae0f9-53eb-496b-a07c-ce1b9cbe510c <<'EOF'
import sys, os, msgpack, hashlib
import sqlalchemy as sa
keymap, table, key = sys.argv[1:]
keymap = os.path.expanduser(keymap)
engine = sa.create_engine(f'sqlite:///{keymap}')
meta = sa.MetaData(engine)
meta.reflect()
table = meta.tables[table]
with engine.connect() as conn:
    query = sa.select([table]).where(table.c.key == key)
    for row in conn.execute(query):
        val = msgpack.loads(row.value)
        print(f'{key} -> {val}')
EOF