brimdata / super

A novel data lake based on super-structured data
https://zed.brimdata.io/
BSD 3-Clause "New" or "Revised" License
1.39k stars 64 forks source link

zqd support for space ids #686

Closed alfred-landrum closed 4 years ago

alfred-landrum commented 4 years ago

Add support for space ids, uuid-like immutable identifiers that will be used instead of space names as the resource id in api calls. The id should be created by zqd when the space is returned, and passed back in the create response struct.

To make the transition easy from the existing name-based calls, I suggest that we continue to treat the directory name under the spaces directory as the space's id. So before this work goes in, if there was a directory spaces/foo, because space foo was created, we treat foo as the id for that space going forward. After this work goes in, though, when a new space is created with name bar, the fs would look like: space/ae123428d33.../ space/ae123428d33.../config.json

where config.json now has an entry name: "bar".

philrz commented 4 years ago

Verified in Brim commit 22604c1 talking to zqd commit f9a90eb.

I triggered the creation of a couple Spaces, one via pcap import and one via Zeek log import. I can see the opaque Space names now below my data/ directory.

~/Library/Application Support/Brim/data/spaces$ tree
.
├── sp_1c8HwI2VOUQWlgvIoELq51XrFZG
│   ├── all.zng
│   ├── config.json
│   ├── info.json
│   └── packets.idx.json
└── sp_1c8Hz0OM81fitoUQagFvW0uLKrk
    ├── all.zng
    ├── config.json
    └── info.json

2 directories, 7 files

Thanks @mattnibs!