application-research / delta-dm

Delta Large Dataset Manager
7 stars 3 forks source link

[Feature] CAR list import from Ptolemy #3

Open jcace opened 1 year ago

jcace commented 1 year ago

API Calls

Get list of Projects (datasets)

curl -X GET http://localhost:8000/v0/projects/

Returns

[
  {
    "project": "delta",
    "status": "completed containerization"
  }
]

Get metadata for project

curl -X GET http://localhost:8000/v0/carfile_meta/<project>

Returns

[
  {
    "payload_cid": "bafybeie3everb4274pg4yxtknsihsic5udxh3d7yji2y4q327vg5ncbpzu",
    "commp": "baga6ea4seaqot45k4kbeu2kgfgfra4zvh7duhmrwbzqkrzf35fxla23bpzulqmi",
    "size": 18148365267,
    "padded_size": 34359738368
  },
  {
    "payload_cid": "bafybeigskdpydrs6hhxttzks5jt3jdbsxgt2d7gs5sxpdfbptsx5cmeyvm",
    "commp": "baga6ea4seaqixnufqwo44jqrtjgmmzjfqs2qblv4sjat54m5ecyjmekmhzwaepy",
    "size": 17978311013,
    "padded_size": 34359738368
  },
  {
    "payload_cid": "bafybeib3v73bjgkhxiyihpxldxtgmbwfdqaab4tie5ie3vz526y3brlmyy",
    "commp": "baga6ea4seaqma6jdi5cyddnafqb3bet4reytsasz4qmv42kbntv3ymtllcnyogy",
    "size": 18153737882,
    "padded_size": 34359738368
  },
...
]

This format matches exactly what's expected by DDM /dataset/content API - so can be loaded directly in there