search

repository-service-tuf / repository-service-tuf-cli

Repository Service for TUF: Command Line Interface
https://pypi.org/project/repository-service-tuf/
MIT License
8 stars 15 forks source link

Task: CLI artifact add command `rstuf artifact add` #39

Open kairoaraujo opened 1 year ago

kairoaraujo commented 1 year ago

Implement to the rstuf CLI the functionality to add artifacts.

Two specific features to add targets:

Add Artifact (Feature)

This target adds the target using the RSTUF REST API. This feature is an interface to add targets using the command line.

1. Simple one artifact file

Note: the user can use the token (-t/--token) when API requires authentication/authorization, if the user uses rstuf --auth it ignores the token parameter.

Giving the artifact

rstuf artifact add <file> [-p/--path <file-path>, -c/--custom, -t/--token <token>]

Example: rstuf artifact add file-v1.0.3.tar.gz -p projectA/download/

It gets from the file the hash, size and add a custom path in the metadata projectA/download/file-v1.0.3.tar.gz

Without file, just the information

rstuf artifact add -p/--path [file-path], -l/lenght [length] -h/hashes [b2sum] -c/--custom

2. From JSON

rstuf artifact add -d @file.json

{
  "targets": [
    {
      "info": {
        "length": "int",
        "hashes": {
          "blake2b-256": "str"
        },
        "custom": {
          "key": "value"
        }
      },
      "path": "str"
    },
    {
      "info": {
        "length": "int",
        "hashes": {
          "blake2b-256": "str"
        },
        "custom": {
          "key": "value"
        }
      },
      "path": "str"
    },
  ]
}

It should return the task id.

Import targets (Feature)

This feature adds a huge amount of targets, importing from CSV directly to RSTUF Database The feature description is detailed in these links

kairoaraujo commented 1 year ago

CLI Implementation

The RSTUF user will be required to create a CSV file that contains the following information path;size;hash algorithm;hash; Example: test/optimistic_lamport-v1.0.tar.gz;12345;blake2b-256;716f6e863f744b9ac22c97ec7b76ea5f5908bc5b2f67c61510bfc4751384ea7a;

The CSV file limit will be half-million of targets (500,000 lines)

The CLI will implement as an admin command, and it will require some parameters

metadata-url: Address where the CLI can download the RSTUF TUF Metadata. i.e.: http://127.0.0.1:8080

db-uri: the RSTUF SQL DB URI. i.e.: postgresql://postgres:secret@127.0.0.1:5433 csv the CSV path. i.e targets-1of2.csv Example:

rstuf admin add-targets -metadata-url http://127.0.0.1:8080 -db-uri postgresql://postgres:secret@127.0.0.1:5433 -csv ../repository-service-tuf/tests/data/targets-1of2.csv The CLI will insert all the targets to the RSTUF SQL DB directly. Running in a Macbook Pro (2019) 2,4 GHz 8-Core Intel Core i9/32GB 2667 MHz DDR4

import time: ~15 seconds

kairoaraujo commented 1 year ago

Requires for implementation:

Add targets (Feature)

Import targets (Feature)

kairoaraujo commented 1 year ago

I updated the Issue description and the comment above.

MVrachev commented 1 year ago

I think we can close that one, am I right @kairoaraujo?

kairoaraujo commented 1 year ago

We added only part of this.

MVrachev commented 1 year ago

Sorry, but the comment you are referring to has three todo items and all of them are marked as complete. What do I miss?

kairoaraujo commented 1 year ago

Sorry, but the comment you are referring to has three todo items and all of them are marked as complete. What do I miss?

There are two feauteres there (subtitles) there: Add targets (Feature) and Import targets (Feature)

kairoaraujo commented 1 year ago

@KAUTH

I suggest breaking down this feature into three sub-tasks

So we can delivery small features and discuss more details in each.

KAUTH commented 1 year ago

@kairoaraujo good point! I was thinking of breaking it similarly as well.

I was also considering that giving file information can also be substituted with giving JSON file with information for only one file. Adding an actual file I see the immediate value, but do you think we need both JSON file and file info as well?

kairoaraujo commented 1 year ago

The JSON file is for adding a list of files, but it is a low priority.

KAUTH commented 1 year ago

The JSON file is for adding a list of files, but it is a low priority.

Yeah, my point is one file is also a list of one. So, do we want both JSON file and passing file information? In any case, I am working on "Implement rstuf artifact add giving a file. I will create tasks for each one and link them to this issue.