Open kairoaraujo opened 1 year ago
rstuf admin add-target
option to import directly to DB. It is related to https://github.com/vmware/repository-service-tuf/issues/188CLI Implementation
The RSTUF user will be required to create a CSV file that contains the following information path;size;hash algorithm;hash; Example: test/optimistic_lamport-v1.0.tar.gz;12345;blake2b-256;716f6e863f744b9ac22c97ec7b76ea5f5908bc5b2f67c61510bfc4751384ea7a;
The CSV file limit will be half-million of targets (500,000 lines)
The CLI will implement as an admin command, and it will require some parameters
metadata-url: Address where the CLI can download the RSTUF TUF Metadata. i.e.: http://127.0.0.1:8080
db-uri: the RSTUF SQL DB URI. i.e.: postgresql://postgres:secret@127.0.0.1:5433 csv the CSV path. i.e targets-1of2.csv Example:
rstuf admin add-targets -metadata-url http://127.0.0.1:8080 -db-uri postgresql://postgres:secret@127.0.0.1:5433 -csv ../repository-service-tuf/tests/data/targets-1of2.csv The CLI will insert all the targets to the RSTUF SQL DB directly. Running in a Macbook Pro (2019) 2,4 GHz 8-Core Intel Core i9/32GB 2667 MHz DDR4
import time: ~15 seconds
Requires for implementation:
I updated the Issue description and the comment above.
I think we can close that one, am I right @kairoaraujo?
We added only part of this.
Sorry, but the comment you are referring to has three todo items and all of them are marked as complete. What do I miss?
Sorry, but the comment you are referring to has three todo items and all of them are marked as complete. What do I miss?
There are two feauteres there (subtitles) there: Add targets (Feature) and Import targets (Feature)
@KAUTH
I suggest breaking down this feature into three sub-tasks
rstuf artifact add
giving a file (#343)rstuf artifact add
giving file information (#402)rstuf artifact add
giving JSON fileSo we can delivery small features and discuss more details in each.
@kairoaraujo good point! I was thinking of breaking it similarly as well.
I was also considering that giving file information can also be substituted with giving JSON file with information for only one file. Adding an actual file I see the immediate value, but do you think we need both JSON file and file info as well?
The JSON file is for adding a list of files, but it is a low priority.
The JSON file is for adding a list of files, but it is a low priority.
Yeah, my point is one file is also a list of one. So, do we want both JSON file and passing file information?
In any case, I am working on "Implement rstuf artifact add
giving a file. I will create tasks for each one and link them to this issue.
Implement to the rstuf CLI the functionality to add artifacts.
Two specific features to add targets:
Add Artifact (Feature)
This target adds the target using the RSTUF REST API. This feature is an interface to add targets using the command line.
1. Simple one artifact file
Note: the user can use the token (
-t/--token
) when API requires authentication/authorization, if the user usesrstuf --auth
it ignores the token parameter.Giving the artifact
rstuf artifact add <file> [-p/--path <file-path>, -c/--custom, -t/--token <token>]
Example:
rstuf artifact add file-v1.0.3.tar.gz -p projectA/download/
It gets from the file the hash, size and add a custom path in the metadata
projectA/download/file-v1.0.3.tar.gz
Without file, just the information
rstuf artifact add -p/--path [file-path], -l/lenght [length] -h/hashes [b2sum] -c/--custom
2. From JSON
rstuf artifact add -d @file.json
It should return the task id.
Import targets (Feature)
This feature adds a huge amount of targets, importing from CSV directly to RSTUF Database The feature description is detailed in these links