Open carlosthe19916 opened 1 month ago
The thing is, that each package has a Vec<Purl>
. We can't provide what we don't have. So we (UX) need to come up with a smart way to deal with this data in the UI.
@ctron V1 certainly was doing it, why V1 was doing it but V2 cannot? If v2 can not render the same thing it represents a step back.
The thing is, that each package has a Vec
It seems what the current backend is returning is an Array of Packages where each Package contains an array of Purls.
I think I am asking to change the response from:
Package {
purls: Vec<String>
}
Response = Vec<Package>
To something like:
let packages: Vec<Package> = ...
let result = packages.flatMap(package => package.purls);
Response = result
where Response is Vec<string> and "string" is a purl that the any client can take and render
So we remove the Package wrapper and only return a set of Purls. Using any programming language a flatMap
iterator would do the magic.
For this specific case I do not think we should accommodate the UI to the models (accommodate the requirements to the limitations of the rest models) , but the other way around: accommodate the rest models to the requirements given (v1 UI parity). If that is not the case let's discuss it in the UX meeting and ask PM's approval
There's a reason we have v2. Becasue we made mistakes in v1.
An SBOM has: SBOM -[1..]-> SBOM Packages -[1..]-> PURLs
And if you take a look at the code for v1, you'll find the same handling: https://github.com/trustification/trustification/blob/main/spog/ui/crates/components/src/spdx/packages.rs
An SBOM has: SBOM -[1..]-> SBOM Packages -[1..]-> PURLs
I fully understand this concept.
What I strongly disagree is to force/limit the UI to render data strictly in the same way the Models are defined (SBOM -[1..]-> SBOM Packages -[1..]-> PURLs)
Have you noticed that for a single SBOM the relationship is always:
1 SBOM => SBOM Packages [1..] => ONLY 1 PURL
From the UX point of view it makes sense to flat an array of array because we know that for a single SBOM there will be only 1 purl per package (in the worst case perhaps 2 but I never saw a single SBOM that does it).
@ctron Here is an idea:
Have you noticed that for a single SBOM the relationship is always:
I didn't. How do you come to this conclusion?
Have you noticed that for a single SBOM the relationship is always:
I didn't. How do you come to this conclusion?
I pasted the response from in the issue description:
{
"items": [
{
"id": "SPDXRef-abb48860-ddc0-43e6-998b-1895403ad55a",
"name": "accordion",
"version": "24.0.5",
"purl": [
{
"uuid": "1e4fb8f9-f6a6-5236-922e-ca02d42f6c72",
"purl": "pkg://maven/org.mvnpm.at.vaadin/accordion@24.0.5?type=jar",
"base": {
"uuid": "4595015b-69e9-5d04-9d27-03a778500f70",
"purl": "pkg://maven/org.mvnpm.at.vaadin/accordion"
},
"version": {
"uuid": "eb5557c3-90fb-55f6-89da-5793e17e854f",
"purl": "pkg://maven/org.mvnpm.at.vaadin/accordion@24.0.5",
"version": "24.0.5"
},
"qualifiers": {
"type": "jar"
}
}
],
"cpe": []
},
{
"id": "SPDXRef-00611213-b79e-42e0-98a9-a318318bf9c5",
"name": "aesh",
"version": "2.7.0.redhat-00002",
"purl": [
{
"uuid": "7123f24d-1d66-5c98-bbf5-714ebb31ad8b",
"purl": "pkg://maven/org.aesh/aesh@2.7.0.redhat-00002?repository_url=https://maven.repository.redhat.com/ga/&type=jar",
"base": {
"uuid": "fd40e5fc-5175-5937-b5e2-a7298b2b1707",
"purl": "pkg://maven/org.aesh/aesh"
},
"version": {
"uuid": "0fb64aae-3112-56c8-a475-87ff77fc0c65",
"purl": "pkg://maven/org.aesh/aesh@2.7.0.redhat-00002",
"version": "2.7.0.redhat-00002"
},
"qualifiers": {
"repository_url": "https://maven.repository.redhat.com/ga/",
"type": "jar"
}
}
],
"cpe": []
},
{
"id": "SPDXRef-ebfb58d6-1de4-4c47-a7c1-79d29339c3a8",
"name": "agroal-api",
"version": "2.1.0.redhat-00001",
"purl": [
{
"uuid": "94f32f08-b60f-583f-b7f0-1efeb99bbfae",
"purl": "pkg://maven/io.agroal/agroal-api@2.1.0.redhat-00001?repository_url=https://maven.repository.redhat.com/ga/&type=jar",
"base": {
"uuid": "53934f39-ce95-5ff3-bd86-3d01d1d7ad84",
"purl": "pkg://maven/io.agroal/agroal-api"
},
"version": {
"uuid": "d4c693c3-51e7-558f-95e9-7c8cd507a9fc",
"purl": "pkg://maven/io.agroal/agroal-api@2.1.0.redhat-00001",
"version": "2.1.0.redhat-00001"
},
"qualifiers": {
"repository_url": "https://maven.repository.redhat.com/ga/",
"type": "jar"
}
}
],
"cpe": []
},
{
"id": "SPDXRef-e221f35d-8bb3-4147-b150-3c5a2bac3fb2",
"name": "agroal-narayana",
"version": "2.1.0.redhat-00001",
"purl": [
{
"uuid": "1c0921dc-57d8-57a5-9e9f-4284bd9c064d",
"purl": "pkg://maven/io.agroal/agroal-narayana@2.1.0.redhat-00001?repository_url=https://maven.repository.redhat.com/ga/&type=jar",
"base": {
"uuid": "55632701-6122-5f60-88de-9d568e39e5bc",
"purl": "pkg://maven/io.agroal/agroal-narayana"
},
"version": {
"uuid": "2fd704f7-461e-5a0a-b480-07167e91138e",
"purl": "pkg://maven/io.agroal/agroal-narayana@2.1.0.redhat-00001",
"version": "2.1.0.redhat-00001"
},
"qualifiers": {
"repository_url": "https://maven.repository.redhat.com/ga/",
"type": "jar"
}
}
],
"cpe": []
},
{
"id": "SPDXRef-2fa0a703-740e-4a6f-9d2f-013e76ec84f6",
"name": "agroal-pool",
"version": "2.1.0.redhat-00001",
"purl": [
{
"uuid": "f1b7e5db-ace1-5b5e-8787-e4b6bf041c73",
"purl": "pkg://maven/io.agroal/agroal-pool@2.1.0.redhat-00001?repository_url=https://maven.repository.redhat.com/ga/&type=jar",
"base": {
"uuid": "cde2f11e-04f7-5125-9966-92e4dfbcbc38",
"purl": "pkg://maven/io.agroal/agroal-pool"
},
"version": {
"uuid": "2ab5051c-fd98-5fc2-8a29-f0ff455d38d0",
"purl": "pkg://maven/io.agroal/agroal-pool@2.1.0.redhat-00001",
"version": "2.1.0.redhat-00001"
},
"qualifiers": {
"repository_url": "https://maven.repository.redhat.com/ga/",
"type": "jar"
}
}
],
"cpe": []
},
{
"id": "SPDXRef-20272492-e735-4aa8-9da6-ee55879de5a4",
"name": "angus-activation",
"version": "2.0.1.redhat-00005",
"purl": [
{
"uuid": "71706104-4358-5dce-98e2-3c51d6457adf",
"purl": "pkg://maven/org.eclipse.angus/angus-activation@2.0.1.redhat-00005?repository_url=https://maven.repository.redhat.com/ga/&type=jar",
"base": {
"uuid": "ab4395d1-e570-5fb2-8c2f-bd617e5aff4b",
"purl": "pkg://maven/org.eclipse.angus/angus-activation"
},
"version": {
"uuid": "6cc9dd7d-cf66-54a2-b3f5-f9520e34d531",
"purl": "pkg://maven/org.eclipse.angus/angus-activation@2.0.1.redhat-00005",
"version": "2.0.1.redhat-00005"
},
"qualifiers": {
"repository_url": "https://maven.repository.redhat.com/ga/",
"type": "jar"
}
}
],
"cpe": []
},
{
"id": "SPDXRef-fd26f50a-295e-47a6-856d-49d930cf45c1",
"name": "angus-mail",
"version": "1.0.0.redhat-00002",
"purl": [
{
"uuid": "c6b499d9-f26c-5586-8fd8-54a070ccb140",
"purl": "pkg://maven/org.eclipse.angus/angus-mail@1.0.0.redhat-00002?repository_url=https://maven.repository.redhat.com/ga/&type=jar",
"base": {
"uuid": "58860197-52d3-55fa-91f6-306596fdc843",
"purl": "pkg://maven/org.eclipse.angus/angus-mail"
},
"version": {
"uuid": "dfc15f14-1edb-579d-9eaf-c306db1416cc",
"purl": "pkg://maven/org.eclipse.angus/angus-mail@1.0.0.redhat-00002",
"version": "1.0.0.redhat-00002"
},
"qualifiers": {
"repository_url": "https://maven.repository.redhat.com/ga/",
"type": "jar"
}
}
],
"cpe": []
},
{
"id": "SPDXRef-242e33c5-d7e6-4e70-a9bf-339ba35f299a",
"name": "annotations",
"version": "24.0.1",
"purl": [
{
"uuid": "c6706f4d-1227-53a1-9f15-b9f0df466cab",
"purl": "pkg://maven/org.jetbrains/annotations@24.0.1?type=jar",
"base": {
"uuid": "874a6312-844b-52cf-93ae-d5d05b6314a6",
"purl": "pkg://maven/org.jetbrains/annotations"
},
"version": {
"uuid": "157b4be1-eda6-5943-ad11-40a2803f8caf",
"purl": "pkg://maven/org.jetbrains/annotations@24.0.1",
"version": "24.0.1"
},
"qualifiers": {
"type": "jar"
}
}
],
"cpe": []
},
{
"id": "SPDXRef-12dd6252-9da5-449a-9f57-c57aa262a52d",
"name": "antlr4-runtime",
"version": "4.10.1.redhat-00001",
"purl": [
{
"uuid": "e296ef0d-5fc5-508f-907d-16704c5dc3c2",
"purl": "pkg://maven/org.antlr/antlr4-runtime@4.10.1.redhat-00001?repository_url=https://maven.repository.redhat.com/ga/&type=jar",
"base": {
"uuid": "14c791c8-12aa-54f9-be1a-8e03e3e9a540",
"purl": "pkg://maven/org.antlr/antlr4-runtime"
},
"version": {
"uuid": "c6c547ce-7e85-5c07-ae03-241f7474232b",
"purl": "pkg://maven/org.antlr/antlr4-runtime@4.10.1.redhat-00001",
"version": "4.10.1.redhat-00001"
},
"qualifiers": {
"repository_url": "https://maven.repository.redhat.com/ga/",
"type": "jar"
}
}
],
"cpe": []
},
{
"id": "SPDXRef-d5e9e485-a895-42a8-9d33-9419d5496ce7",
"name": "aopalliance",
"version": "1.0",
"purl": [
{
"uuid": "ddd0e340-18b0-519b-b9ca-9127c9576f59",
"purl": "pkg://maven/aopalliance/aopalliance@1.0?type=jar",
"base": {
"uuid": "be4bb357-7956-5f7c-80e0-e6eb057ec1eb",
"purl": "pkg://maven/aopalliance/aopalliance"
},
"version": {
"uuid": "f0a4e83a-177a-5c0e-adf7-3f9fd86ab354",
"purl": "pkg://maven/aopalliance/aopalliance@1.0",
"version": "1.0"
},
"qualifiers": {
"type": "jar"
}
}
],
"cpe": []
}
],
"total": 1053
}
Taking my previous JSON and looking at items.purl.length
is always ONE. I never saw a single SBOM where items.purl.length > ONE .
That is why I am suggesting to allow the UI to get ALL the packages (backends does not handle pagination) and let the client manipulate the data in its best interest without being limited by the pagination done in the backend.
You also don't see any entry in the CPE array. And yet, we have them. So again, multiple PURLs are possible. V1 deals with them, V2 should too.
You also don't see any entry in the CPE array. And yet, we have them. So again, multiple PURLs are possible. V1 deals with them, V2 should too.
Sorry I didn't get this point. Where in V1 are we showing CPEs?
If you are suggesting to also render CPEs then that as far as I know is out of scope for the next release. Further improvement can be done after it but not now.
@ctron Sorry to insist, but what is your answer to my suggestion about: "returning ALL Packages"? If the backend cannot accommodate its response to the UI needs then could it at least not push its constraints to the UI?
You also don't see any entry in the CPE array. And yet, we have them. So again, multiple PURLs are possible. V1 deals with them, V2 should too.
Sorry I didn't get this point. Where in V1 are we showing CPEs?
I was talking about PURLs. The point about CPEs was simply, that the response you provided also doesn't list any actual CPEs. And yet, they do exist and need to visualized.
@ctron Sorry to insist, but what is your answer to my suggestion about: "returning ALL Packages"? If the backend cannot accommodate its response to the UI needs then could it at least not push its constraints to the UI?
I am sorry, I don't understand what you mean by "all packages". Are there packages you're missing?
@ctron Sorry to insist, but what is your answer to my suggestion about: "returning ALL Packages"? If the backend cannot accommodate its response to the UI needs then could it at least not push its constraints to the UI?
I am sorry, I don't understand what you mean by "all packages". Are there packages you're missing?
@ctron the current way of getting packages for an sbom is:
"/api/v1/sbom/{id}/packages?limit=10&offset=0"
Which would be the previous rest query to fetch all packages in one single shoot? The backend uses pagination now right? Perhaps it's possible already; if so, How the previous rest query should change?
I don't understand why you would want to get all packages in a single call? That can be quite an amount.
I don't understand why you would want to get all packages in a single call? That can be quite an amount.
@ctron the fact that the backend handles pagination, sorting, filtering, forces the client/UI to be limited to render data in the same structure the rest endpoints are defined. In this case, it's clear that the current backend response forces the UI not to be able to render data in the same way v1 was doing because Purls are embedded inside other arrays ( I think we agreed that the current response returns packages that have VEC
On the other hand, if only the backend would return all packages then the UI could successfully be able to deliver what it was asked (the v1 way of rendering packages)
As I mentioned before, even if the packages were thousands it should not make the UI slow.
Then I'd say you just the set the limit to u32::MAX
.
I do not see why could not make endpoint that just returns top level purl (and not expose the full glory of base, versioned and qualified purls) ... for the main use cases it seems the most obvious and if we ever need to access the other stuff we can add a flag or endpoint to do so.
Wanted to make a few other observations:
We do pagination to protect the server from someone 'calling everything' ... this is a constant battle in web ux vs backend eg. handling multiple calls slightly complexifies client calling code ... eg. instead of 1 http call #n calls to get 'work done'. Doing filtering/sorting in the database is a good optimisation, relieving client code to have to handle.
This is how most (if not all) Red Hat internal services do it (ex. BZ, errata, etc etc) ... and for good reason .. otherwise client code would quickly topple over those services - in some rare instances one might provide hyper specific lightweight endpoints (for example to support 'autocorrection') but the trade off between server operation and slightly more complicated client code is a well known one.
If v1 allowed one to pull down everything then I would say that was a mistake. Of course that statement needs to be backed by evidence. It might be we never have enough data to worry about such calls, etc.
I do not see why could not make endpoint that just returns top level purl (and not expose the full glory of base, versioned and qualified purls) […]
I think that would be a real improvement. However, that would still result in a Vec<Purl>
for each SBOM package. As far as I understood (maybe I got that wrong) the request was to return exactly one per SBOM package.
If v1 allowed one to pull down everything then I would say that was a mistake. Of course that statement needs to be backed by evidence. It might be we never have enough data to worry about such calls, etc.
I think it was. But there was no alternative at that point, as there was no real backend to pick data from. It was all or nothing in that case.
Then I'd say you just the set the limit to
u32::MAX
.
Not relevant to the discussion, but I think 0
works, too.
Then I'd say you just the set the limit to
u32::MAX
.Not relevant to the discussion, but I think
0
works, too.
I'll try 0
or u32::MAX
.
We do pagination to protect the server from someone 'calling everything'
Yeah but if the current response of the backend blocks a requirement done for the UI then I need a workaround. If the backend won't give the UI what it needs, at least it should not block it.
We do pagination to protect the server from someone 'calling everything'
Yeah but if the current response of the backend blocks a requirement done for the UI then I need a workaround. If the backend won't give the UI what it needs, at least it should not block it.
Sorry, but what's blocking you that disabling paging can fix?
We do pagination to protect the server from someone 'calling everything'
Yeah but if the current response of the backend blocks a requirement done for the UI then I need a workaround. If the backend won't give the UI what it needs, at least it should not block it.
Sorry, but what's blocking you that disabling paging can fix?
// Taking into account the JSON I pasted in the description of this issue
const purls: Array<Purl> = items.flatMap(e => e.purl)
and I get all the purls ready to be rendered in the table just as the requirement describes. Then the UI can handle pagination/filtering/sorting on the client side.
And you can't apply that logic on a page of data that got returned?
And you can't apply that logic on a page of data that got returned?
He could, but the things on the 2nd page won't necessarily be sorted relative to the 1st, iiuc. I think the essential problem is that the backend doesn't yet support the sorting the front-end requires.
And you can't apply that logic on a page of data that got returned?
He could, but the things on the 2nd page won't necessarily be sorted relative to the 1st, iiuc. I think the essential problem is that the backend doesn't yet support the sorting the front-end requires.
Ah, so sorting is another issue. Maybe it makes sense to create individual issues.
Sorting, also filtering, and even pagination. As there is no way for the client to know the total count of purls the UI table has unless the client has ALL data fetched.
Filtering, pagination and sorting come all together. It's all done either in the backend or in the client side. In this case as I explained before unless we change the response of the backend I'll need to do that in the client side which should be perfectly fine from the UI point of view
ah sorting ... right ... well we need to fix that (thx @jcrossley3 for teasing that out) btw I think our rest endpoints do return total counts ... eg. a client can know total number.
ah sorting ... right ... well we need to fix that (thx @jcrossley3 for teasing that out) btw I think our rest endpoints do return total counts ... eg. a client can know total number.
Not true for the client side, in the context of the intended UI table to be created. The UI needs a table of purls.
The current endpoint does return a total count but it's a total count of packages not purls. A package contains an array of purls.
Same for filtering... Well it's late here. Tomorrow I'll go for setting the total to Zero as Jim suggested or setting it to MAX as Jens suggested.
@carlosthe19916 I see ... setting limit to return all is not ideal eg. it might be we have to create a different endpoint to satisfy the requirement (which provides purl centric view).
@carlosthe19916 I see ... setting limit to return all is not ideal eg. it might be we have to create a different endpoint to satisfy the requirement (which provides purl centric view).
I know... Guys, I need us all to be on the same page in regards on what we will deliver in the next release. The requirement, although vague, is "V1 UI Parity" and we all need work towards that goal.
I think you should stop on focusing on the idea of "purls". packages are not purls. purls identify packages. there may be multiple identifiers for a package. thus, multiple purls for the same package. Don't design a thing that doesn't match the actual structure of things.
The following screenshot is what we INTENT TO HAVE in V2 UI IN LIGHT OF THE V1 vs V2 UI Compatibility.
And the following image is what we used to have in V1
This is what I request?
Currently the endpoint that returns the packages of an SBOM is:
And the response is something like:
I don't see how I can take that response and create a table with a single Row that has the column
type
. The reason is that thetype
field can be extracted from theitems[0].purl[0].purl
field of the response. The purl is within an array (an array of arrays).So rather than the current response:
We need to have something like:
Also we will need to add sorting to all the columns of the proposed table and I don't see how I will do SORT BY VERSION with the current rest endpoint.
The image below is how we used to render data in the UI; you see an expanded version for each row (each expanded version with only a single purl)