search

pulp / pulp_container

Pulp Container Registry
https://docs.pulpproject.org/pulp_container/
GNU General Public License v2.0
23 stars 44 forks source link

Char/text field Choices are not enforced/validated at DB level #1046

Closed ipanova closed 7 months ago

ipanova commented 2 years ago 
$ http https://PULP3-SOURCE-FEDORA36.puffy.example.com/pulp/api/v3/content/container/manifests/ --auth admin:password
HTTP/1.1 200 OK
Access-Control-Expose-Headers: Correlation-ID
Allow: GET, HEAD, OPTIONS
Connection: keep-alive
Content-Length: 722
Content-Type: application/json
Correlation-ID: 981cc643c6084edf9cf8950120187798
Date: Thu, 08 Sep 2022 14:47:26 GMT
Referrer-Policy: same-origin
Server: nginx
Vary: Accept, Cookie
X-Content-Type-Options: nosniff
X-Frame-Options: DENY

{
    "count": 1,
    "next": null,
    "previous": null,
    "results": [
        {
            "artifact": "/pulp/api/v3/artifacts/cf459685-dc38-435c-ab1c-07865b28b92d/",
            "blobs": [
                "/pulp/api/v3/content/container/blobs/ed537584-ab98-47d6-9b06-1fa366389ed8/",
                "/pulp/api/v3/content/container/blobs/6629a8b9-4ac8-4698-b10c-dc94e7d25f22/",
                "/pulp/api/v3/content/container/blobs/4c375c8b-5a49-4ad5-a226-b371ac2b6192/"
            ],
            "config_blob": null,
            "digest": "sha256:3bf3f08a5c4ca0b04ecdc0a95fa11530f86f36bd9d372a568254680e6332a2c0",
            "listed_manifests": [],
            "media_type": "application/vnd.docker.distribution.manifest.v1+prettyjws",
            "pulp_created": "2022-09-08T14:47:17.896191Z",
            "pulp_href": "/pulp/api/v3/content/container/manifests/fd2cae74-7fe1-4c8a-ab3c-00a06d6fdc55/",
            "schema_version": 1
        }
    ]
}

(pulp) [vagrant@pulp3-source-fedora36 ~]$ django-admin shell_plus
# Shell Plus Model Imports
from pulpcore.app.models.access_policy import AccessPolicy, Group
from django.contrib.auth.models import Permission, User
from django.contrib.contenttypes.models import ContentType
from django.contrib.sessions.models import Session
from pulp_ansible.app.models import AnsibleCollectionDeprecated, AnsibleDistribution, AnsibleRepository, Collection, CollectionImport, CollectionRemote, CollectionVersion, CollectionVersionSignature, GitRemote, RoleRemote
from pulpcore.app.models.role import GroupRole, Role, UserRole
from pulp_container.app.models import Blob, BlobManifest, ContainerDistribution, ContainerNamespace, ContainerPushRepository, ContainerRemote, ContainerRepository, Manifest, ManifestListManifest, ManifestSignature, ManifestSigningService, Tag
from pulpcore.app.models.upload import Upload, UploadChunk
from pulp_file.app.models import FileAlternateContentSource, FileContent, FileDistribution, FilePublication, FileRemote, FileRepository
from pulp_rpm.app.models.acs import RpmAlternateContentSource
from pulp_rpm.app.models.advisory import UpdateCollection, UpdateCollectionPackage, UpdateRecord, UpdateReference
from pulp_rpm.app.models.comps import PackageCategory, PackageEnvironment, PackageGroup, PackageLangpacks
from pulp_rpm.app.models.custom_metadata import RepoMetadataFile
from pulp_rpm.app.models.distribution import Addon, Checksum, DistributionTree, Image, Variant
from pulp_rpm.app.models.modulemd import Modulemd, ModulemdDefaults, ModulemdObsolete
from pulp_rpm.app.models.package import Package
from pulp_rpm.app.models.repository import RpmDistribution, RpmPublication, RpmRemote, RpmRepository, UlnRemote
from pulpcore.app.models.acs import AlternateContentSource, AlternateContentSourcePath
from pulpcore.app.models.base import Label
from pulpcore.app.models.content import Artifact, AsciiArmoredDetachedSigningService, Content, ContentArtifact, PulpTemporaryFile, RemoteArtifact, SigningService
from pulpcore.app.models.exporter import Export, ExportedResource, Exporter, FilesystemExport, FilesystemExporter, PulpExport, PulpExporter
from pulpcore.app.models.importer import Import, Importer, PulpImport, PulpImporter, PulpImporterRepository
from pulpcore.app.models.progress import GroupProgressReport, ProgressReport
from pulpcore.app.models.publication import BaseDistribution, ContentGuard, ContentRedirectContentGuard, Distribution, Publication, PublishedArtifact, PublishedMetadata, RBACContentGuard
from pulpcore.app.models.repository import Remote, Repository, RepositoryContent, RepositoryVersion, RepositoryVersionContentDetails
from pulpcore.app.models.status import ContentAppStatus
from pulpcore.app.models.task import CreatedResource, Task, TaskGroup, TaskSchedule, Worker
from pulpcore.app.models.telemetry import SystemID
# Shell Plus Django Imports
from django.core.cache import cache
from django.conf import settings
from django.contrib.auth import get_user_model
from django.db import transaction
from django.db.models import Avg, Case, Count, F, Max, Min, Prefetch, Q, Sum, When
from django.utils import timezone
from django.urls import reverse
from django.db.models import Exists, OuterRef, Subquery
Python 3.10.5 (main, Jun  9 2022, 00:00:00) [GCC 12.1.1 20220507 (Red Hat 12.1.1-1)]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.4.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: Manifest.objects.first()
Out[1]: <Manifest: pk=fd2cae74-7fe1-4c8a-ab3c-00a06d6fdc55>

In [3]: Manifest.objects.first().media_type
Out[3]: 'application/vnd.docker.distribution.manifest.v1+prettyjws'

In [4]: m=Manifest.objects.first()

In [5]: m
Out[5]: <Manifest: pk=fd2cae74-7fe1-4c8a-ab3c-00a06d6fdc55>

In [6]: m.media_type
Out[6]: 'application/vnd.docker.distribution.manifest.v1+prettyjws'

In [10]: m.media_type='application/vnd.docker.distribution.manifest.v5+prettyjws'

In [11]: m.save()

In [12]: m=Manifest.objects.first()

In [13]: m.media_type
Out[13]: 'application/vnd.docker.distribution.manifest.v5+prettyjws'

In [14]: 

In [14]: 

In [14]: m.media_type='blablabla'

In [15]: m.save()

In [16]: m=Manifest.objects.first()

In [17]: m.media_type
Out[17]: 'blablabla'

In [18]:                                                                                                                                                                                                                                      
Do you really want to exit ([y]/n)? ^[[A^[[B^C

In [18]:                                                                                                                                                                                                                                      
Do you really want to exit ([y]/n)? y
(pulp) [vagrant@pulp3-source-fedora36 ~]$ http https://PULP3-SOURCE-FEDORA36.puffy.example.com/pulp/api/v3/content/container/manifests/ --auth admin:password
HTTP/1.1 200 OK
Access-Control-Expose-Headers: Correlation-ID
Allow: GET, HEAD, OPTIONS
Connection: keep-alive
Content-Length: 674
Content-Type: application/json
Correlation-ID: 623478c99c6a4b28908be1e57d69205a
Date: Thu, 08 Sep 2022 14:57:08 GMT
Referrer-Policy: same-origin
Server: nginx
Vary: Accept, Cookie
X-Content-Type-Options: nosniff
X-Frame-Options: DENY

{
    "count": 1,
    "next": null,
    "previous": null,
    "results": [
        {
            "artifact": "/pulp/api/v3/artifacts/cf459685-dc38-435c-ab1c-07865b28b92d/",
            "blobs": [
                "/pulp/api/v3/content/container/blobs/ed537584-ab98-47d6-9b06-1fa366389ed8/",
                "/pulp/api/v3/content/container/blobs/6629a8b9-4ac8-4698-b10c-dc94e7d25f22/",
                "/pulp/api/v3/content/container/blobs/4c375c8b-5a49-4ad5-a226-b371ac2b6192/"
            ],
            "config_blob": null,
            "digest": "sha256:3bf3f08a5c4ca0b04ecdc0a95fa11530f86f36bd9d372a568254680e6332a2c0",
            "listed_manifests": [],
            "media_type": "blablabla",
            "pulp_created": "2022-09-08T14:47:17.896191Z",
            "pulp_href": "/pulp/api/v3/content/container/manifests/fd2cae74-7fe1-4c8a-ab3c-00a06d6fdc55/",
            "schema_version": 1
        }
    ]
}

https://github.com/pulp/pulp_container/blob/main/pulp_container/app/models.py#L84

ipanova commented 2 years ago

https://adamj.eu/tech/2020/01/22/djangos-field-choices-dont-constrain-your-data/

lubosmj commented 1 year ago

This was supposedly fixed in django 4.0: https://code.djangoproject.com/ticket/30581.

lubosmj commented 1 year ago

@ipanova, can you verify whether this bug still persists in django 4.2?

lubosmj commented 1 year ago

The issue still persists:

In [1]: m=Manifest.objects.first()

In [2]: m.media_type
Out[2]: 'application/vnd.docker.distribution.manifest.list.v2+json'

In [3]: m.media_type = "lalala"

In [4]: m.save()

In [5]:                                                                                                               
Do you really want to exit ([y]/n)? 
[root@751098a6e335 /]# exit
[lmjachky@localhost oci_env]$ http :5001/pulp/api/v3/content/container/manifests/
HTTP/1.1 200 OK
Access-Control-Expose-Headers: Correlation-ID
Allow: GET, HEAD, OPTIONS
Connection: keep-alive
Content-Length: 9079
Content-Type: application/json
Correlation-ID: cd05143a86ed4203a5a6273704d93e8e
Cross-Origin-Opener-Policy: same-origin
Date: Thu, 20 Jul 2023 10:07:36 GMT
Referrer-Policy: same-origin
Server: nginx/1.14.1
Vary: Accept
X-Content-Type-Options: nosniff
X-Frame-Options: DENY

{
    "count": 13,
    "next": null,
    "previous": null,
    "results": [
        {
            "artifact": "/pulp/api/v3/artifacts/018972c3-0d65-7142-af1d-3ade0e0cc29e/",
            "blobs": [],
            "config_blob": null,
            "digest": "sha256:2eeacd9391c1380f8613788b185d2196702f3042c30b8d74aa1ce26b24ed60b1",
            "listed_manifests": [
                "/pulp/api/v3/content/container/manifests/018972c3-0eb5-7e7e-84d1-b3568b18dbec/",
                "/pulp/api/v3/content/container/manifests/018972c3-0ed4-7312-bfdb-3fa8572f0dfb/",
                "/pulp/api/v3/content/container/manifests/018972c3-0edb-7bd9-8824-557d696626af/",
                "/pulp/api/v3/content/container/manifests/018972c3-0f43-7617-b7b3-7eee18678952/",
                "/pulp/api/v3/content/container/manifests/018972c3-0ecb-7d0a-876a-4de9d465a72e/",
                "/pulp/api/v3/content/container/manifests/018972c3-0e69-7309-8105-60d522784135/",
                "/pulp/api/v3/content/container/manifests/018972c3-0ebf-776d-b3b5-9c689cc67d49/",
                "/pulp/api/v3/content/container/manifests/018972c3-0ec6-73ea-97c5-df14df7ae900/",
                "/pulp/api/v3/content/container/manifests/018972c3-0e9c-71c6-b96d-1ea560c1c116/"
            ],
            "media_type": "application/vnd.docker.distribution.manifest.list.v2+json",
            "pulp_created": "2023-07-20T10:05:35.726457Z",
            "pulp_href": "/pulp/api/v3/content/container/manifests/018972c3-0f75-7cb7-8b45-ab14141e677a/",
            "schema_version": 2
        },
        {
            "artifact": "/pulp/api/v3/artifacts/018972c3-0d56-7849-a30c-76bed9b1a4c1/",
            "blobs": [],
            "config_blob": null,
            "digest": "sha256:8a2bd13f5d1cbc36c2a0a5ee2fdc544b5320c2393768a3e9b96b476c8faeab68",
            "listed_manifests": [
                "/pulp/api/v3/content/container/manifests/018972c3-0e69-7309-8105-60d522784135/",
                "/pulp/api/v3/content/container/manifests/018972c3-0e87-7110-9b12-9f8c6336c484/",
                "/pulp/api/v3/content/container/manifests/018972c3-0e9c-71c6-b96d-1ea560c1c116/",
                "/pulp/api/v3/content/container/manifests/018972c3-0ea4-7a44-9283-8fe37fb6ae7d/",
                "/pulp/api/v3/content/container/manifests/018972c3-0eb5-7e7e-84d1-b3568b18dbec/",
                "/pulp/api/v3/content/container/manifests/018972c3-0ebf-776d-b3b5-9c689cc67d49/",
                "/pulp/api/v3/content/container/manifests/018972c3-0ec6-73ea-97c5-df14df7ae900/",
                "/pulp/api/v3/content/container/manifests/018972c3-0ecb-7d0a-876a-4de9d465a72e/",
                "/pulp/api/v3/content/container/manifests/018972c3-0ed4-7312-bfdb-3fa8572f0dfb/",
                "/pulp/api/v3/content/container/manifests/018972c3-0edb-7bd9-8824-557d696626af/",
                "/pulp/api/v3/content/container/manifests/018972c3-0f43-7617-b7b3-7eee18678952/"
            ],
            "media_type": "lalala",         <---------------------------- HERE
            "pulp_created": "2023-07-20T10:05:35.685490Z",
            "pulp_href": "/pulp/api/v3/content/container/manifests/018972c3-0daa-7f97-995f-5259923c3f3d/",
            "schema_version": 2
        },
MichalPysik commented 7 months ago

A fix to this issue was proposed in a PR: https://github.com/pulp/pulp_container/pull/1527 , but in the end, we decided to close the issue in favor of point 4 here: https://github.com/pulp/pulp_container/issues/1494 . There is no point in DB-level validation if we plan to disable the app-level validation for media_type.