Closed Nicnl closed 10 months ago
BlackDex
commented
10 months ago It probably instructs the clients via headers that the stream is compressed, but either isn't, or isn't using the compression it reports it's using.
I haven't looked into it, but I'm afraid it's not something we can fix easily on our side if that is the case.
Nicnl
commented
10 months ago For some reason, the official bitwarden implementation/server seems to handle it just fine.
https://vault.bitwarden.com/icons/www.labanquepostale.fr/icon.png
I guess that the reqwest library behaves differently than what they use in their c# server.
I'm afraid it's not something we can fix easily on our side if that is the case.
I may have an idea.
When a DecompressError is produced, why not retry the request while forcing the Accept-Encoding : identity HTTP header?
That would disable compression altogether and effectively bypass the issue.
BlackDex
commented
10 months ago The servers are very different, so yea not comparable.
That might be an option, but as mentioned, not yet checked it out. It also never had been reported before, so this is a nice one too test it 😄
BlackDex
commented
10 months ago It could also be a bug in reqwest btw.
Nicnl
commented
10 months ago I'm not proficient with Rust specifically, but at least I did a GET using the reqwest library.
I'm probably not using the same lib version as vaultwarden.
Anyway: no issue with the request on my side.
I dug further.
The provided favicon is named favicon.ico, but the file actually contains a bitmap.
root@docker ~/icons # file favicon.ico
favicon.ico: PC bitmap, Windows 3.x format, 32 x 32 x 8
root@docker ~/icons # xxd favicon.ico | head -n 1
00000000: 424d 3608 0000 0000 0000 3604 0000 2800 BM6.......6...(.Meanwhile, the server answers with this content-type:
Content-Type: image/x-icon
I think that the decoding trusts either the extension or the Content-Type, which conflicts with the actual file content.
BlackDex
commented
10 months ago We always return *.png extensions, but the browser check the image headers/magic instead of trusting the extensions or the server headers returned.
The download is probably working just fine, but since we have gzip/brotli enabled, that might cause an issue.
BlackDex
commented
10 months ago @Nicnl I did some testing, but the problem is actually at the sites endpoint. For example, we sent gzip, br, deflate which indicates to prefer gzip but it ignores it.
Using curl to download the compressed data in deflate format, it returns an invalid file. I tested some others sites with the same technique and they all return valid deflate compressed data.
If i use gzip on that site it also works just fine.
We can do two things, leave it as-is, since the site is at fault here, and adding a check for this seems a bit to much in my opinion.
We can also remove the deflate feature, which also seems to be working. Looking at https://w3techs.com/technologies/details/ce-compression, it doesn't seem to be used that much at all.
My go would probably be the removal of the deflate feature.
Subject of the issue
The icon of a website isn't being downloaded.
Website: https://www.labanquepostale.fr/ (It's a French bank, so quite a large thing)
A favicon seems to exist: https://www.labanquepostale.fr/favicon.ico
Here is the logs I'm getting:
I see a
DecompressError, not sure if it's related to the HTTP stream or if it's some kind of decoding of the dot-ico file.Deployment environment
Docker with docker-compose.
Support string:
Your environment (Generated via diagnostics page)
Config (Generated via diagnostics page)
Show Running Config
**Environment settings which are overridden:** ADMIN_TOKEN ```json { "_duo_akey": null, "_enable_duo": true, "_enable_email_2fa": false, "_enable_smtp": true, "_enable_yubico": true, "_icon_service_csp": "", "_icon_service_url": "", "_ip_header_enabled": true, "_smtp_img_src": "cid:", "admin_ratelimit_max_burst": 3, "admin_ratelimit_seconds": 300, "admin_session_lifetime": 20, "admin_token": "***", "allowed_iframe_ancestors": "", "attachments_folder": "data/attachments", "auth_request_purge_schedule": "30 * * * * *", "authenticator_disable_time_drift": false, "data_folder": "data", "database_conn_init": "", "database_max_conns": 10, "database_timeout": 30, "database_url": "***************", "db_connection_retries": 15, "disable_2fa_remember": false, "disable_admin_token": false, "disable_icon_download": false, "domain": "*****://****************************************************", "domain_origin": "*****://*******************", "domain_path": "*********************************", "domain_set": true, "duo_host": null, "duo_ikey": null, "duo_skey": null, "email_attempts_limit": 3, "email_change_allowed": true, "email_expiration_time": 600, "email_token_size": 6, "emergency_access_allowed": true, "emergency_notification_reminder_schedule": "0 3 * * * *", "emergency_request_timeout_schedule": "0 7 * * * *", "enable_db_wal": true, "event_cleanup_schedule": "0 10 0 * * *", "events_days_retain": null, "extended_logging": true, "helo_name": null, "hibp_api_key": null, "icon_blacklist_non_global_ips": true, "icon_blacklist_regex": null, "icon_cache_folder": "data/icon_cache", "icon_cache_negttl": 259200, "icon_cache_ttl": 2592000, "icon_download_timeout": 10, "icon_redirect_code": 302, "icon_service": "internal", "incomplete_2fa_schedule": "30 * * * * *", "incomplete_2fa_time_limit": 3, "invitation_expiration_hours": 120, "invitation_org_name": "Vaultwarden", "invitations_allowed": true, "ip_header": "X-Real-IP", "job_poll_interval_ms": 30000, "log_file": null, "log_level": "Info", "log_timestamp_format": "%Y-%m-%d %H:%M:%S.%3f", "login_ratelimit_max_burst": 10, "login_ratelimit_seconds": 60, "org_attachment_limit": null, "org_creation_users": "", "org_events_enabled": false, "org_groups_enabled": false, "password_hints_allowed": true, "password_iterations": 600000, "push_enabled": false, "push_installation_id": "***", "push_installation_key": "***", "push_relay_uri": "https://push.bitwarden.com", "reload_templates": false, "require_device_email": false, "rsa_key_filename": "data/rsa_key", "send_purge_schedule": "0 5 * * * *", "sendmail_command": null, "sends_allowed": true, "sends_folder": "data/sends", "show_password_hint": false, "signups_allowed": false, "signups_domains_whitelist": "", "signups_verify": false, "signups_verify_resend_limit": 6, "signups_verify_resend_time": 3600, "smtp_accept_invalid_certs": false, "smtp_accept_invalid_hostnames": false, "smtp_auth_mechanism": null, "smtp_debug": false, "smtp_embed_images": true, "smtp_explicit_tls": null, "smtp_from": "", "smtp_from_name": "Vaultwarden", "smtp_host": null, "smtp_password": null, "smtp_port": 587, "smtp_security": "starttls", "smtp_ssl": null, "smtp_timeout": 15, "smtp_username": null, "templates_folder": "data/templates", "tmp_folder": "data/tmp", "trash_auto_delete_days": null, "trash_purge_schedule": "0 5 0 * * *", "use_sendmail": false, "use_syslog": false, "user_attachment_limit": null, "web_vault_enabled": true, "web_vault_folder": "web-vault/", "websocket_address": "0.0.0.0", "websocket_enabled": false, "websocket_port": 3012, "yubico_client_id": null, "yubico_secret_key": null, "yubico_server": null } ```Steps to reproduce
Try to download the icon of labanquepostale.fr through the icon API endpoint.
Expected behaviour
The icon of labanquepostale.fr (https://www.labanquepostale.fr/favicon.ico) should be successfully forwarded.
Actual behaviour
The console log produces a
DecompressErrorand the default icon is provided.